Robot, power management method, device and program product thereof

Abstract

The application relates to the field of robots, in particular to a robot and a power management method, device and program product thereof. The method comprises: acquiring first state data of the robot at a current time and a work task to be performed at the current time; inputting the first state data into a reinforcement learning model, and predicting rewards corresponding to the robot performing the work task according to different power management strategies through the reinforcement learning model; and selecting a first power management strategy corresponding to a maximum first reward according to the rewards corresponding to the different power management strategies, and performing power management on the robot. Through the selection of the first power management strategy corresponding to the maximum first reward for power management, the robot can more efficiently complete the task, battery charging and replacement are reduced, and the reliability and stability of the robot are improved.

Description

Technical Field

[0001] This application relates to the field of robotics, and in particular to robots and their power management methods, devices and software products. Background Technology

[0002] With the rapid development of artificial intelligence and robotics, humanoid robots have been widely used in various fields, including home, healthcare, service, education, automotive, and logistics. These applications demonstrate the importance and potential of humanoid robots in modern society. However, existing humanoid robots are often limited by battery life, which not only affects the user experience but also restricts their application in certain key areas.

[0003] Current energy management methods mostly employ simple power monitoring and management algorithms. The shortcomings of this energy management not only limit the robot's endurance but also increase maintenance costs, such as requiring frequent charging and battery replacements. Furthermore, it may affect the robot's reliability and stability, for example, insufficient power may lead to task interruption or failure. Summary of the Invention

[0004] In view of this, embodiments of this application provide a robot and its power management method, device and program product to solve the problem that the energy management schemes in the prior art limit the robot's endurance, increase the operation and maintenance costs, and affect the robot's reliability and stability.

[0005] A first aspect of this application provides a method for managing the power of a robot, the method comprising:

[0006] Obtain the robot's first state data at the current time and the work task to be performed at the current time;

[0007] The first state data is input into a reinforcement learning model, and the reinforcement learning model predicts the reward corresponding to the robot when it performs the work task according to different power management strategies.

[0008] Based on the rewards corresponding to different power management strategies, the first power management strategy corresponding to the highest first reward is selected to manage the power of the robot.

[0009] In conjunction with the first aspect, in a first possible implementation of the first aspect, before inputting the first state data into a reinforcement learning model and predicting the reward corresponding to the robot executing different power management strategies through the reinforcement learning model, the method further includes:

[0010] Initialize the robot's second state data and initialize the reward table, which includes the rewards obtained by executing different power management strategies in different states;

[0011] Based on the second state data, a second power management strategy is determined at the time of the second state data through a predetermined exploration strategy;

[0012] The system obtains the second reward and third state data after executing the second power management strategy under the second state data. It updates the second state data with the third state data, updates the reward table with the second reward, and then jumps to execute the step of determining the second power management strategy under the second state data through a predetermined exploration strategy based on the second state data, until the reward table converges or the number of learning times reaches a predetermined number, thus completing the training of the reinforcement learning model.

[0013] In conjunction with the first possible implementation of the first aspect, in the second possible implementation of the first aspect, obtaining the second reward after executing the second power management strategy under the state of the second state data includes:

[0014] After the robot executes the second power management strategy under the state of the second state data, it obtains at least one of the task completion efficiency, energy consumption and battery health indicators.

[0015] The second reward after implementing the second power management strategy is determined by at least one of the task completion efficiency, the energy consumption, and the battery health indicators.

[0016] In conjunction with the second possible implementation of the first aspect, in the third possible implementation of the first aspect, before obtaining the second reward after executing the second power management strategy in the state of the second state data, the method further includes:

[0017] Calibrate energy consumption models for different tasks executing different power management strategies under different environments;

[0018] After the robot executes the second power management strategy under the second state data, it obtains the energy consumption, including:

[0019] Input the second state data and the second power management strategy into the calibrated energy consumption model, and output the energy consumption corresponding to the second state data and the second power management strategy.

[0020] In conjunction with the first possible implementation of the first aspect, in the fourth possible implementation of the first aspect, based on the second state data, a second power management strategy is determined at the time of the second state data through a predetermined exploration strategy, including:

[0021] Select the second power management strategy with the highest reward under the second state data according to the predetermined first ratio, based on the reinforcement learning model.

[0022] Select a random power management strategy according to a predetermined second ratio, and determine the random power management strategy as the second power management strategy.

[0023] In conjunction with the first aspect, in the fifth possible implementation of the first aspect, after selecting the first power management strategy corresponding to the largest first reward based on the rewards corresponding to different power management strategies, and performing power management on the robot, the method further includes:

[0024] When the robot is determined to be in the first state data, the third reward is executed after the first power management strategy;

[0025] The reinforcement learning model is updated with the third reward determined after executing the first power management strategy when the robot is in the first state data.

[0026] In conjunction with the first aspect, in the sixth possible implementation of the first aspect, after obtaining the robot's first state data at the current time and the work task to be performed at the current time, the method further includes:

[0027] Determine the sub-tasks included in the work task and the remaining battery power of the robot;

[0028] Based on the remaining battery power and the sub-task, it is determined whether the robot should cooperate with the automated guided vehicle to complete the work task.

[0029] A second aspect of this application provides a power management device for a robot, the device comprising:

[0030] The first state data acquisition unit is used to acquire the first state data of the robot at the current time and the work task to be performed at the current time.

[0031] The reward determination unit is used to input the first state data into the reinforcement learning model and predict the reward corresponding to the robot when it performs the work task according to different power management strategies through the reinforcement learning model.

[0032] The power management unit is used to select the first power management strategy corresponding to the largest first reward based on the rewards corresponding to different power management strategies, and to manage the power of the robot.

[0033] A third aspect of this application provides a robot including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the robot performs the method as described in any of the first aspects.

[0034] A fourth aspect of this application provides a computer program product that, when run on a computer, causes the computer to execute the methods described in the first aspect or its various implementations.

[0035] A fifth aspect of this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the method as described in any of the first aspects.

[0036] A sixth aspect of this application provides a chip for implementing the methods in the various implementations of the first aspect described above. Specifically, the chip includes a processor for calling and running a computer program from a memory, causing a device equipped with the chip to perform the methods as described in the first aspect or its various implementations.

[0037] The beneficial effects of this application embodiment compared with the prior art are as follows: This application embodiment obtains the robot's first state data at the current time and the work task to be performed at the current time, and inputs the first state data into a pre-trained reinforcement learning model. When the robot is in the first state data, the reward corresponding to different power management strategies is executed, and the first power management strategy corresponding to the largest first reward is selected to manage the robot's power. By selecting the first power management strategy corresponding to the largest first reward for power management, the robot can complete the task more efficiently, reduce battery charging and replacement, and improve the robot's reliability and stability. Attached Figure Description

[0038] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0039] Figure 1 This is a schematic diagram illustrating the implementation process of a robot power management method provided in an embodiment of this application;

[0040] Figure 2 This is a schematic diagram of partial data from a training reward table provided in an embodiment of this application;

[0041] Figure 3 This is a schematic diagram illustrating the implementation process of a reinforcement learning model training method provided in an embodiment of this application;

[0042] Figure 4 This is a schematic diagram illustrating the implementation process of a method for updating rewards provided in an embodiment of this application;

[0043] Figure 5 This is a schematic diagram of a robot power management device provided in an embodiment of this application;

[0044] Figure 6 This is a schematic diagram of a robot provided in an embodiment of this application. Detailed Implementation

[0045] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this application with unnecessary detail.

[0046] To illustrate the technical solution described in this application, specific embodiments are provided below.

[0047] With the rapid development of artificial intelligence and robotics, humanoid robots have been widely applied in various fields, including home, healthcare, service, education, automotive, and logistics. In the home, humanoid robots can assist with housework, care for the elderly and children, and provide educational tutoring, effectively alleviating the problem of insufficient manpower in households. In the healthcare field, humanoid robots can assist in surgery, provide rehabilitation treatment, and offer medical care, significantly improving the quality and efficiency of medical services. In education, humanoid robots, as interactive and adaptive teaching assistants, can participate in interactive learning activities, providing personalized learning experiences and tutoring support. In the automotive and logistics fields, humanoid robots can perform tasks such as precision assembly, handling, and quality inspection, improving production efficiency and supply chain efficiency. These applications demonstrate the importance and potential of humanoid robots in modern society.

[0048] However, existing humanoid robots are typically limited by battery life. Constrained by the overall design of the robot, battery size is limited, resulting in most humanoid robots having a runtime of only 3-6 hours, and some even less. This not only affects the user experience but also restricts their application in certain critical areas. For example, in industrial applications, this discharge behavior is frequent and continuous, making continuous battery operation impossible. Thermal management is also a key issue for humanoid robots.

[0049] Current energy management methods mostly employ simple power monitoring and management algorithms. This inadequacy not only limits the robot's endurance but also increases operational costs. For example, most humanoid robots only operate for about an hour and a half, while charging takes up to an hour. This forces factories or warehouses to use two or more robots in rotation for each task. Furthermore, frequent charging and battery replacements not only increase operating costs but can also affect the robot's reliability and stability. For instance, when lithium batteries are overcharged, excess lithium ions may become permanently fixed in the crystal lattice, shortening the battery's cycle life. Insufficient power can also lead to task interruptions or failures, especially when performing critical tasks, which can have serious consequences.

[0050] To address the aforementioned problems, this application proposes a method for managing the power consumption of a robot, such as... Figure 1 As shown, the method includes:

[0051] In S101, the robot's first state data at the current time and the work task to be performed at the current time are obtained.

[0052] The robot in this application embodiment may include a humanoid robot or other non-humanoid robots.

[0053] The current time in this embodiment can be any moment during the robot's operation when it needs to determine the power management strategy.

[0054] The work tasks include one or more of the following: handling tasks, grasping tasks, and human interaction tasks.

[0055] The power management strategies in this application embodiment may include adjusting motion modes, task execution strategies, sensor power consumption, battery charging and discharging strategies, environmental perception and adaptation strategies, and adaptive energy-saving strategies, to adjust the robot's energy consumption, task completion efficiency, and battery health indicators. Adjusting motion modes includes, for example, adjusting normal mode, energy-saving mode, and acceleration mode. Adjusting task execution strategies includes, for example, prioritizing high-priority tasks or breaking down complex tasks into multiple sub-tasks and completing them step-by-step based on battery status. Adjusting sensor power consumption includes, for example, turning off some unnecessary sensors (such as vision sensors) when the robot is performing simple standing-standby tasks to reduce the power consumption of the computing module and save power. Adjusting battery charging and discharging strategies includes, for example, when the battery level is below a predetermined level, the robot can automatically switch to a low-power mode, limit the execution of non-critical tasks, and find the nearest charging station for charging. Adjusting environmental perception and adaptation strategies includes, for example, when the ambient temperature is high, the robot can reduce power consumption to reduce heat generation and avoid battery overheating; when the ambient temperature is low, the robot can take insulation measures to optimize battery performance. Adjusting adaptive power-saving strategies includes limiting the execution of non-critical tasks in low-power mode to reduce power consumption, adjusting task priorities to prioritize critical tasks when the battery is low, and issuing a power warning when the battery level is below a certain threshold to remind users or the system to take measures.

[0056] The first state data in this application embodiment may include at least one of the following: robot motion data, robot environment data, robot battery status data, and robot position data.

[0057] Motion data of a robot can be acquired using motion sensors such as inertial measurement units (IMUs) and accelerometers installed on the robot. For example, information such as the robot's current walking speed of 0.5 m / s, acceleration of 0.2 m / s², and direction of movement towards the kitchen table can be obtained. This motion data can help determine the robot's movement pattern and workload, providing a basis for power management.

[0058] Environmental data can be acquired using the robot's environmental sensing sensors, such as temperature and humidity sensors and light sensors. For example, the temperature in the kitchen could be 28 degrees Celsius, the humidity 70%, and the light intensity 300 lux. Temperature in the environmental data affects the battery's charging and discharging efficiency, while humidity and light intensity information can indirectly influence the robot's operating status and energy consumption, thus requiring consideration in power management.

[0059] Battery status data can be acquired in real time using battery power sensors. This includes information such as remaining battery power, battery temperature, and battery operating status. For example, it can show that the battery is discharging, with a discharge current of 2A. Battery status data is one of the core data sets for power management, directly affecting the robot's continuous operating time and the selection of power management strategies.

[0060] The robot's location data can be obtained through its positioning system, such as lidar, vision, or wireless signal-based systems. This includes information like the robot's current location, its straight-line distance to a charging station, and its distance to the target area. This location data can be used to determine whether the robot is approaching the target area or a charging station, enabling informed decisions in power management, such as whether to adjust power allocation in advance or proceed to charging.

[0061] In S102, the first state data is input into the reinforcement learning model, and the reinforcement learning model predicts the reward corresponding to the robot when it executes different power management strategies.

[0062] The rewards for different power management strategies can be determined based on at least one of the following: task completion efficiency, energy consumption, and battery health indicators.

[0063] A reinforcement learning model is a type of model that calculates rewards based on a robot's state data and the power management strategies it executes. The reward reflects the merit of executing a particular power management strategy in a given state, and its calculation comprehensively considers multiple factors such as task completion efficiency, energy consumption, and battery health indicators. For example, in the aforementioned household cleaning task, the robot has several power management strategies to choose from, such as continuing cleaning at the current power, reducing power, or pausing the cleaning task. After inputting the initial state data into the reinforcement learning model, the model calculates the reward corresponding to each power management strategy based on historical data and a pre-defined reward mechanism. For example, the reward for continuing cleaning at the current power is 0.6, the reward for reducing power is 0.8, and the reward for pausing the cleaning task is 0.7.

[0064] Task completion efficiency can be quantified using at least one of the following parameters: task completion time, task completion power, task load capacity, and task complexity. Energy consumption can be quantified using at least one of the following parameters: battery charge change after task execution, average power consumption, and energy consumption per unit of task. Battery health indicators can be quantified using at least one of the following parameters: remaining battery life, battery temperature, battery internal resistance, and battery self-discharge rate.

[0065] This application embodiment can find the corresponding reward for executing different power management strategies in the current state based on the reward table in a pre-trained reinforcement learning model (such as a Q-learning model or a deep extended model). Alternatively, the reward corresponding to different power management strategies in the current state can also be determined using a deep Q-network or a multinomial function.

[0066] For example, cleaning robots that perform cleaning tasks in the home environment, after completing training, obtain... Figure 2 The reward table shown is as follows. This table includes the following rewards: when the robot performs tasks such as grasping objects, moving parts, and interacting with humans, the rewards are 10.5, 9.1, and 8.7 respectively when the power management strategy is in energy-saving mode; 8.2, 7.8, and 6.37 respectively when the power management strategy is in standard mode; and 3.1, 2.5, and 1.8 respectively when the power management strategy is in acceleration mode. Therefore, in the object grasping state, the rewards for energy-saving mode, standard mode, and acceleration mode are 10.5, 8.2, and 3.1 respectively; in the part moving state, the rewards are 9.1, 7.8, and 2.5 respectively; and in the human interaction state, the rewards are 8.7, 6.3, and 1.8 respectively.

[0067] When rewards are related to energy consumption, before calculating energy consumption, embodiments of this application can pre-calibrate energy consumption models for different power management strategies under different environments. Based on these energy consumption models, the energy consumption corresponding to the acquired second state data and the second power management strategy can be calculated. Correspondingly, task completion efficiency models and battery health indicator models for different power management strategies under different environments can also be pre-calibrated, thereby quickly obtaining data such as task completion efficiency and battery health indicators based on the calibrated models.

[0068] In S103, based on the rewards corresponding to different power management strategies, the first power management strategy corresponding to the highest first reward is selected to manage the power of the robot.

[0069] After receiving the rewards corresponding to different power management strategies, the power management strategy with the highest reward is selected as the current power management strategy. For example, in Figure 2 In the example shown, during object grasping, the power management strategy that maximizes the reward is selected as the energy-saving mode to manage the robot's power. This reduces energy consumption and extends battery life while ensuring task completion, and also takes into account battery health, preventing damage from over-discharge.

[0070] In this embodiment, before determining the reward corresponding to the power management strategy under different states using a reward table determined by a reinforcement learning model or a deep Q-network and a multinomial function, this application also includes a process of training the reinforcement learning model, such as... Figure 3 As shown, the process includes:

[0071] In S301, the second state data of the robot is initialized, and the reward table is initialized, which includes the rewards obtained by executing different power management strategies in different states.

[0072] The second state data may include at least one of the following: remaining battery power, current robot task type, robot motion mode, and current environmental parameters. The initialized reward table can be set to 0 or other fixed values ​​to correspond to different power management strategies for different state data.

[0073] In S302, based on the second state data, a second power management strategy is determined at the time of the second state data through a predetermined exploration strategy.

[0074] In this embodiment, search strategies such as the ε-greedy algorithm, Softmax strategy, and upper confidence bound strategy can be used as the predetermined power management strategy. The ε-greedy algorithm is a balance between exploration and utilization, where ε represents the probability of exploration and 1-ε represents the probability of utilization. For example, setting ε to 0.1 means there is a 90% probability of choosing the currently known optimal power management strategy (the power management strategy with the highest determined reward) and a 10% probability of choosing a random power management strategy. Based on the trained data, in the second state data, the known optimal power management strategy might be standard mode cleaning. However, according to the strategy, there is a 10% probability of randomly choosing other power management strategies, such as energy-saving mode cleaning or accelerated mode cleaning, ultimately determining the second power management strategy as standard mode cleaning.

[0075] In S303, the second reward and third state data after executing the second power management strategy under the second state data are obtained. The second state data is updated with the third state data, the reward table is updated with the second reward, and the process jumps to execute the step of determining the second power management strategy under the second state data through a predetermined exploration strategy based on the second state data, until the reward table converges or the number of learning times reaches a predetermined number, thus completing the training of the reinforcement learning model.

[0076] Based on the example above, after executing a power management strategy for cleaning in standard mode, the robot acquires the reward for that strategy and new third-state data. The reward is determined based on task completion, energy consumption, and battery health. If the task is successfully completed, energy consumption is within a reasonable range, and battery health is unaffected, a positive reward is given, such as +10. New state data may include a decrease in remaining battery power or a change in robot position. Based on the reward and the new third-state data, the reward can be updated using the following reward update formula. The reward update formula can be expressed as:

[0077] Q(s,a)←Q(s,a)+α[r+γmax(Q(s',a'))-Q(s,a)]

[0078] Where s is the current state, a is the currently selected power management strategy, r is the reward obtained after executing the power management strategy, s' is the new state after executing the power management strategy, γ is the discount factor, representing the importance attached to future rewards, and α is the learning rate, used to control the degree of influence of new information on the update of rewards.

[0079] The new third-state data can be used to update the second-state data. That is, the third-state data is used as the updated second-state data. The corresponding reward is re-determined through the new second-state data, and the reward table is updated according to the reward until the reward table converges or the number of training times reaches a predetermined number. Finally, the training of the reinforcement learning model is completed, and the rewards corresponding to different state data and power management strategies are obtained. Alternatively, based on the training results, the relationship between state data, power management strategies and rewards can be represented by a deep Q-network or a multinomial function, so that the reinforcement learning model can provide the robot with the optimal power management strategy decision in various states.

[0080] In this embodiment of the application, the reward in the reward update formula can be determined by task completion efficiency, energy consumption, and battery health indicators.

[0081] For example, the Pareto optimization method can be used for multi-objective optimization. Task completion efficiency, energy consumption, and battery health indicators are each assigned a certain weight, and a comprehensive reward value is calculated based on the actual performance of each indicator. For instance, task completion efficiency might have a weight of 0.5, energy consumption 0.3, and battery health 0.2. Based on the performance of this cleaning task, a high task completion efficiency would receive a higher reward score, such as 8 points (out of 10); energy consumption would be within a reasonable range, receiving 7 points; and a good battery health would receive 9 points. The comprehensive reward value would be 0.5 × 8 + 0.3 × 7 + 0.2 × 9 = 7.7, providing positive feedback to the robot and reinforcing its tendency to adopt this power management strategy in similar situations.

[0082] Furthermore, to further improve the reinforcement learning model, the reward can be updated based on the determined power management strategy during actual use, such as... Figure 4 As shown, the process may include:

[0083] In S401, when the robot is determined to be in the first state data, the third reward following the first power management strategy is executed.

[0084] After the robot selects energy-saving mode for cleaning (first power management strategy), it acquires new status data after a period of execution. At this point, the battery has 20% remaining power, the task is nearing completion, and the robot is close to a charging station. Based on this new status data and factors such as task completion, energy consumption, and battery health after executing the energy-saving mode cleaning strategy, a third reward is calculated. Assuming the third reward is 0.9, it indicates that executing the energy-saving mode cleaning strategy was a relatively good decision under the first status data.

[0085] In S402, the reinforcement learning model is updated with the third reward determined after executing the first power management strategy when the robot is in the first state data.

[0086] Using the reward update formula:

[0087] Q(s,a)←Q(s,a)+α[r+γmax(Q(s',a'))-Q(s,a)]

[0088] The reinforcement learning model is updated. The third reward is substituted into the formula to update the reward corresponding to the first state data. In this way, when the robot is in a similar first state data situation next time, the model can more accurately select the optimal power management strategy based on the updated reward, continuously improving the performance and adaptability of power management.

[0089] In possible implementations, embodiments of this application may further determine the sub-tasks included in the work task and the remaining battery power of the robot; and determine whether the robot should cooperate with the automated guided vehicle to complete the work task based on the remaining battery power and the sub-tasks.

[0090] For example, in a logistics warehouse, humanoid robots and AGVs (Automated Guided Vehicles) work together to move goods from one area to another. The tasks include sub-tasks such as retrieving goods from shelves and moving them to designated locations. The robot can detect whether the distance to be moved exceeds a predetermined threshold. If it does, it sends a coordination command to the AGV, enabling the AGV to assist the robot in moving the goods. This allows the robot to save power during the moving process, complete tasks more reliably, and improve work efficiency.

[0091] The power management method in this application embodiment has significant advantages in extending the battery life of humanoid robots through intelligent algorithms and adaptive strategies. The robot can dynamically adjust its energy consumption according to different task types, movement patterns, and environmental conditions, avoiding energy waste in traditional methods, significantly improving battery efficiency, and extending the robot's working time in complex environments.

[0092] In traditional energy management solutions, robots typically operate based on simple preset strategies or fixed battery usage patterns. For example, a robot might always walk or carry objects at the same speed, regardless of battery level or environmental changes. However, an energy management system optimized through reinforcement learning can automatically adjust the robot's power management strategy or switch to energy-saving mode when the battery is low, based on task type and battery status, thereby extending the robot's working time. Alternatively, the robot can automatically adjust its action strategy according to task requirements, such as increasing power output when efficient movement is needed (e.g., carrying heavy objects) and reducing power consumption during low-load tasks. When the battery is low, the robot can intelligently decide whether to recharge or reschedule tasks to reduce battery depletion.

[0093] Assuming a robot performs a logistics task in collaboration with an AGV, the battery consumes 60% per hour in the traditional model. However, by using a power management method optimized through reinforcement learning, energy consumption can be reduced to 25%, while the task completion time remains the same or is shortened. This extends the robot's working time for each task by 30%, significantly improving the robot's operational efficiency in practical applications, reducing the frequency of battery replacement and charging, and thus saving on maintenance costs.

[0094] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0095] Figure 5 This is a schematic diagram of a robot power management device provided in an embodiment of this application. The device includes:

[0096] The first state data acquisition unit 501 is used to acquire the first state data of the robot at the current time and the work task to be performed at the current time.

[0097] The reward determination unit 502 is used to input the first state data into the reinforcement learning model and predict the reward corresponding to the robot when it performs the work task according to different power management strategies through the reinforcement learning model.

[0098] The power management unit 503 is used to select the first power management strategy corresponding to the largest first reward according to the rewards corresponding to different power management strategies, and to manage the power of the robot.

[0099] Figure 5 The robot's power management device shown is, with Figure 1 The power management method for the robot shown corresponds to this.

[0100] Figure 6 This is a schematic diagram of a robot provided in an embodiment of this application. Figure 6 As shown, the robot 6 in this embodiment includes a processor 60, a memory 61, and a computer program 62 stored in the memory 61 and executable on the processor 60, such as a robot power management program. When the processor 60 executes the computer program 62, it implements the steps in the power management method embodiments of the various robots described above. Alternatively, when the processor 60 executes the computer program 62, it implements the functions of each module / unit in the various device embodiments described above.

[0101] For example, the computer program 62 may be divided into one or more modules / units, which are stored in the memory 61 and executed by the processor 60 to complete this application. The one or more modules / units may be a series of computer program instruction segments capable of performing a specific function, which describe the execution process of the computer program 62 in the robot 6.

[0102] The robot may include, but is not limited to, a processor 60 and a memory 61. Those skilled in the art will understand that... Figure 6 This is merely an example of robot 6 and does not constitute a limitation on robot 6. It may include more or fewer parts than shown, or combine certain parts, or different parts. For example, the robot may also include input / output devices, network access devices, buses, etc.

[0103] The processor 60 may be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor.

[0104] The memory 61 can be an internal storage unit of the robot 6, such as a hard drive or memory. The memory 61 can also be an external storage device of the robot 6, such as a plug-in hard drive, Smart Media Card (SMC), Secure Digital (SD) card, or Flash Card. Furthermore, the memory 61 can include both internal and external storage units of the robot 6. The memory 61 is used to store the computer program and other programs and data required by the robot. The memory 61 can also be used to temporarily store data that has been output or will be output.

[0105] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0106] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0107] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0108] In the embodiments provided in this application, it should be understood that the disclosed devices / terminal equipment and methods can be implemented in other ways. For example, the device / terminal equipment embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual coupling or direct coupling or communication connection may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0109] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0110] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0111] If the integrated module / unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments can also be implemented by hardware related to computer program instructions. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include: any entity or device capable of carrying the computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc.

[0112] In addition, this application also provides a computer program product that, when run on a computer, causes the computer to execute the methods in the above-described implementations.

[0113] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A method for managing the power of a robot, characterized in that, The method includes: Obtain the robot's first state data at the current time and the work task to be performed at the current time; The process involves initializing the robot's second state data and a reward table, which includes rewards for executing different power management strategies in different states. Based on the second state data, a second power management strategy is determined using a predetermined exploration strategy. Energy consumption models for different tasks executing different power management strategies in different environments are calibrated. A second reward and third state data are obtained after executing the second power management strategy in the second state data state. The second state data is updated with the third state data, and the reward table is updated with the second reward. The process then jumps to the step of determining the second power management strategy based on the second state data using a predetermined exploration strategy. This continues until the reward table converges or the learning iterations reach a predetermined number, completing the reinforcement learning model. Training involves inputting the first state data into a reinforcement learning model, and using the reinforcement learning model to predict the rewards corresponding to the robot performing the work task according to different power management strategies. Obtaining the second reward after executing the second power management strategy in the state of the second state data includes: after the robot executes the second power management strategy in the state of the second state data, obtaining at least one of task completion efficiency, energy consumption, and battery health indicators, including: inputting the second state data and the second power management strategy into a calibrated energy consumption model, outputting the energy consumption corresponding to the second state data and the second power management strategy; and determining the second reward after executing the second power management strategy using at least one of the task completion efficiency, the energy consumption, and the battery health indicators. Based on the rewards corresponding to different power management strategies, the first power management strategy corresponding to the highest first reward is selected to manage the power of the robot.

2. The method according to claim 1, characterized in that, Based on the second state data, a second power management strategy is determined using a predetermined exploration strategy at the time of the second state data, including: Select the second power management strategy with the highest reward under the second state data according to the predetermined first ratio, based on the reinforcement learning model. Select a random power management strategy according to a predetermined second ratio, and determine the random power management strategy as the second power management strategy.

3. The method according to claim 1, characterized in that, After selecting the first power management strategy corresponding to the highest first reward based on the rewards corresponding to different power management strategies, and performing power management on the robot, the method further includes: When the robot is determined to be in the first state data, the third reward is executed after the first power management strategy; The reinforcement learning model is updated with the third reward determined after executing the first power management strategy when the robot is in the first state data.

4. The method according to claim 1, characterized in that, After acquiring the robot's first state data at the current time and the work task to be performed at the current time, the method further includes: Determine the sub-tasks included in the work task and the remaining battery power of the robot; Based on the remaining battery power and the sub-task, it is determined whether the robot should cooperate with the automated guided vehicle to complete the work task.

5. A power management device for a robot, characterized in that, The device includes: The first state data acquisition unit is used to acquire the first state data of the robot at the current time and the work task to be performed at the current time. The reward determination unit is used to initialize the robot's second state data and initialize a reward table, which includes rewards obtained by executing different power management strategies in different states; determine a second power management strategy for the second state data based on the second state data using a predetermined exploration strategy; calibrate energy consumption models for different tasks executing different power management strategies in different environments; obtain a second reward and third state data after executing the second power management strategy in the second state data state; update the second state data with the third state data; update the reward table with the second reward; and jump to execute the step of determining the second power management strategy for the second state data based on the second state data using a predetermined exploration strategy, until the reward table converges or the learning count reaches a predetermined number, completing the reinforcement. The training of the learning model involves inputting the first state data into a reinforcement learning model, and using the reinforcement learning model to predict the rewards corresponding to the robot performing the work task according to different power management strategies. Obtaining the second reward after executing the second power management strategy in the state of the second state data includes: after the robot executes the second power management strategy in the state of the second state data, obtaining at least one of task completion efficiency, energy consumption, and battery health indicators, including: inputting the second state data and the second power management strategy into a calibrated energy consumption model, outputting the energy consumption corresponding to the second state data and the second power management strategy; and determining the second reward after executing the second power management strategy based on at least one of the task completion efficiency, the energy consumption, and the battery health indicators. The power management unit is used to select the first power management strategy corresponding to the largest first reward based on the rewards corresponding to different power management strategies, and to manage the power of the robot.

6. A robot comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it causes the robot to perform the method as described in any one of claims 1-4.

7. A computer program product comprising computer program instructions, characterized in that, When the computer program is run, the method as described in any one of claims 1-4 is performed.

Images