Robot control method, apparatus, device, medium, and product

Abstract

This application discloses a robot control method, device, equipment, medium, and product, relating to the field of automatic control technology. The method includes: acquiring the actual operating condition of the robot in the current control cycle of a target task; extrapolating multiple different predicted trajectories starting from the robot's current position, and determining the energy consumption index and trajectory deviation index corresponding to each predicted trajectory; determining the local theoretical Pareto front based on the energy consumption index and trajectory deviation index; determining the robot's current control action through the robot's current control strategy; extrapolating the predicted operating condition of the robot in the next control cycle based on the robot's actual operating condition and current control action in the current control cycle; generating an integer reward based on the distance of the predicted operating condition relative to the local theoretical Pareto front; adjusting the robot's control strategy based on the integer reward using a deep reinforcement learning algorithm, and driving the robot's movement through the adjusted control strategy.

Description

Technical Field

[0001] This application relates to the field of automatic control technology, and in particular to a robot control method, device, equipment, medium and product. Background Technology

[0002] With the rapid development of robotics technology, mobile robots are gradually replacing humans in performing repetitive and high-risk tasks. Currently, robot operating efficiency and energy consumption levels have become key factors affecting practicality and deployment scale. Existing robots mostly employ rule-based or model-predictive control methods, completing walking and obstacle avoidance control through preset paths and fixed control parameters. However, in actual operation, factors such as changes in road conditions, battery performance degradation, load fluctuations, and external interference can easily cause deviations between the control model and real-world conditions, leading to problems such as abnormal energy consumption, trajectory deviation, and even motion instability in complex environments. Some research has introduced reinforcement learning methods to improve decision-making capabilities, but most solutions focus on trajectory tracking or obstacle avoidance performance optimization, lacking system modeling and joint constraints on energy consumption and task efficiency. In other words, they cannot balance energy consumption and task efficiency, making it difficult to maintain stable returns under uncertain conditions. Summary of the Invention

[0003] This application provides a robot control method, apparatus, equipment, medium, and product to solve or at least partially solve the defects or deficiencies in related technologies.

[0004] In a first aspect, this application provides a robot control method, the robot control method comprising: The actual operating status of the robot during the current control cycle of the target task is collected; Starting from the robot's current position, multiple different predicted trajectories within a preset future time domain are deduced, and the energy consumption index and trajectory deviation index corresponding to each predicted trajectory are determined. Based on the energy consumption index and trajectory deviation index corresponding to all the predicted trajectories, the local theoretical Pareto front is determined. The current control action of the robot corresponding to the current control cycle is determined by the robot's current control strategy; Based on the actual operating condition of the robot in the current control cycle of the target task and the current control action of the robot, the predicted operating condition of the robot in the next control cycle of the target task is deduced. The shaping reward is generated based on the distance of the robot's predicted operating condition state in the next control cycle of the target task relative to the local theoretical Pareto front. The control strategy for the robot is adjusted based on the integer reward using a deep reinforcement learning algorithm, and the robot is driven to move by the adjusted control strategy.

[0005] Secondly, this application provides a robot control device, the robot control device comprising: The status acquisition module is configured to acquire the actual operating status of the robot in the current control cycle of the target task; The trajectory index determination module is configured to deduce multiple different predicted trajectories within a preset time domain starting from the robot's current position, and determine the energy consumption index and trajectory deviation index corresponding to each predicted trajectory. The boundary determination module is configured to determine the local theoretical Pareto front based on the energy consumption index and trajectory deviation index corresponding to all the predicted trajectories. The strategy output module is configured to determine the current control action of the robot corresponding to the current control cycle based on the robot's current control strategy. The state deduction module is configured to deduce the predicted operating state of the robot in the next control cycle of the target task based on the actual operating state of the robot in the current control cycle of the target task and the current control action of the robot. The reward generation module is configured to generate a shaped reward based on the distance of the robot’s predicted operating condition state in the next control cycle of the target task relative to the local theoretical Pareto front. The strategy adjustment driving module is configured to adjust the control strategy for the robot based on the integer reward using a deep reinforcement learning algorithm, and drive the robot to move using the adjusted control strategy.

[0006] Thirdly, this application provides a robot device, including: a memory, a processor, and a program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of the robot control method described in the first aspect above.

[0007] Fourthly, this application provides a readable storage medium having a program stored thereon, which, when executed by a processor, implements the steps of the robot control method described in the first aspect above.

[0008] Fifthly, this application provides a program product, including a program that, when executed by a processor, implements the steps of the robot control method described in the first aspect.

[0009] According to the specific embodiments provided in this application, the following technical effects are disclosed: This application provides a robot control method, device, equipment, medium, and product. Within the robot's control cycle, multiple different predicted trajectories are deduced, and the energy consumption index and trajectory deviation index corresponding to each predicted trajectory are determined. Then, based on the energy consumption index and trajectory deviation index corresponding to all predicted trajectories, a local theoretical Pareto front is determined, thereby finding the minimum energy consumption solution set that balances multiple objectives (energy consumption and control accuracy). This achieves system modeling and joint constraints on energy consumption and task efficiency (task efficiency is affected by control accuracy). Based on the robot's actual operating condition in the current control cycle and the current control action output by the robot control strategy for the current control cycle, the predicted operating condition of the robot in the next control cycle can be deduced. Based on the distance of this predicted operating condition relative to the local theoretical Pareto front, an integer reward can be generated. This integer reward can guide the robot control strategy to always operate close to the minimum energy efficiency boundary, realizing a robot control strategy that smoothly balances energy consumption and task efficiency. Attached Figure Description

[0010] Figure 1 A flowchart illustrating a robot control method provided in one embodiment of this application; Figure 2 A flowchart illustrating a robot control method according to another embodiment of this application; Figure 3 A schematic diagram of the functional modules of a robot control device provided in an embodiment of this application; Figure 4 This is a schematic diagram of the structure of a robot device provided in an embodiment of this application. Detailed Implementation

[0011] The present application will now be described in further detail with reference to the accompanying drawings and specific embodiments.

[0012] In the embodiments of this application, such as Figure 1 and Figure 2 As shown, two robot control methods are provided, both of which can be executed by robot devices. Specifically, the robot devices can be mobile robots that need to move according to the movement path generated by the task scheduling to achieve their functions, such as inspection robots, tour guide robots, and delivery robots.

[0013] In one embodiment of this application, such as Figure 1 As shown, the robot control method includes steps 101 to 107. Wherein: Step 101: Collect the actual operating status of the robot in the current control cycle of the target task.

[0014] Controlling a robot to achieve a target task (such as an inspection task) can be divided into multiple control cycles, each with the same fixed time step.

[0015] In one implementation, this step can be achieved by: collecting the robot's operating condition information during the current control cycle of the target task; filtering and denoising the robot's operating condition information during the current control cycle of the target task; and concatenating the filtered and denoised data according to a preset dimension order to construct a sparse state vector. This sparse state vector represents the actual operating condition of the robot during the current control cycle of the target task.

[0016] In this embodiment, the operational status information may include: forward distance, lateral distance, battery state of charge, body angular velocity, and drive wheel torque. To comprehensively perceive the robot's current physical state, the operational status information specifically covers the following five dimensions: forward and lateral distances obtained through ranging sensors, used to characterize the spatial compactness of surrounding obstacles; battery state of charge read through the battery management system, used to measure energy reserves; body angular velocity read through the inertial measurement unit, used to reflect the current rotational attitude; and drive wheel torque obtained through the motor driver, used to reflect the current load and ground resistance characteristics.

[0017] After the initial data acquisition is complete, the above-mentioned operating condition information can be filtered and denoised to eliminate potential high-frequency interference from the acquisition equipment. After filtering and denoising, the original redundant data can be discarded. Instead, according to a predefined data arrangement protocol, the forward distance, lateral distance, battery state of charge, body angular velocity, and drive wheel torque are concatenated in a preset dimensional order. This process encapsulates the scattered sensor data into a compact low-dimensional array, i.e., a sparse state vector, thus providing a standardized input basis for subsequent dynamics analysis.

[0018] To address the noise interference problem in multi-source sensor data fusion, directly using the raw data can lead to oscillations in the control strategy. By filtering and denoising the collected operating condition information and concatenating it according to a preset dimensional order, a standardized sparse state vector is constructed. This not only removes high-frequency signal noise but also provides a unified and low-dimensional data interface for subsequent dynamics deduction, significantly improving the robot control system's tolerance to input data quality.

[0019] Existing technologies such as visual SLAM navigation based on feature point matching (Visual SLAM) or visual servo control based on convolutional neural networks (CNNs) often rely on high-dimensional visual data for environmental perception. These technologies are prone to failure in strong light or monotonous texture environments and have extremely high computational loads, making them unsuitable for real-time control requirements at the edge. In contrast, this application employs a method of constructing sparse state vectors. By integrating ranging sensor data, battery status, and dynamic feedback, redundant visual information can be eliminated, directly identifying physical features strongly correlated with motion control. This ensures that the perception process remains stable and reliable even under conditions of limited computational resources or visual interference.

[0020] Step 102: Starting from the robot's current position, multiple different predicted trajectories are derived within a preset time domain in the future, and the energy consumption index and trajectory deviation index corresponding to each predicted trajectory are determined.

[0021] In this step, starting from the robot's current position and using the aforementioned sparse state vector as initial conditions, multiple different predicted trajectories within a preset future time domain are derived through a dynamic model. These multiple predicted trajectories are obtained based on the assumptions of various robot driving styles (or driving techniques, such as "rapid acceleration," "constant speed," and "turning"), and are used for parallel comparison to select the optimal solution.

[0022] In one implementation, the step of deducing multiple different predicted trajectories within a preset time domain from the robot's current position can be achieved as follows: sampling is performed in a preset action space using low-difference sequences to generate multiple sets of drive control quantity sequences; using sparse state vectors as the starting point, the state changes of the robot within a finite time domain are deduced in parallel based on each set of drive control quantity sequences using a dynamic model to generate multiple different predicted trajectories, wherein one predicted trajectory can be deduced based on one set of drive control quantity sequences.

[0023] The preset motion space is the set of all possible control actions (such as command torque applied to the robot's drive wheels) that the robot can take in a given state.

[0024] A low-discrepancy sequence is a deterministic numerical sequence. By sampling within a specified space using a low-discrepancy sequence, a point set that covers the specified space more uniformly and densely than a purely random sequence can be obtained. Optionally, the low-discrepancy sequence in this embodiment can be a Sobol sequence, which utilizes its uniform distribution characteristics in multidimensional space to cover a wider solution space with a limited number of samplings, thereby improving inference efficiency.

[0025] A sequence of drive control quantities comprises a set of drive wheel command torques (or their equivalent drive wheel command currents or drive wheel command voltages) arranged chronologically within a future finite time domain. It is not a single value, but rather a series of pre-set drive commands for each moment within the future finite time domain. Optionally, each drive wheel command torque in the drive control quantity sequence can specifically be a two-dimensional vector containing the command torques of the left and right wheels.

[0026] Next, with sparse state vectors Starting from this point, the dynamic model is used to extrapolate the future time domain based on the sequences of each set of driving control variables. The robot's state changes, generating multiple different predicted trajectories. For the first... The predicted trajectory, in the first The state at time is based on the first Time-of-use sampling control quantity The state transition process is represented as: .

[0027] in, Indicates the first The predicted trajectory is in the first The state of the simulation at any moment, This represents the dynamic model, i.e., the differential dynamic equation function. Indicates the first The predicted trajectory in the first The state of the time-lapse (when) At that time, this value is the sparse state vector. ), Indicates the first The i-th drive control quantity sequence corresponding to the predicted trajectory corresponds to the th The command torque of the drive wheel at any given moment.

[0028] This dynamic model uses the command control quantity of the drive wheel as the input quantity, rather than the conventional kinematic speed command, thus directly reflecting the drive energy consumption from a mechanical perspective.

[0029] To address the problem of low efficiency in motion space search during real-time control, traditional random sampling methods struggle to cover the optimal solution quickly. This application addresses this issue by using low-discrepancy sequences for sampling within a predefined motion space. Leveraging their uniform spatial distribution, this approach effectively covers the feasible control domain even with a very small sample size, significantly improving the success rate of optimal control sequence search while maintaining control frequency.

[0030] In this embodiment, the differential dynamic equation function specifically refers to a rigid body dynamics model that describes the motion of the robot chassis under stress. Depending on the specific mechanical configuration of the robot, the differential dynamic equation function can be a two-wheel differential motion model, an Ackerman steering model, or an omnidirectional movement model; there are no specific limitations on this.

[0031] In an exemplary embodiment, a two-wheel differential motion model based on the Newton-Euler equations can be employed. This model predetermines physical parameters such as overall machine mass, moment of inertia, wheel radius, and wheelbase, and specifically sets the ground friction coefficient and viscous damping coefficient to simulate environmental resistance. During the simulation, the two-wheel differential motion model first calculates the robot's linear and angular accelerations based on the input drive wheel control commands, and then uses numerical integration to convert these two accelerations into the simulation state for the next moment.

[0032] Existing technologies mostly employ static strategies trained offline, which cannot cope with dynamic physical constraints such as changes in the ground friction coefficient. "Offline training" here refers to pre-training the robot's control model (i.e., policy network) in a laboratory environment. Existing technologies typically fix the parameters of the trained model (i.e., static strategy) and directly install it into the robot. This means the robot can only respond based on the experience learned before leaving the factory. Once the physical parameters in the actual environment (such as slippery ground or changes in load) exceed its previous training range, it cannot adjust online, leading to control failure. This application's embodiment, by combining a dynamic model with finite-time-domain parallel extrapolation, can simulate the physical consequences of various future control actions online within milliseconds, transforming delayed physical feedback into forward-looking state predictions, thus solving the technical problem that static strategies cannot adapt to changes in dynamic environmental parameters.

[0033] For each predicted trajectory obtained through deduction, its corresponding energy consumption index and trajectory deviation index are determined. Prior to this, in one implementation, the robot control method may further perform the following steps, and then, based on the local reference state sequence determined in the following steps, determine the energy consumption index and trajectory deviation index corresponding to the predicted trajectory. The following steps specifically include: Obtain the discrete waypoint sequence provided by the task scheduler for the target task, and determine the first unreached waypoint in the discrete waypoint sequence; Generate a smooth path from the robot's current position to the first unreached waypoint; Based on the displacement of adjacent points on the smooth path and the preset time interval, a local reference state sequence representing the reference state of the robot at each point on the smooth path is determined.

[0034] The aforementioned local reference state sequence will serve as the basis for determining the trajectory deviation index corresponding to each predicted trajectory, and will be used to constrain the deviation range of the predicted trajectory.

[0035] In the above steps, firstly, a sequence of discrete waypoints is received from the upper-level task scheduling process for the target task. This sequence includes multiple waypoints, i.e., key locations, that must be traversed to achieve the target task. Then, the sequence of discrete waypoints can be traversed to identify and lock the first unreached waypoint in the sequence, which is determined as the local target pose that needs to be tracked at the current moment. This local target pose includes the target position (i.e., the first unreached waypoint) and the robot's desired orientation at the target position (i.e., what orientation the robot is expected to face when it reaches the first unreached waypoint, which is determined by the target task itself).

[0036] Subsequently, to plan a smooth path that conforms to kinematic constraints between the robot's current position and the first unreached waypoint, four key geometric control points can be constructed in the planar coordinate system: the robot's current position is determined as the starting geometric control point; the target position in the local target pose (i.e., the first unreached waypoint) is determined as the ending geometric control point; and the starting geometric control point extends along the robot's current body orientation. rice( (If positive), determine the first tangential geometric control point to constrain the tangential direction at the start; at the termination geometric control point, extend in the opposite direction to the desired orientation in the local target attitude. rice( (If the value is positive), determine the second tangential geometric control point to constrain the tangential direction when reaching the target position. Using the above four geometric control points, perform three Bezier interpolation operations to fit a smooth path with continuously changing curvature between the starting geometric control point (i.e., the robot's current position) and the ending geometric control point (i.e., the first waypoint not yet reached).

[0037] Subsequently, based on the control cycle of the robot control system, a preset time interval is set for sampling positions along the smooth path. The control cycle can be an integer multiple of the preset time interval. Based on the preset time interval, a series of discrete position points are sequentially collected along the smooth path. The reference linear velocity of each position point on the smooth path is calculated based on the displacement of adjacent position points and the preset time interval. The reference angular velocity of each position point on the smooth path is also calculated based on the rate of change of the tangent angle of the smooth path. This yields the robot's reference state (i.e., reference linear velocity and reference angular velocity) at each position point on the smooth path. Each position point sampled along the smooth path, along with its reference linear velocity and reference angular velocity, is then packaged and combined to generate a local reference state sequence containing the robot's motion reference state over a future period. This sequence serves as a benchmark in subsequent steps to constrain the deviation range of the predicted trajectory.

[0038] Existing technologies typically use a "straight-line connection" method to track discrete points, i.e., point-to-point (P2P). This can cause robots to experience harsh working conditions of "sudden stop-sudden turn-sudden start," for example: A broken path can cause "wall-crashing" turns: If discrete waypoints are connected directly by a straight line, the path will form a sharp angle (breakpoint) at the waypoint.

[0039] Before entering a curve (emergency braking): When the robot approaches the waypoint, it finds that the path ahead suddenly turns 90 degrees. In order to avoid running off the track, the control algorithm will command the drive wheels to reverse at full power (the current will surge in reverse instantaneously).

[0040] When cornering (hard turn): After the speed decreases, the steering motor needs to overcome the huge static friction of the ground to turn in place (Torque Spike).

[0041] After exiting the corner (slam on the accelerator): After aligning with the new direction, the motor will accelerate at full power from zero (starting current surge).

[0042] This drastic switching from "high forward current" to "high reverse current" is like hitting a motor and gearbox with a hammer, resulting in "severe load fluctuations".

[0043] In addition, existing technologies for discrete waypoint tracking control also have the problem of command mutation. Command mutation refers to the step change that occurs in the speed or steering command issued by the control system in a very short time, that is, a mathematical discontinuity appears. For example, the robot's current coordinate is A, and the next waypoint coordinate is B. The system requires the robot to "teleport" to B.

[0044] In existing discrete waypoint tracking control technologies, the aforementioned problems of trajectory discontinuity and command abrupt changes often occur simultaneously, leading to drastic fluctuations in robot motor load. However, the embodiments of this application utilize a cubic Bezier interpolation algorithm to generate a local reference state sequence. By constructing a smooth, continuous curved path between the robot's current position and the waypoint using geometric control points, a reference benchmark conforming to kinematic constraints is provided for the underlying control. This resolves the physical conflict between discrete commands and continuous motion, ensuring the smoothness of the robot's tracking process.

[0045] In one implementation, based on the above steps, the energy consumption index and trajectory deviation index corresponding to each predicted trajectory can be determined in the following ways: For each predicted trajectory, a mechanical work term is generated based on the driving wheel command torque and wheel speed at each moment in the future preset time domain. A smoothness penalty term is generated based on the rate of change of the driving wheel command torque between adjacent moments in the future preset time domain. The weighted result of the mechanical work term and the smoothness penalty term is determined as the energy consumption index of the predicted trajectory. For each predicted trajectory, the process deviation is determined based on the deviation between the predicted trajectory's projected state at each moment in the future preset time domain and the reference state at the corresponding moment in the local reference state sequence. The terminal deviation is formed based on the deviation between the predicted trajectory's final projected state at the last moment in the future preset time domain and the robot's reference state at the first waypoint not reached in the local reference state sequence. The sum of the process deviation and the terminal deviation is determined as the trajectory deviation index of the predicted trajectory.

[0046] In this embodiment, for the first The predicted trajectory is used to calculate its corresponding energy consumption index using the following formula. .

[0047] .

[0048] Among them, for the future preset time domain The first At that moment, the The corresponding drive wheel command torque at any given moment With the rotational speed of the robot wheels absolute value of the product The term for work done by a machine, of which the first... Drive wheel torque at any given moment For the first The i-th drive control quantity sequence corresponding to the predicted trajectory corresponds to the th The command torque of the drive wheel at time , the first The corresponding robot wheel speed at any given time By Substituting into the dynamic model, we can deduce that... Indicates applying force to the robot Under the given assumptions, what should the corresponding wheel speed of the robot be? At adjacent time points (i.e., the...) Time and the The square of the rate of change of the command torque of the drive wheels at time ( ). For smoothness penalty term, This represents the squaring operation (for scalars) or the squaring operation of the L2 norm (for vectors), in energy consumption indicators. In the calculation formula, This represents the squaring operation.

[0049] In the above formula, the control cycle is used. (10ms~100ms, in order to balance control precision and computational efficiency,) (Optional 50ms) and smoothing weighting coefficient (0.01~1.0, to avoid mechanical wear caused by frequent sudden changes in motor commands,) (Optional 0.1 for smooth and energy-efficient motion control) The weighted result of the mechanical work term and the smoothness penalty term will be used in a future preset time domain. Accumulate within to obtain the first... The predicted trajectory will be in the future preset time domain Internal corresponding energy consumption indicators .

[0050] In this embodiment, for the first The predicted trajectory is calculated using the following formula to determine its corresponding trajectory deviation index. .

[0051] .

[0052] in, Indicates the first The projected state of the predicted trajectory at time k. Represents the first in the local reference state sequence Time (corresponding to the time obtained by upsampling the above smooth path) The reference state of (a number of location points), This represents the preset process weight matrix. Represents the process weight matrix The square of the weighted Euclidean distance, Indicates the first The predicted trajectory in the first The projected state at any given moment (i.e., the terminal or end point). Indicates the local target attitude. This represents the preset terminal weight matrix. Indicates based on terminal weight matrix The square of the weighted Euclidean distance (i.e., the quadratic terminal cost).

[0053] in, For predicting trajectories In the future preset time domain The deduced state at time k. Let k be the reference state corresponding to time k in the local reference state sequence. This is the process deviation, used to track and constrain the linear and angular velocities throughout the entire current control cycle. This is the terminal deviation (or end-point deviation), used to ensure the convergence of the terminal position and orientation of the current control cycle. The sum of the process deviation and the terminal deviation is the trajectory deviation index corresponding to the predicted trajectory i in the future preset time domain H. .

[0054] In the above implementation, weighted Euclidean distance is used when determining the trajectory deviation index. Its advantage is that it solves the problem of inconsistent dimensions between different physical quantities such as "position (meter)" and "angle (radian)".

[0055] In the above embodiments, for and The problem of dimension mismatch ( Position, linear velocity, and angular velocity are the three dimensions. (Containing two dimensions: location point and orientation), the terminal weight matrix can be used. To adapt the dimensions, for state variables that do not require constraints (such as the instantaneous velocity of the terminal), their weights are reset to 0 in the terminal weight matrix P, or are set to the default value. The target speed is set to 0 (meaning it stops upon arrival), thus achieving rigorous mathematical calculation.

[0056] To adapt to the actual motion characteristics of the robot, the above process weight matrix... and terminal weight matrix Both can be used in the form of a diagonal matrix, and this setting must follow the configuration logic below: First, eliminate the influence of dimensions. Since the state vector (deduced state and reference vector) contains different units such as position (meters) and angle (radians), direct calculation will lead to numerical deviations. Therefore, the weights are usually set in reverse according to the maximum allowable error range of each physical quantity.

[0057] Second, ensure endpoint convergence. To prevent the robot from stopping prematurely when approaching the target point in pursuit of low energy consumption, thus incurring steady-state errors, the terminal weight matrix... The value is usually set significantly larger than the process weight matrix. (For example, magnify 5 to 10 times) to force the robot to eliminate deviations at the end of the path.

[0058] As a concrete implementation example, assume the sparse state vector is ,in These are the coordinates of the location point (unit: m). Heading angle (unit: rad). Here, represents linear velocity (unit: m / s). Considering that angular deviation typically has a greater impact on trajectory tracking than positional deviation, and to ensure the positioning accuracy of the terminal, the specific forms of the two weight matrices mentioned above can be configured as follows: ; .

[0059] In the example above: 1. Process weight matrix In the middle, heading angle The weighting coefficient (10.0) is significantly higher than the position weight (2.0), reflecting the emphasis on directional control during movement; 2. Terminal weight matrix All coefficients in the matrix are process weight matrices. The corresponding coefficient is 10 times. This parameter configuration ensures that when the robot reaches the target point, both the position error and the angle error converge to a very small range, thereby achieving precise docking.

[0060] The above implementation normalizes the deviations of different dimensions through a weight matrix, avoiding the confusion of physical meaning caused by direct addition. It gives the control system a flexible strategy optimization capability, allowing the weight of specific dimensions to be increased in a targeted manner according to different working conditions (such as high-precision centering in narrow spaces), thereby achieving more precise control.

[0061] Step 103: Determine the local theoretical Pareto front based on the energy consumption index and trajectory deviation index corresponding to all predicted trajectories.

[0062] In one implementation, the local theoretical Pareto front can be determined by the following methods: All predicted trajectories are mapped to a two-dimensional target space as coordinate values ​​to form a discrete set of target distribution points. The target distribution point set is filtered using the fast convex hull algorithm to extract the set of vertices that constitute the lower envelope of the target distribution point set; Interpolation is performed on adjacent vertices in the vertex set to generate a local theoretical Pareto front that characterizes the theoretical minimum energy consumption feasible boundary of the robot.

[0063] In this embodiment, the energy consumption index and trajectory deviation index corresponding to all predicted trajectories can be mapped to a two-dimensional target space as coordinate values ​​(one as the abscissa and the other as the ordinate) to form a discrete target distribution point set. Optionally, the trajectory deviation index can be used as the abscissa and the energy consumption index as the ordinate. Subsequently, the target distribution point set is filtered using the QuickHull algorithm to remove dominated non-optimal solutions and extract the set of vertices that constitute the envelope of the target distribution point set. Then, linear interpolation is performed on adjacent vertices in this vertex set to fit and generate a curve representing the theoretical minimum energy consumption limit under the dynamic model, i.e., the local theoretical Pareto front. The Pareto front of this local theory It describes the minimum energy cost required to achieve a specific trajectory deviation under ideal dynamics.

[0064] The optimization targets of existing technologies are usually preset fixed values ​​that cannot be dynamically adjusted according to the current physical limits. However, the embodiments of this application extract the lower envelope of the distribution formed by the energy consumption index and trajectory deviation index corresponding to the predicted trajectory, thereby fitting the local theoretical Pareto front and constructing a dynamic baseline that characterizes the limit of energy consumption and accuracy trade-off under dynamic theory. This ensures that the optimization target of the control system always fits the current physical capability boundary, rather than blindly approaching an unattainable ideal value.

[0065] To address the issue that a single optimization objective cannot balance energy consumption and accuracy, existing technologies often suffer from trade-offs. This application's embodiments quantify the physical performance across multiple dimensions, including mechanical work, smoothness penalty, process deviation, and terminal cost, by simultaneously calculating the energy consumption and trajectory deviation indices for each predicted trajectory. This provides comprehensive data support for finding the physical limits of energy consumption while considering multiple physical dimensions, avoiding suboptimal decisions caused by a single indicator.

[0066] Step 104: Determine the current control action of the robot corresponding to the current control cycle based on the robot's current control strategy.

[0067] In this embodiment, the robot's control strategy can be represented by a policy network based on reinforcement learning, which represents the sparse state vector that characterizes the robot's actual operating state in the current control cycle of the target task. Inputting this policy network allows the robot to output its current control action for the current control cycle. Current control action This indicates the command torque that needs to be applied to the robot's drive wheel for the current control cycle, and the current control action. Specifically, it can be a two-dimensional vector containing the left wheel command torque and the right wheel command torque.

[0068] Step 105: Based on the actual operating condition of the robot in the current control cycle of the target task and the robot's current control actions, deduce the predicted operating condition of the robot in the next control cycle of the target task.

[0069] In this step, the actual operating state of the robot in the current control cycle of the target task (i.e., the sparse state vector) can be obtained. ) and the robot's current control actions As input, these parameters are substituted into the dynamic model for deduction, yielding the predicted operating state of the robot in the next control cycle of the target task. It is important to note that This indicates the predicted operating condition for the next control cycle within the current control cycle, rather than the actual operating condition for the next control cycle. .

[0070] Step 106: Generate a shaping reward based on the distance of the robot's predicted operating condition in the next control cycle of the target task relative to the local theoretical Pareto front.

[0071] At the underlying motion control level, in order to guide the robot to always operate within the safety and energy efficiency boundaries, it is necessary to calculate the shaping return.

[0072] In one implementation, the integer return can be generated by the following methods: First, extract the first predicted trajectory with the highest prediction probability from multiple predicted trajectories derived from the robot's current position. Then, starting from the next predicted position of the robot's current position in the target predicted trajectory, multiple predicted trajectories corresponding to the next predicted position are deduced, and the second predicted trajectory with the highest prediction probability is extracted from the multiple predicted trajectories corresponding to the next predicted position. The energy consumption index and trajectory deviation index corresponding to the second predicted trajectory are determined, and two-dimensional evaluation points are formed, denoted as . ,in, This represents the trajectory deviation index corresponding to the second predicted trajectory. This represents the energy consumption index corresponding to the second predicted trajectory, a two-dimensional evaluation point. It can be placed within a two-dimensional target space composed of trajectory deviation index and energy consumption index; Then, the two-dimensional evaluation points Along the Pareto front of local theory The normal direction (i.e., perpendicular to the local theoretical Pareto front) The tangent direction is projected onto the local theoretical Pareto front. Above, the projection point is obtained; Two-dimensional evaluation points The Euclidean distance between the projection point and the boundary point is defined as the boundary distance. Boundary constraint penalty terms are constructed based on boundary distance to penalize behaviors that deviate from the safety and energy efficiency boundary; Determine the environmental antagonistic components And based on the current control action Compared to the robot's historical control actions in the previous control cycle of the target task. Changes and environmental resistance components An adaptive damping term is constructed to suppress control oscillations under harsh operating conditions; Finally, the boundary constraint penalty and adaptive damping term are deducted from the pre-set basic reward for the target task to obtain the shaped reward.

[0073] In this embodiment, the above process can be expressed as the following formula: in, This indicates the return on investment for cosmetic surgery. This indicates the pre-set base reward for the target task. This represents the preset boundary constraint penalty weight coefficient. Representing two-dimensional evaluation points To the Pareto Frontier of Local Theory Boundary distance, This represents the boundary constraint penalty term. This represents the preset adaptive damping weight coefficient. Indicates the environmental antagonistic component. This represents the square operation of the L2 norm. Indicates the current control action Historical control actions relative to the previous control cycle The square of the change, This represents the adaptive damping term.

[0074] In this embodiment, the robot's trajectory is predicted one more step, and the predicted trajectory with the highest probability is selected to punish behaviors that deviate from the safety and energy efficiency boundary. "Highest probability" usually corresponds to the average state of the standard working condition. If the most likely motion trajectory exceeds the safety and energy efficiency boundary, then a penalty should be imposed.

[0075] In one implementation, the environmental antagonistic component The determination method is as follows: Collect the feedback torque of the robot's drive wheels at the current moment. The nominal torque under the current working condition is obtained analytically based on the dynamic model. ; Calculate the feedback torque of the drive wheel With nominal torque The absolute value of the difference between them is used as the torque fluctuation; based on the torque fluctuation, the environmental resistance component is calculated using a saturated linear function. Torque fluctuation is used to characterize the degree of deviation between the actual driving load of the robot and the nominal load of the dynamic model.

[0076] It is important to note that the "theoretical drive wheel torque" in this embodiment does not refer to the rated torque of the drive motor (i.e., the fixed performance parameters of the motor at the factory), but rather to the theoretical reference torque calculated in real time by the control system based on dynamic principles under the current motion state. Specifically, the calculation of this theoretical drive wheel torque comprehensively considers the robot's inertial characteristics and standard damping characteristics. It refers to the theoretically required drive torque that the drive motor must output to drive the robot to generate the current angular acceleration and maintain the current rotational speed under preset standard ground friction coefficient and mechanical transmission damping conditions.

[0077] In one implementation, the theoretical driving wheel torque of the robot at the current moment... This can be determined in the following three steps: The first step is to collect motion data. The system uses sensors (such as encoders) to read the robot's current wheel speed and angular acceleration in real time. These two data points reflect how fast the robot is currently running and how rapidly it is accelerating.

[0078] The second step involves theoretical calculations based on the physical model, substituting the collected wheel speed and angular acceleration into the built-in dynamic equations. This calculation mainly consists of two parts: Calculate the moment of inertia: Based on the preset robot rotational inertia multiplied by the current angular acceleration, calculate the theoretical driving force that the motor needs to provide in order for the wheels to produce the current acceleration effect, that is, the force required for the robot to accelerate.

[0079] Calculate the damping torque: Based on the preset damping coefficient multiplied by the current wheel speed, calculate the theoretically required maintaining force of the motor to overcome bearing friction and air resistance in order to maintain the current speed, i.e. the force required for the robot to overcome friction.

[0080] The third step is to add the calculated "force required for acceleration" to the "force required to overcome friction" to obtain the theoretical driving wheel torque of the robot at the current moment under ideal road conditions and no collision interference. .

[0081] Based on this torque fluctuation, the environmental resistance component is calculated using the following saturated linear function. .

[0082] .

[0083] in, It is a saturated linear function. For torque fluctuation, Optionally, a preset dead zone threshold can be set. The value range can be 0.2 Nm to 0.5 Nm; Optionally, as the preset maximum permissible disturbance limit, The value range can be 2.0 Nm to 5.0 Nm.

[0084] Sparse rewards lead to slow convergence and a lack of safety constraints in deep reinforcement learning. This paper constructs a shaped reward by utilizing the projected distance of the predicted trajectory onto a local theoretical Pareto front (in this method, it is the local theoretical Pareto front; in the next method, it will be the safe reachable front). This forms a virtual potential field that pulls the system state toward the optimal energy efficiency boundary, providing a dense gradient signal with clear physical meaning for the control strategy. This solves the problem of the lack of a clear energy efficiency guidance mechanism in traditional methods.

[0085] Traditional methods struggle to suppress jitter while maintaining response speed in addressing control oscillations under harsh operating conditions. This paper proposes an adaptive damping term based on environmental adversarial components and the rate of change of control actions. When strong external disturbances are detected, the penalty weight in the reward function is automatically increased, forcing the control strategy to output smooth commands from the optimization objective level. This effectively prevents overheating and wear of the robot actuator under high dynamic loads.

[0086] Step 107: Adjust the control strategy for the robot based on the integer reward using a deep reinforcement learning algorithm, and drive the robot to move using the adjusted control strategy.

[0087] In this step, based on the integer reward, a deep reinforcement learning algorithm is used to update the robot's control strategy using gradients. Specifically, the integer reward is directly used as the weights to update the policy network. This update of the control strategy does not involve the robot "learning from scratch," but rather "fine-tuning based on the current situation," making the robot more adaptable to the current working environment. Afterwards, the updated control strategy can be used to output control commands (i.e., update the current control action) to drive the robot's movement, thereby controlling the robot to execute the motion process corresponding to the current control cycle.

[0088] Within each control cycle of the target task, the robot's control strategy can be adjusted in real time through the above process.

[0089] In one possible implementation, the robot control method further includes: After driving the robot to move using the adjusted control strategy, the average energy consumption deviation of the energy consumption index on the local theoretical Pareto front relative to the preset reference energy efficiency curve is determined. If the average energy consumption deviation exceeds the preset energy consumption deviation threshold, a rescheduling operation is performed for the target task. The rescheduling operation includes: reducing the target speed of the robot at the unreached waypoints in the discrete waypoint sequence, or rescheduling the discrete waypoint sequence to avoid the area where the robot is currently located.

[0090] In this embodiment, after driving the robot to move through the adjusted control strategy, the task scheduling can be rescheduled at the macro task scheduling level based on the robot's actual energy consumption performance.

[0091] At the macro-level task scheduling, to prevent robots from excessively consuming energy in low-energy-efficiency areas, rescheduling operations can be performed based on the target task. Specifically, this can be done within a preset trajectory deviation range. Within, determine the local theoretical Pareto front. Energy consumption indicators Compared to the preset reference energy efficiency curve The integral difference value, after being normalized, is determined as the average energy consumption deviation. This indicator This quantifies the additional energy consumption required under current road conditions, even with the optimal control strategy, compared to ideal road conditions. The calculation formula is as follows: .

[0092] in, For average energy consumption deviation, This is the upper limit of the preset trajectory deviation range. This is the lower limit of the preset trajectory deviation range. Pareto Frontier of Local Theory The trajectory deviation index on the surface Pareto Frontier of Local Theory Energy consumption indicators on This is a reference energy efficiency curve (characterizing energy efficiency under ideal operating conditions) formed by statistical analysis of historical operating data. The calculated average energy consumption deviation... When the preset energy consumption deviation threshold is exceeded, the underlying controller can send a rescheduling request to the task scheduling process. In response to the rescheduling request, the task scheduling process can perform any of the following rescheduling operations for the target task: reduce the target speed of the robot at unreached waypoints in the discrete waypoint sequence, or replan the discrete waypoint sequence to avoid the area where the robot is currently located (considered a low-energy-efficiency area or a high-energy-consumption area), thereby achieving global closed-loop optimization of the robot's task efficiency.

[0093] To address the disconnect between micro-control and macro-scheduling, existing technologies often fail to recognize the execution difficulty at the lower levels. The aforementioned implementation calculates the average energy consumption deviation of the local theoretical Pareto front (in this embodiment, it's the local theoretical Pareto front; in the next embodiment, it's the safe reachable front) relative to the ideal reference energy efficiency curve and feeds this deviation back to the task scheduling process. This establishes a reverse data channel from the lower physical execution layer to the upper task planning layer, enabling the scheduling system to dynamically adjust task allocation based on the actual energy efficiency difficulty of road conditions. This solves the problem of low global efficiency caused by unidirectional information flow in hierarchical systems. Furthermore, this implementation, combined with a task rescheduling mechanism triggered by energy consumption deviation feedback, achieves global coordination between micro-torque control and macro-path planning, maximizing operational efficiency while ensuring system robustness.

[0094] The robot control method provided in this embodiment extrapolates multiple different predicted trajectories within the robot's control cycle and determines the energy consumption index and trajectory deviation index corresponding to each predicted trajectory. Then, based on the energy consumption index and trajectory deviation index corresponding to all predicted trajectories, a local theoretical Pareto front is determined, thereby finding the lowest energy consumption solution set that balances multiple objectives (energy consumption and control accuracy). This achieves system modeling and joint constraints on energy consumption and task efficiency (task efficiency is affected by control accuracy). Based on the robot's actual operating condition in the current control cycle and the current control action output by the robot control strategy for the current control cycle, the predicted operating condition of the robot in the next control cycle can be extrapolated. Based on the distance of the predicted operating condition relative to the local theoretical Pareto front, an integer reward can be generated. Using this integer reward, the robot control strategy can be guided to always operate close to the lowest energy efficiency boundary, realizing a robot control strategy that smooths energy consumption and task efficiency.

[0095] In another embodiment of this application, such as Figure 2 As shown, the robot control method includes steps 201 to 209. Wherein: Step 201: Collect the actual operating status of the robot in the current control cycle of the target task.

[0096] The implementation of step 201 can refer to the implementation of step 101 in the previous method embodiment.

[0097] Step 202: Starting from the robot's current position, multiple different predicted trajectories are derived within a preset time domain in the future, and the energy consumption index and trajectory deviation index corresponding to each predicted trajectory are determined.

[0098] The implementation of step 202 can refer to the implementation of step 102 in the previous method embodiment.

[0099] Step 203: Determine the local theoretical Pareto front based on the energy consumption index and trajectory deviation index corresponding to all predicted trajectories.

[0100] The implementation of step 203 can refer to the implementation of step 103 in the previous method embodiment.

[0101] Step 204: Determine the operational risk quantification factor based on the prediction residual between the robot's actual state and the predicted state at the current moment, and the torque fluctuation between the robot's actual drive wheel torque and the theoretical drive wheel torque at the current moment.

[0102] In this embodiment, in order to eliminate the influence of physical model mismatch (internal factor) and external environmental disturbance (external factor) on the control system under the current operating conditions, the influence of these factors is first quantified to obtain the operational risk quantification factor. Then, the operational risk quantification factor is used to perform affine migration on the local theoretical Pareto front, thereby transforming the uncertainty and disturbance at the physical level into the geometric constraint boundary at the control level.

[0103] Specifically, in this step, the risk quantification factor includes the model confidence component. Combating the environment .

[0104] In one implementation, the model confidence component The determination method is as follows: within the current control cycle, read the sparse state vector. As the robot's actual state at the current moment (i.e., the measured value), the predicted state derived from the dynamic model for the current moment in the previous control cycle is retrieved. As the robot's predicted state (i.e., predicted value) at the current moment, calculate and The Euclidean distance between the two states is used as the prediction residual. This prediction residual characterizes the deviation between the robot's actual state at the current moment and the previously predicted state for the current moment.

[0105] Based on this prediction residual, the model confidence components are determined using the following negative exponential function. .

[0106] .

[0107] in, This represents the predicted residuals. This represents the noise variance of the dynamic model. Indicates the index of the current control cycle.

[0108] Environmental resistance The determination method can refer to step 106 of the previous method embodiment regarding the environmental adversarial component. The method of determination.

[0109] Step 205: Based on the operational risk quantification factor, perform affine shift on the energy consumption index dimension of the local theoretical Pareto front to generate a safe reachable front.

[0110] In this step, the model confidence components included in the operational risk quantification factor can be used as a basis. Combating the environment Pareto Front for Local Theory Affine shift is performed on the energy consumption index dimension to generate a safe reach frontier. Thus, through affine transformation, the local theoretical Pareto front is obtained. Corrections are made. The reason for choosing affine transformation for offset is that affine transformation preserves convexity and monotonicity, which can ensure that the generated safe reachable front does not undergo geometric distortion, thus guaranteeing the unique solution of subsequent projection calculations and the numerical stability of the control system.

[0111] In one implementation, it can be based on model confidence components. Construct a model uncertainty penalty term based on environmental adversarial components. Constructing a disturbance-resistant energy reserve term to utilize the local theoretical Pareto front Energy consumption indicators Linearly superimposed with the uncertainty penalty term and the disturbance resistance energy reserve term of the above model to obtain the local theoretical Pareto front. An affine transformation is performed on the energy consumption index dimension to generate a safe and achievable frontier. By incorporating model errors and environmental disturbances into the control objective constraint boundary, the robustness of the robot control strategy in real-world scenarios is improved.

[0112] For the Pareto frontier of local theory Any given trajectory deviation index Its safety can reach the forefront. On the safe energy consumption value The calculation is as follows: .

[0113] in, This is a penalty term for model uncertainty. This is the preset model uncertainty penalty coefficient. As a disturbance-resistant energy reserve, This is the preset disturbance rejection energy reserve coefficient.

[0114] If, on the local theoretical Pareto front, the energy consumption index is represented by the ordinate value and the trajectory deviation index by the abscissa value, then for each point on the local theoretical Pareto front, the energy consumption index (i.e., the ordinate value) can be calculated using this formula to obtain the ordinate value after affine migration, while the abscissa value remains unchanged.

[0115] By performing an affine shift on the energy consumption index dimension of the local theoretical Pareto front, uncertainties and disturbances at the physical level can be transformed into geometric constraint boundaries at the control level.

[0116] To address the control divergence problem caused by dynamic model mismatch, existing methods lack real-time assessment of the reliability of the dynamic model itself. This embodiment determines the confidence level of the dynamic model by calculating the prediction residuals, and then performs an affine transformation on the local theoretical Pareto front based on these residuals. This transforms the uncertainty in the dynamic model predictions into a geometric contraction of energy consumption constraints, automatically reserving more energy consumption margin when the dynamic model becomes distorted, thus solving the risk of control failure caused by the inconsistency between the dynamic simulation model and the real physical environment.

[0117] To address system instability caused by external environmental disturbances, simple feedback adjustment results in a delayed response. This embodiment calculates the degree of environmental resistance by collecting actual drive wheel torque and estimating theoretical drive wheel torque, and performs affine shift on the local theoretical Pareto front accordingly. This quantifies invisible external disturbances into visible geometric safety boundaries, forcing the control system to proactively select conservative strategies under harsh operating conditions, thus achieving feedforward defense against environmental disturbances.

[0118] Step 206: Determine the current control action of the robot corresponding to the current control cycle based on the robot's current control strategy.

[0119] The implementation of step 206 can refer to the implementation of step 104 in the previous method embodiment.

[0120] Step 207: Based on the actual operating condition of the robot in the current control cycle of the target task and the robot's current control actions, the predicted operating condition of the robot in the next control cycle of the target task is deduced.

[0121] The implementation of step 207 can refer to the implementation of step 105 in the previous method embodiment.

[0122] Step 208: Generate a shaping report based on the distance of the robot's predicted operating condition state in the next control cycle of the target task relative to the safe reach frontier.

[0123] In this embodiment, the local theoretical Pareto front is obtained through steps 204-205. Revised to be Safely Reachable Frontier Based on this, this step can generate integer returns through the following implementation methods, including: First, extract the first predicted trajectory with the highest prediction probability from multiple predicted trajectories derived from the robot's current position. Then, starting from the next predicted position of the robot's current position in the target predicted trajectory, multiple predicted trajectories corresponding to the next predicted position are deduced, and the second predicted trajectory with the highest prediction probability is extracted from the multiple predicted trajectories corresponding to the next predicted position. The energy consumption index and trajectory deviation index corresponding to the second predicted trajectory are determined, and two-dimensional evaluation points are formed, denoted as . ,in, This represents the trajectory deviation index corresponding to the second predicted trajectory. This represents the energy consumption index corresponding to the second predicted trajectory, a two-dimensional evaluation point. It can be placed within a two-dimensional target space composed of trajectory deviation index and energy consumption index; Then, the two-dimensional evaluation points Along the safe reach frontier The normal direction (i.e., perpendicular to the safe reach front) The tangential direction is projected onto the safe reach front. Above, the projection point is obtained; Two-dimensional evaluation points The Euclidean distance between the projection point and the boundary point is defined as the boundary distance. Boundary constraint penalty terms are constructed based on boundary distance to penalize behaviors that deviate from the safety and energy efficiency boundary; Determine the environmental antagonistic components And based on the current control action Compared to the robot's historical control actions in the previous control cycle of the target task. Changes and environmental resistance components An adaptive damping term is constructed to suppress control oscillations under harsh operating conditions; Finally, the boundary constraint penalty and adaptive damping term are deducted from the pre-set basic reward for the target task to obtain the shaped reward.

[0124] In this embodiment, the above process can be expressed as the following formula: in, This indicates the return on investment for cosmetic surgery. This indicates the pre-set base reward for the target task. This represents the preset boundary constraint penalty weight coefficient. Representing two-dimensional evaluation points To the Pareto Frontier of Local Theory Boundary distance, This represents the boundary constraint penalty term. This represents the preset adaptive damping weight coefficient. Indicates the environmental antagonistic component. This represents the square operation of the L2 norm. Indicates the current control action Historical control actions relative to the previous control cycle The square of the change, This represents the adaptive damping term.

[0125] Step 209: Adjust the control strategy for the robot based on the integer reward using a deep reinforcement learning algorithm, and drive the robot to move using the adjusted control strategy.

[0126] The implementation of step 209 can refer to the implementation of step 107 in the previous method embodiment.

[0127] In one possible implementation, the robot control method further includes: After driving the robot to move using the adjusted control strategy, the average energy consumption deviation of the energy consumption index on the safe reach front relative to the preset reference energy efficiency curve is determined. If the average energy consumption deviation exceeds the preset energy consumption deviation threshold, a rescheduling operation is performed for the target task. The rescheduling operation includes: reducing the target speed of the robot at the unreached waypoints in the discrete waypoint sequence, or rescheduling the discrete waypoint sequence to avoid the area where the robot is currently located.

[0128] In this embodiment, after driving the robot to move through the adjusted control strategy, the task scheduling can be rescheduled at the macro task scheduling level based on the robot's actual energy consumption performance.

[0129] At the macro-level task scheduling, to prevent robots from excessively consuming energy in low-energy-efficiency areas, rescheduling operations can be performed based on the target task. Specifically, this can be done within a preset trajectory deviation range. Inside, it is confirmed that safety is accessible to the frontier. Energy consumption indicators Compared to the preset reference energy efficiency curve The integral difference value, after being normalized, is determined as the average energy consumption deviation. This indicator This quantifies the additional energy consumption required under current road conditions, even with the optimal control strategy, compared to ideal road conditions. The calculation formula is as follows: .

[0130] in, For average energy consumption deviation, This is the upper limit of the preset trajectory deviation range. This is the lower limit of the preset trajectory deviation range. For safe access to the forefront The trajectory deviation index on the surface For safe access to the forefront Energy consumption indicators on This is a reference energy efficiency curve (characterizing energy efficiency under ideal operating conditions) formed by statistical analysis of historical operating data. The calculated average energy consumption deviation... When the preset energy consumption deviation threshold is exceeded, the underlying controller can send a rescheduling request to the task scheduling process. In response to the rescheduling request, the task scheduling process can perform any of the following rescheduling operations for the target task: reduce the target speed of the robot at unreached waypoints in the discrete waypoint sequence, or replan the discrete waypoint sequence to avoid the area where the robot is currently located (considered a low-energy-efficiency area or a high-energy-consumption area), thereby achieving global closed-loop optimization of the robot's task efficiency.

[0131] The robot control method provided in this embodiment deduces multiple different predicted trajectories within the robot's control cycle and determines the energy consumption index and trajectory deviation index corresponding to each predicted trajectory. Then, based on the energy consumption index and trajectory deviation index corresponding to all predicted trajectories, it determines the local theoretical Pareto front, thereby finding the lowest energy consumption solution set that balances multiple objectives (energy consumption and control accuracy). This achieves system modeling and joint constraints on energy consumption and task efficiency (task efficiency is affected by control accuracy). Based on the prediction residual between the robot's actual state and predicted state at the current moment (quantifying the internal factors of dynamic model adaptation) and the torque fluctuation between the robot's actual drive wheel torque and theoretical drive wheel torque at the current moment (quantifying the external factors of external environmental disturbances), the method further enhances the robot's control capabilities. By performing affine shift on the energy consumption index dimension of the local theoretical Pareto front, a safe reachable front is generated, thus forming a safety constraint on the minimum energy consumption solution set. This effectively prevents control divergence, enabling the robot to operate with low energy consumption under harsh conditions while ensuring safety. Based on the robot's actual operating condition in the current control cycle and the current control action output by the robot control strategy for the current control cycle, the predicted operating condition of the robot in the next control cycle can be deduced. Based on the distance of the predicted operating condition relative to the local theoretical Pareto front, an shaped reward can be generated. Using this shaped reward, the robot control strategy can be guided to always operate close to the minimum energy efficiency boundary, realizing a robot control strategy that smooths energy consumption and task efficiency.

[0132] Based on the same inventive concept, this application also provides a robot control device for implementing the robot control method described above. The solution provided by this device is similar to the solution described in the above method; therefore, the specific limitations in one or more robot control device embodiments provided below can be found in the limitations of the robot control method described above, and will not be repeated here.

[0133] In one exemplary embodiment, such as Figure 3 As shown, a robot control device is provided, which includes: The status acquisition module is configured to acquire the actual operating status of the robot in the current control cycle of the target task; The trajectory index determination module is configured to deduce multiple different predicted trajectories within a preset time domain starting from the robot's current position, and determine the energy consumption index and trajectory deviation index corresponding to each predicted trajectory. The boundary determination module is configured to determine the local theoretical Pareto front based on the energy consumption index and trajectory deviation index corresponding to all the predicted trajectories. The strategy output module is configured to determine the current control action of the robot corresponding to the current control cycle based on the robot's current control strategy. The state deduction module is configured to deduce the predicted operating state of the robot in the next control cycle of the target task based on the actual operating state of the robot in the current control cycle of the target task and the current control action of the robot. The reward generation module is configured to generate a shaped reward based on the distance of the robot’s predicted operating condition state in the next control cycle of the target task relative to the local theoretical Pareto front. The strategy adjustment driving module is configured to adjust the control strategy for the robot based on the integer reward using a deep reinforcement learning algorithm, and drive the robot to move using the adjusted control strategy.

[0134] Optionally, in some embodiments of this application, the robot control device is further used for: Based on the prediction residual between the robot's actual state and the predicted state at the current moment, and the torque fluctuation between the robot's actual drive wheel torque and the theoretical drive wheel torque at the current moment, the operational risk quantification factor is determined. Based on the aforementioned operational risk quantification factor, an affine shift is performed on the energy consumption index dimension of the local theoretical Pareto front to generate a safe and reachable front. The step of generating a shaping reward based on the distance of the robot's predicted operating condition state in the next control cycle of the target task relative to the local theoretical Pareto front includes: A shaping report is generated based on the robot's predicted operating condition in the next control cycle of the target task and its distance relative to the safe reach frontier.

[0135] Optionally, in some embodiments of this application, generating a shaping reward based on the distance of the robot's predicted operating condition state in the next control cycle of the target task relative to the safe reach frontier includes: From multiple predicted trajectories derived from the robot's current position, the first predicted trajectory with the highest prediction probability is extracted. Then, taking the next predicted position of the robot's current position in the target predicted trajectory as the starting point, multiple predicted trajectories corresponding to the next predicted position are deduced, and the second predicted trajectory with the highest prediction probability is extracted from the multiple predicted trajectories corresponding to the next predicted position. Determine the energy consumption index and trajectory deviation index corresponding to the second predicted trajectory to form a two-dimensional evaluation point; The two-dimensional evaluation points are projected onto the safe reachable frontier along the normal direction of the safe reachable frontier to obtain projection points; The distance between the two-dimensional evaluation point and the projection point is defined as the boundary distance; Construct a boundary constraint penalty term based on the boundary distance; Determine the environmental adversarial component, and construct an adaptive damping term based on the change of the current control action relative to the robot's historical control actions in the previous control cycle of the target task and the environmental adversarial component; The bounding constraint penalty and the adaptive damping term are deducted from the preset basic reward for the target task to obtain the shaped reward.

[0136] Optionally, in some embodiments of this application, the robot control device is further used for: Obtain the discrete waypoint sequence provided by the task scheduling for the target task, and determine the first unreached waypoint in the discrete waypoint sequence; Generate a smooth path from the robot's current position to the first unreached waypoint; Based on the displacement of adjacent points on the smooth path and a preset time interval, a local reference state sequence representing the reference state of the robot at each point on the smooth path is determined; The determination of the energy consumption index and trajectory deviation index corresponding to each predicted trajectory includes: For each predicted trajectory, a mechanical work term is generated based on the driving wheel command torque and wheel speed at each moment in the future preset time domain. A smoothness penalty term is generated based on the rate of change of the driving wheel command torque between adjacent moments in the future preset time domain. The weighted result of the mechanical work term and the smoothness penalty term is determined as the energy consumption index of the predicted trajectory. For each predicted trajectory, the process deviation is determined based on the deviation between the predicted trajectory's projected state at each moment in the future preset time domain and the reference state at the corresponding moment in the local reference state sequence. The terminal deviation is formed based on the deviation between the predicted trajectory's final projected state at the last moment in the future preset time domain and the robot's reference state at the first waypoint not reached in the local reference state sequence. The sum of the process deviation and the terminal deviation is determined as the trajectory deviation index of the predicted trajectory.

[0137] Optionally, in some embodiments of this application, the robot control device is further used for: After driving the robot to move using the adjusted control strategy, the average energy consumption deviation of the energy consumption index on the local theoretical Pareto front relative to the preset reference energy efficiency curve is determined. If the average energy consumption deviation exceeds a preset energy consumption deviation threshold, a rescheduling operation is performed for the target task; the rescheduling operation includes: reducing the target speed of the robot at unreached waypoints in the discrete waypoint sequence, or rescheduling the discrete waypoint sequence to avoid the area where the robot is currently located.

[0138] Optionally, in some embodiments of this application, determining the local theoretical Pareto front based on the energy consumption index and trajectory deviation index corresponding to all the predicted trajectories includes: All the energy consumption indicators and trajectory deviation indicators corresponding to the predicted trajectories are mapped as coordinate values ​​to a two-dimensional target space to form a discrete set of target distribution points. The target distribution point set is filtered using the fast convex hull algorithm to extract the set of vertices that constitute the lower envelope of the target distribution point set; Interpolation is performed on adjacent vertices in the vertex set to generate a local theoretical Pareto front that characterizes the theoretical minimum energy consumption feasible boundary of the robot.

[0139] In one exemplary embodiment, a robotic device is provided, including a memory and a processor, wherein the memory stores a program, and the processor executes the program to implement the steps in the above-described method embodiments.

[0140] The internal structure diagram of the above-mentioned robot equipment can be shown as follows: Figure 4As shown, the robot device includes a processor, memory, input / output (I / O) interfaces, and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, programs, and a database. The internal memory provides the environment for the operating system and programs stored in the non-volatile storage media. The database stores data. The I / O interfaces are used for exchanging information between the processor and external devices. The communication interface is used for communicating with external terminals via a network. When the program is executed by the processor, it implements a robot control method.

[0141] Those skilled in the art will understand that Figure 4 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the robot device to which the present application is applied. Specific robot devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0142] In one exemplary embodiment, a readable storage medium is provided storing a program that, when executed by a processor, implements the steps in the above-described method embodiments.

[0143] In one exemplary embodiment, a program product is provided, including a program that, when executed by a processor, implements the steps in the above-described method embodiments.

[0144] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant regulations.

[0145] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a non-volatile readable storage medium, and when executed, it can include the processes of the embodiments described above. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM).

[0146] The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0147] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0148] This document uses specific examples to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the methods and core ideas of this application. Furthermore, those skilled in the art will recognize that, based on the ideas of this application, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A robot control method characterized by, The robot control method includes: The actual operating status of the robot during the current control cycle of the target task is collected; Starting from the robot's current position, multiple different predicted trajectories within a preset future time domain are deduced, and the energy consumption index and trajectory deviation index corresponding to each predicted trajectory are determined. Based on the energy consumption index and trajectory deviation index corresponding to all the predicted trajectories, the local theoretical Pareto front is determined. The current control action of the robot corresponding to the current control cycle is determined by the robot's current control strategy; Based on the actual operating condition of the robot in the current control cycle of the target task and the current control action of the robot, the predicted operating condition of the robot in the next control cycle of the target task is deduced. The shaping reward is generated based on the distance of the robot's predicted operating condition state in the next control cycle of the target task relative to the local theoretical Pareto front. The control strategy for the robot is adjusted based on the integer reward using a deep reinforcement learning algorithm, and the robot is driven to move by the adjusted control strategy. The robot control method further includes: Based on the prediction residual between the robot's actual state and the predicted state at the current moment, and the torque fluctuation between the robot's actual drive wheel torque and the theoretical drive wheel torque at the current moment, the operational risk quantification factor is determined. Based on the aforementioned operational risk quantification factor, an affine shift is performed on the energy consumption index dimension of the local theoretical Pareto front to generate a safe and reachable front. The step of generating a shaping reward based on the distance of the robot's predicted operating condition state in the next control cycle of the target task relative to the local theoretical Pareto front includes: A shaping report is generated based on the distance of the robot's predicted operating condition state relative to the safe reach frontier in the next control cycle of the target task. The step of generating a shaped report based on the distance of the robot's predicted operating condition state in the next control cycle of the target task relative to the safe reach front includes: From multiple predicted trajectories derived from the robot's current position, the first predicted trajectory with the highest prediction probability is extracted. Then, taking the next predicted position of the robot's current position in the target predicted trajectory as the starting point, multiple predicted trajectories corresponding to the next predicted position are deduced, and the second predicted trajectory with the highest prediction probability is extracted from the multiple predicted trajectories corresponding to the next predicted position. Determine the energy consumption index and trajectory deviation index corresponding to the second predicted trajectory to form a two-dimensional evaluation point; The two-dimensional evaluation points are projected onto the safe reachable frontier along the normal direction of the safe reachable frontier to obtain projection points; The distance between the two-dimensional evaluation point and the projection point is defined as the boundary distance; Construct a boundary constraint penalty term based on the boundary distance; Determine the environmental adversarial component, and construct an adaptive damping term based on the change of the current control action relative to the robot's historical control actions in the previous control cycle of the target task and the environmental adversarial component; The boundary constraint penalty and the adaptive damping term are deducted from the preset basic reward for the target task to obtain the shaped reward. The robot control method further includes: Obtain the discrete waypoint sequence provided by the task scheduling for the target task, and determine the first unreached waypoint in the discrete waypoint sequence; Generate a smooth path from the robot's current position to the first unreached waypoint; Based on the displacement of adjacent points on the smooth path and a preset time interval, a local reference state sequence representing the reference state of the robot at each point on the smooth path is determined; The determination of the energy consumption index and trajectory deviation index corresponding to each predicted trajectory includes: For each predicted trajectory, a mechanical work term is generated based on the driving wheel command torque and wheel speed at each moment in the future preset time domain. A smoothness penalty term is generated based on the rate of change of the driving wheel command torque between adjacent moments in the future preset time domain. The weighted result of the mechanical work term and the smoothness penalty term is determined as the energy consumption index of the predicted trajectory. For each predicted trajectory, the process deviation is determined based on the deviation between the predicted trajectory's projected state at each moment in the future preset time domain and the reference state at the corresponding moment in the local reference state sequence. The terminal deviation is formed based on the deviation between the predicted trajectory's final projected state at the last moment in the future preset time domain and the robot's reference state at the first waypoint not reached in the local reference state sequence. The sum of the process deviation and the terminal deviation is determined as the trajectory deviation index of the predicted trajectory.

2. The robot control method according to claim 1, characterized by, The robot control method further includes: After driving the robot to move using the adjusted control strategy, the average energy consumption deviation of the energy consumption index on the local theoretical Pareto front relative to the preset reference energy efficiency curve is determined. If the average energy consumption deviation exceeds a preset energy consumption deviation threshold, a rescheduling operation is performed for the target task; the rescheduling operation includes: reducing the target speed of the robot at unreached waypoints in the discrete waypoint sequence, or rescheduling the discrete waypoint sequence to avoid the area where the robot is currently located.

3. The robot control method according to claim 1, wherein, The step of determining the local theoretical Pareto front based on the energy consumption index and trajectory deviation index corresponding to all the predicted trajectories includes: All the energy consumption indicators and trajectory deviation indicators corresponding to the predicted trajectories are mapped as coordinate values ​​to a two-dimensional target space to form a discrete set of target distribution points. The target distribution point set is filtered using the fast convex hull algorithm to extract the set of vertices that constitute the lower envelope of the target distribution point set; Interpolation is performed on adjacent vertices in the vertex set to generate a local theoretical Pareto front that characterizes the theoretical minimum energy consumption feasible boundary of the robot.

4. A robot control device characterized by comprising: The robot control device is used to implement the steps of the robot control method according to any one of claims 1-3, and the robot control device includes: The status acquisition module is configured to acquire the actual operating status of the robot in the current control cycle of the target task; The trajectory index determination module is configured to deduce multiple different predicted trajectories within a preset time domain starting from the robot's current position, and determine the energy consumption index and trajectory deviation index corresponding to each predicted trajectory. The boundary determination module is configured to determine the local theoretical Pareto front based on the energy consumption index and trajectory deviation index corresponding to all the predicted trajectories. The strategy output module is configured to determine the current control action of the robot corresponding to the current control cycle based on the robot's current control strategy. The state deduction module is configured to deduce the predicted operating state of the robot in the next control cycle of the target task based on the actual operating state of the robot in the current control cycle of the target task and the current control action of the robot. The reward generation module is configured to generate a shaped reward based on the distance of the robot’s predicted operating condition state in the next control cycle of the target task relative to the local theoretical Pareto front. The strategy adjustment driving module is configured to adjust the control strategy for the robot based on the integer reward using a deep reinforcement learning algorithm, and drive the robot to move using the adjusted control strategy.

5. A robotic device comprising: A memory, a processor, and a program stored in the memory and executable on the processor, characterized in that the processor executes the program to implement the steps of the robot control method according to any one of claims 1-3.

6. A readable storage medium, having stored thereon a program, characterized in that, When the program is executed by the processor, it implements the steps of the robot control method according to any one of claims 1-3.

7. A program product, comprising a program, characterized in that, When the program is executed by the processor, it implements the steps of the robot control method according to any one of claims 1-3.

Images