Robot control method, computer device and computer-readable storage medium

Abstract

The embodiment of the application discloses a robot control method, computer equipment and a computer readable storage medium. Wherein, the method comprises the following steps: in response to the operation instruction input by the user, the acquired robot data and the operation instruction are processed into feature data; determine the task type corresponding to the operation instruction; determine the task alignment degree corresponding to the feature data according to the task type; calculate the confidence of the operation instruction according to the task alignment degree; execute the operation instruction according to the confidence. Therefore, when the application can receive the operation instruction of the user, the robot can understand the control intention of the user, the safety and performance risk caused by human error operation is reduced, the dependence on high-level operators is also reduced, and the universality of takeover is improved. The confidence mechanism realizes the "consensus control" of man-machine, and improves the success rate of complex tasks.

Description

Technical Field

[0001] This application belongs to the field of robot control technology, and in particular relates to a robot control method, computer equipment, and computer-readable storage medium. Background Technology

[0002] Humanoid robots are increasingly being applied in manufacturing, logistics, and service industries. During task execution, due to environmental complexity and model limitations, robots often trigger abnormal scenarios. To ensure task closure and safety, the mainstream approach is to use a remote takeover system, allowing a human operator to remotely intervene and control the robot. This mechanism is widely used in autonomous driving systems (such as Waymo), quadruped robots (such as Boston Dynamics), and service robots. Traditional remote takeover mechanisms are mainly divided into two types: 1. Complete takeover: The operator gains low-level control over the robot and controls its actions in real time. 2. High-level assistance: The operator assists the robot in completing tasks through commands, instructions, path planning, etc.

[0003] In existing systems, human intervention completely dominates the control flow. The system does not judge the rationality of the operation, nor does it filter or optimize the instructions; it directly executes the human takeover operation. Especially in high-degree-of-freedom, multi-dimensional control scenarios for humanoid robots, operator errors in instructions or delayed decisions can lead to execution failures or even mechanical damage. How to improve the safety of human takeover operations is a technical problem that urgently needs to be solved by those skilled in the art.

[0004] The preceding description is intended to provide general background information and does not necessarily constitute prior art. Summary of the Invention

[0005] The purpose of this application is to provide a robot control method, computer device, and computer-readable storage medium that can effectively improve the safety and efficiency of robot operation.

[0006] To achieve the above objectives:

[0007] In a first aspect, embodiments of this application provide a robot control method, comprising the following steps: responding to an operation command input by a user, processing acquired robot data and the operation command into feature data; determining the task type corresponding to the operation command; determining the task alignment degree corresponding to the feature data based on the task type; calculating the confidence degree of the operation command based on the task alignment degree; and executing the operation command based on the confidence degree.

[0008] In an optional embodiment of this application, determining the task type corresponding to the operation instruction includes: obtaining the robot's current task, splitting the current task into at least one sub-task, and determining the task type corresponding to the operation instruction based on the matching relationship between the sub-task and the operation instruction.

[0009] In an optional embodiment of this application, determining the task alignment degree corresponding to the feature data according to the task type includes: calculating the spatial distance alignment degree and the action semantic alignment degree respectively based on the feature data; determining the weighting coefficients corresponding to the spatial distance alignment degree and the action semantic alignment degree respectively based on the task type; and performing weighted fusion of the spatial distance alignment degree and the action semantic alignment degree according to the corresponding weighting coefficients to obtain the task alignment degree.

[0010] In an optional embodiment of this application, calculating the spatial distance alignment based on feature data includes: obtaining the target position and the current position in the feature data; and calculating and determining the spatial distance alignment based on the target position, the current position, and a preset maximum reference distance.

[0011] In an optional embodiment of this application, calculating the action semantic alignment degree based on feature data includes: obtaining action data from the feature data; calculating the current action feature vector based on the action data; matching the template action data corresponding to the action data; calculating the template action feature vector based on the template action data; and calculating the similarity between the current action feature vector and the template action feature vector as the action semantic alignment degree.

[0012] In an optional embodiment of this application, the task type includes mobile tasks; determining the task alignment degree corresponding to the feature data according to the task type includes: when the task type is a mobile task, acquiring the robot's movement data; correcting the task alignment degree according to the movement data and a preset penalty coefficient, and using the corrected task alignment degree as the final task alignment degree under the mobile task.

[0013] In an optional embodiment of this application, calculating the confidence level of the operation command based on the task alignment includes: calculating the motion imbalance compensation coefficient based on the robot's motion imbalance data; calculating the environmental impact compensation coefficient based on the robot's environmental impact data; calculating the motion smoothing compensation coefficient based on the operation command; weighting and summing the motion imbalance compensation coefficient, the environmental impact compensation coefficient, and the motion smoothing compensation coefficient to obtain the comprehensive compensation degree; and calculating the confidence level based on the preset compensation coefficient, the comprehensive compensation degree, and the task alignment.

[0014] In an optional embodiment of this application, executing an operation instruction based on a confidence level includes at least one of the following: if the confidence level is higher than a first threshold, then controlling the robot to perform the corresponding operation according to the operation instruction; if the confidence level is lower than or equal to the first threshold and greater than or equal to a second threshold, then issuing a warning to the user and controlling the robot to perform the corresponding operation according to the operation instruction; if the confidence level is lower than the second threshold, then pausing the execution of the operation instruction and sending a reconfirmation request to the user, and controlling the robot to execute or terminate the operation according to the user's feedback.

[0015] Secondly, embodiments of this application provide a computer device, including: a processor and a memory storing a computer program, wherein when the processor runs the computer program, the steps of the above-described method are implemented.

[0016] Thirdly, embodiments of this application provide a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the above-described method.

[0017] The embodiments of this application have the following beneficial effects:

[0018] The method of this application includes the following steps: responding to user-inputted operation commands, processing acquired robot data and operation commands into feature data; determining the task type corresponding to the operation command; determining the task alignment degree corresponding to the feature data based on the task type; calculating the confidence degree of the operation command based on the task alignment degree; and executing the operation command based on the confidence degree. Therefore, this application can, upon receiving a user's operation command, enable the robot to understand the user's control intention through a comprehensive judgment of the operation command, task type, and robot data, thereby possessing the ability of "autonomous judgment + collaborative control" during remote takeover response. Ultimately, the response is based on the confidence degree of the operation command, rather than directly controlling according to the operation command, thereby reducing the safety and performance risks caused by human error. Furthermore, control through confidence degree also reduces reliance on highly skilled operators and improves the universality of takeover. The confidence degree mechanism achieves human-machine "consensus control," improving the success rate of complex tasks. Simultaneously, for the robot, it reduces reliance on operators during remote takeover, realizing the robot's autonomy and intelligence.

[0019] The above description is merely an overview of the technical solution of this application. In order to better understand the technical means of this application and to implement it according to the contents of the specification, and to make the above and other objects, features and advantages of this application more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings. It should be understood that the above general description and the following detailed description are merely exemplary and explanatory, and do not limit this application. Attached Figure Description

[0020] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0021] Figure 1 This is a flowchart illustrating a robot control method provided in one embodiment.

[0022] Figure 2 This is a schematic block diagram of the structure of a computer device provided in one embodiment. Detailed Implementation

[0023] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. In the following description relating to the drawings, unless otherwise indicated, the same numerals in different drawings denote the same or similar elements.

[0024] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element. Furthermore, components, features, and elements with the same names in different embodiments of this application may have the same meaning or different meanings, the specific meaning of which must be determined by its interpretation in that specific embodiment or further in conjunction with the context of that specific embodiment.

[0025] It should be understood that although the terms first, second, third, etc., may be used herein to describe various information, such information should not be limited to these terms. These terms are used only to distinguish information of the same type from one another. For example, without departing from the scope of this document, first information may also be referred to as second information, and similarly, second information may also be referred to as first information. Depending on the context, the word "if," as used herein, can be interpreted as "when," "when," or "in response to determination." Furthermore, as used herein, the singular forms "a," "an," and "the" are intended to also include the plural forms unless the context indicates otherwise. It should be further understood that the terms "comprising," "including," indicate the presence of the stated feature, step, operation, element, component, item, kind, and / or group, but do not exclude the presence, occurrence, or addition of one or more other features, steps, operations, elements, components, items, kinds, and / or groups. The terms "or" and "and / or" as used herein are to be interpreted as inclusive, or mean any one or any combination thereof. Therefore, "A, B, or C" or "A, B, and / or C" means "any one of the following: A; B; C; A and B; A and C; B and C; A, B, and C". Exceptions to this definition will only occur if the combination of elements, functions, steps, or operations is inherently mutually exclusive in some way.

[0026] It should be understood that although the steps in the flowcharts of this application's embodiments are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some of the steps in the figures may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be performed alternately or in turn with other steps or at least a portion of the sub-steps or stages of other steps.

[0027] It should be noted that step designations such as S110 and S120 are used in this document for the purpose of more clearly and concisely describing the corresponding content, and do not constitute a substantial limitation on the order. In specific implementation, those skilled in the art may execute S120 first and then S110, etc., but these should all be within the protection scope of this application.

[0028] It should be understood that the specific embodiments described herein are merely illustrative of this application and are not intended to limit this application.

[0029] In the following description, the use of suffixes such as "module," "part," or "unit" to denote elements is solely for the purpose of illustrative purposes and has no specific meaning in itself. Therefore, "module," "part," or "unit" may be used interchangeably.

[0030] To facilitate understanding of this application, the following explanations are provided for the terms and technical terms that may be used in this application:

[0031] Humanoid robots are robots with human body structures (such as head, limbs, and joints) and similar human behavioral abilities. The motion module, which is the hardware that responds to user commands, consists of at least two parts: an upper-body robotic arm unit and a lower-body movement unit. The robotic arm unit mimics a human arm and can perform rotation, extension, or grasping movements within a certain range to complete specific tasks. The movement unit controls the overall movement of the robot, enabling it to perform more complex tasks such as carrying and transferring.

[0032] Teleoperation refers to the process by which a remote operator accesses and controls a robot's behavior based on abnormal events during task execution. It's the process of a human operator controlling a robot or mechanical device remotely via a network or communication system. Simply put: the operator issues commands from location A, and the robot executes the actions from location B. This is achieved by the operator side inputting actions (via controllers, VR devices, exoskeletons, etc.); the robot side executing and feeding back data (video, force feedback, status); and transmitting control commands and sensory information via a communication link.

[0033] Remote assistance: The operator does not directly control the robot, but only provides suggestions, path selection, or task solutions—high-level strategies. Remote assistance differs from "remote control" in that it emphasizes that the robot and human collaborate to complete the task, with the human providing guidance or correction only at key stages, rather than complete control. For example, when a robot encounters uncertainty while performing a maintenance task, it sends a distress signal to the operator. Upon receiving the distress signal, the operator provides suggestions via remote video or an AR interface. The robot then continues to complete the task based on instructions and its own judgment.

[0034] Operation Confidence: A comprehensive quantitative score given by the system regarding the reasonableness, effectiveness, and probability of success of human operator commands in the current context. The value ranges from 0 to 1, where 1 indicates complete confidence and 0 indicates complete uncertainty. Data sources for calculation can include: the reliability of perceived data (visual recognition confidence), the accuracy of action execution (mechanical errors, feedback stability), and environmental uncertainties (obstacles, lighting, interference), etc. It is used to dynamically adjust the ratio of autonomous to manual control of the robot, or to initiate remote assistance mechanisms (requesting human intervention when confidence is too low), and can also be used for decision fusion (combining the confidence scores of multiple sensors and multiple models for weighted calculation).

[0035] Intention understanding: Based on its current state, the robot infers possible goals and behavioral paths to determine whether the operator's input is consistent with the intended action. Simply put, it lets the robot know not only what you "said" or "did," but also what you "wanted to achieve."

[0036] In existing technologies, during robot task execution, due to environmental complexity and model limitations, abnormal scenarios often occur, requiring human intervention. This necessitates remote control, where the robot is remotely controlled to perform corresponding actions to complete the task. However, existing robots often lack dynamic evaluation of the rationality of operations when executing commands issued by operators. They may directly execute commands, potentially leading to failure or even system damage due to misoperation or environmental changes. Alternatively, inexperienced or incapable operators may hesitate in complex scenarios, repeatedly adjusting their posture, wasting time and potentially causing mechanical wear. Therefore, to improve the efficiency of remote takeover control of robots, this application proposes a robot control method, including steps S110-S150. For a clear description of the method provided in this embodiment, please refer to... Figure 1 .

[0037] This application provides a robot control method, which can be executed by a computer device provided in this application. This device can be implemented using software and / or hardware. In this embodiment, the executing entity of the method can be a program module deployed in a robot controller; it can also be a remote server acting as the executing entity, issuing control commands after determining the operation instructions; or it can be a distributed system where the server and the robot controller operate collaboratively. For specific execution, this application uses a robot as the executing entity in a preferred embodiment. Specific execution methods requiring server intervention will be described in detail later.

[0038] Step S110: In response to the user's input operation command, process the acquired robot data and operation command into feature data.

[0039] In one implementation, during task execution, due to environmental complexity and model limitations, the robot often triggers abnormal scenarios requiring human intervention. Specific situations may include, but are not limited to, the robot becoming stuck and unable to move, the robotic arm's movement being obstructed, or the sensing system losing its target. In such cases, the robot typically sends a distress signal to the host computer or control system, requesting human intervention. Upon receiving the signal, the user, represented by the operator, will remotely take over the robot, inputting operational commands to guide it out of the abnormal state or to complete the interrupted task. Therefore, the operational commands are the remote control signals issued by the user to resolve the current abnormal scenario, specifically including data such as handle / motion capture / joint angle input.

[0040] Upon receiving an operation command, robot data can be acquired simultaneously. This data can include two categories: robot body data and environmental perception data. Robot body data refers to the data that the robot itself can collect, representing its current state. This includes information such as pose, joint states, movement speed, and end effector feedback. Specifically, it may include, but is not limited to, pose information, joint angles, speeds, torques, and operational status data from the end effector, IMU (Inertial Measurement Unit), and force sensors. Environmental perception data originates from various sensors on the robot, such as depth cameras, LiDAR, and ultrasonic sensors. It reflects the state of the robot's surrounding environment and may include, but is not limited to, perception results such as obstacle distribution, visual point clouds, depth maps, RGB images, and semantic segmentation maps.

[0041] The operation instructions and robot data are used as inputs and normalized to eliminate the influence of different units and data ranges, thereby obtaining feature data for subsequent analysis and processing. The normalization process can be referenced in the following formula.

[0042] (1)

[0043] In the above formula, These are the normalized feature data; It is the first in the original feature matrix Frame, First The original value of the dimensional feature, It is the first The mean of the dimensional feature over the entire time series. It is the first The standard deviation of the three features over the entire time series can be obtained directly from operation instructions or robot data. The determination process is existing technology and will not be elaborated here. By normalizing multiple data points into feature data, it is ensured that subsequent calculations will not be biased by differences in the numerical ranges of data from different sensors.

[0044] Step S120: Determine the task type corresponding to the operation instruction.

[0045] In one embodiment, step S120: determining the task type corresponding to the operation instruction includes: obtaining the robot's current task, splitting the current task into at least one sub-task; and determining the task type corresponding to the operation instruction based on the matching relationship between the sub-task and the operation instruction.

[0046] In one implementation, the task being performed by the robot can be divided into multiple sub-tasks. Simultaneously, the operation instructions target a specific sub-task, such as controlling the robot to navigate through obstacles or controlling the robot's robotic arm to perform a grasping action. Different task types will have different impacts on subsequent task alignment calculations. Assume the robot's current task is to move from its current position to point A to grasp a target object and transport it to point B. This task can be broken down into four sub-tasks: "moving to point A," "grabbing the object," "transporting to point B," and "placing the object." It can be seen that each sub-task performs different actions and requires different operation instructions. For example, "moving to point A" requires navigation-related instructions, while "grabbing the object" relies on robotic arm operation instructions. By analyzing the content of the operation instructions, it can be determined which sub-task stage it belongs to, thus identifying the corresponding task type. Through long-term observation and data collection of operations in industrial scenarios, most tasks in industrial scenarios can be abstracted into three main categories: "movement," "performing actions," and "moving while performing actions." This task type can also be divided into three categories: movement tasks, action tasks, and parallel action tasks.

[0047] Motion-related tasks, or "MOVE" tasks, aim to reach a specific spatial location, such as "walking to the shelf." These typically involve the robot's lower body and the movement of its mobile units. Motion-related tasks primarily focus on locomotion / navigation, moving the robot from one location to another rather than performing actions on the environment. Simply put, the robot's primary concern is "where to go," not "what to do." Mobile units can include, but are not limited to: wheeled movement (AGVs, AMRs), tracked movement (engineering or rescue robots), legged movement (humanoid robots, quadruped robots), hovering / flying movement (drones), and rail-based movement (assembly or welding robots on assembly lines). For these, only distance is considered, without limitation. Motion-related tasks may encounter problems such as inaccurate positioning, changes in the external environment causing the robot to get stuck in obstacles, or path conflicts between robots leading to congestion. In these situations, a request can be sent to the user for remote takeover to resolve the issue.

[0048] Action tasks, or "ACTION" tasks, aim to perform specific actions, such as "grabbing a box" or "placing an object." These typically involve the robot's upper body and the individual movement of its motion units. Action tasks refer to tasks where the robot manipulates objects in its environment using its robotic arm or end effector. The core is the "precision of the hand or end effector's movements and operational strategies." Simply put, it's about what the robot "does," not "where it goes." A specific task could be, for example, a robotic arm grasping an object on a conveyor belt. Achieving this requires target recognition, grasp point estimation, and six-degree-of-freedom control. The challenges lie in handling changes in object posture, the effects of lighting, and grasp stability. Action tasks may fail to complete due to factors such as perception errors, force control failures, object deviation, or failure to meet expected conditions. For example, an object recognition error might lead to grasping the wrong object, or the robot might refuse to grasp the recognized object. In such cases, user intervention can help complete the task.

[0049] Parallel motion tasks, also known as "MOVE+ACTION," aim to perform actions while moving, such as "carrying and walking simultaneously," which involves the robot's whole-body coordinated movements. Parallel motion tasks refer to robots possessing both mobility and maneuvering capabilities, with both occurring in parallel and coordinated during task execution. Simply put, the robot "works while moving," needing to both "go where" and "do what." This is the most complex task type, integrating: navigation control of the mobile platform (lower-level movement), motion control of the robotic arm (upper-level manipulation), and coordination between the two (kinematic coupling). For example, when the robot chassis is moving, the robotic arm must still keep the end effector aligned with the target position or perform the operation; the system must compensate for attitude changes caused by platform movement in real time. Therefore, for the problems encountered in the aforementioned mobile or motion tasks, a problem in any of these aspects will lead to the need for remote assistance from the robot.

[0050] Determining the task type will affect the subsequent calculation of task alignment. This classification clarifies the importance of "lower body (position movement)" and "upper body (action execution)" at different task stages, providing a basis for the weighted calculation of intentional task alignment. The specific ways in which this influences the task alignment will be explained later when calculating task alignment; please refer to the following text for details.

[0051] Step S130: Determine the task alignment degree corresponding to the feature data based on the task type.

[0052] In one embodiment, step S130: determining the task alignment degree corresponding to the feature data according to the task type includes: calculating the spatial distance alignment degree and the action semantic alignment degree respectively according to the feature data; determining the weighting coefficients corresponding to the spatial distance alignment degree and the action semantic alignment degree respectively according to the task type; and performing weighted fusion of the spatial distance alignment degree and the action semantic alignment degree according to the corresponding weighting coefficients to obtain the task alignment degree.

[0053] In one implementation, the subsequent step is to understand the operational intent of the user's issued commands, which is quantified in this application as task alignment. The calculation of task alignment is directly related to the task type, operational commands, and robot data, the latter two of which are normalized into feature data. Task alignment can be specifically divided into two dimensions: spatial distance alignment and action semantic alignment. The former measures the proximity between the robot's current position and the target position, while the latter reflects the degree of matching between the current action and the command semantics; both will be calculated separately.

[0054] In one embodiment, calculating the spatial distance alignment based on feature data includes: acquiring the target position and the current position from the feature data; and calculating and determining the spatial distance alignment based on the target position, the current position, and a preset maximum reference distance.

[0055] In one embodiment, spatial distance alignment is used to measure the proximity between the current position and the target position when controlling the robot's movement or the movement of the robotic arm. The calculation requires both the target position and the current position, which can be directly obtained from feature data. This position can be either the robot's position or the robotic arm's position, specifically determined based on the operation instructions and task type, and is not limited thereto. The calculation method for spatial distance alignment can be found in the following formula.

[0056] (2)

[0057] In the above formula, Spatial distance alignment. This is a Euclidean function used to calculate the Euclidean distance between two locations. Current position For the target location; This is the preset maximum reference distance, used to normalize the spatial distance and ensure that the alignment value range is within [0,1]. The smaller the relative distance between the target position and the current position, the higher the spatial distance alignment. The smaller the value, the better.

[0058] In one embodiment, calculating the action semantic alignment degree based on feature data includes: acquiring action data from the feature data; calculating the current action feature vector based on the action data; matching the action data with template action data corresponding to the action data; calculating the template action feature vector based on the template action data; and calculating the similarity between the current action feature vector and the template action feature vector as the action semantic alignment degree.

[0059] In one implementation, the semantic alignment of actions can be determined by comparing the action data corresponding to the operation command with a defined action template. It is understood that robots execute actions based on corresponding action templates. When a robot encounters an unsolvable anomaly, meaning the existing action template cannot cover the action requirements of the current scenario, alignment with the action template can help understand the user's intended action. This not only allows the robot to learn and expand the action template so that it can handle similar difficulties later, but also enables it to proactively identify intention deviations and provide optimization suggestions when the user hesitates or repeatedly executes the operation command, thereby improving the smoothness and accuracy of task execution.

[0060] It is also understandable that the actions executed by robots under control by operational instructions are complex and diverse. To address the recognition challenges of long, variable-speed, and segmented actions in industrial scenarios, this application employs a two-layer hybrid similarity calculation method: a real-time layer and a verification layer. The real-time layer, deployed within the robot, can quickly match corresponding action templates and perform semantic alignment calculations with low latency, keeping the calculation time within milliseconds to ensure real-time response. The verification layer, deployed in the cloud, is responsible for in-depth comparison and optimization analysis of complex action sequences and is also used for subsequent accuracy alignment and template updates.

[0061] In this embodiment, a real-time layer is described as a preferred implementation. The real-time layer can satisfy the semantic alignment calculation of actions in most scenarios. Generally, the real-time layer slices the continuous skeletal joint sequence or upper body motion signal of the robot in the operation command according to a multi-scale sliding window (e.g., 0.5 seconds, 1 second, and 2 seconds, with a step size of 0.1 seconds). The data of each window is processed by a lightweight temporal feature extraction network and mapped to a fixed-dimensional feature vector using a Temporal Convolutional Network (TCN). The cosine similarity of this feature vector with the feature vector corresponding to the task action template is calculated to obtain the action similarity of the current window. Moving average and trend detection are performed on the similarity of multiple consecutive windows to determine whether the operator's actions are gradually approaching the template actions, and the intent alignment is calculated.

[0062] The specific calculation process begins by acquiring motion data from the feature data, and then calculating the current motion feature vector based on this motion data. Motion data is obtained directly from operation commands and represents the actions that the robot needs to perform in actual control. Furthermore, due to the limited computing power of the robot itself, only the most critical motion data is used to calculate the temporal motion vector. The temporal motion vector calculation process is as follows.

[0063] (3)

[0064] In the above formula, This is a timing action vector. For the first Each joint at any time Angle value (radians or degrees); For the end effector at time The three-dimensional position vector, in meters; For the end effector at time The three-dimensional orientation of a can be represented by Euler angles; The joint angular velocity is expressed in radians per second. The acceleration is the joint angular velocity, expressed in radians per second². This refers to the grasping state of the hand or gripper, indicating the opening / closing angle or grasping force, measured in Newtons or degrees. All of this data is included in the motion data and can be obtained directly.

[0065] Subsequently, the sliding window and feature representation are used to determine the sliding window matrix of the action data. The calculation process is as follows.

[0066] (4)

[0067] In the above formula, A sliding window matrix for motion data; The feature dimension for a single frame is: number of joint angles + position dimension + pose dimension + velocity / acceleration + total number of gripping states. The window length, also known as the number of sampling points, is usually equal to the window duration multiplied by the sampling frequency. For window span, equal to .

[0068] Then the sliding window matrix The mapping is a fixed-dimensional quantity, and the calculation process is shown in the following formula.

[0069] (5)

[0070] In the above formula, This is the mapped feature vector of the current action, with a fixed dimension; The mapping function is a preset function and is related to the convolutional network used in the real-time layer. For example, in a preferred embodiment of this application, a temporal convolutional network is used, and the mapping function is determined by the temporal convolutional network. m The feature dimension after mapping can be set to 64 in a preferred embodiment of this application.

[0071] Subsequently, matching template motion data can be determined based on the motion data. It can be understood that the motion data generated by the operation commands is used to control the robot to perform corresponding operations, which may have a matching relationship with existing motion templates in the robot's database. A preset retrieval algorithm can be used to match the motion data with the motion templates in the database, thereby determining the closest template motion data. The data content contained in the template motion data is consistent with the motion data, including joint angle sequences, end-effector pose trajectories, velocity and acceleration curves, and temporal information such as gripping force changes. Therefore, the calculation process of the template motion feature vector is the same as calculating the current motion feature vector based on the motion data. The process is the same, and please refer to the previous text for details, so I will not repeat it here. For the calculated template action feature vector, denoted as... .

[0072] Calculate the current action feature vector With template action feature vector The similarity is used as the semantic alignment of the action. The calculation process can be to first calculate the current action feature vector. With template action feature vector The cosine similarity between them is calculated using the following formula.

[0073] (6)

[0074] In the above formula, The current action feature vector With template action feature vector Cosine similarity between them, with a range of values ​​[ [1,1], the larger the value, the more similar the two sides are; Let Euclidean norm be the vector. Cosine similarity is used to... Next, calculate and determine the semantic alignment of the action. The calculation method is as follows.

[0075] (7)

[0076] In the above formula, This represents the semantic alignment of the action, with a value range of [0,2]. The smaller the value, the more the action conforms to the template.

[0077] Complete spatial distance alignment Alignment with action semantics Then, the task alignment can be determined based on the task type. As mentioned earlier, different task types will affect the spatial alignment. Alignment with action semantics The weighting coefficients have an impact. Spatial distance alignment. The weighting coefficients are called spatial weighting coefficients, denoted as Action semantic alignment The weighting coefficients are called action weighting coefficients, denoted as Spatial weighting coefficients Action weighting coefficient Different weights can be set according to different task types, but at the same time, the two must satisfy the corresponding constraints, which are as follows.

[0078] (8)

[0079] To facilitate understanding, the weighting coefficient configurations for the three task types are given below. In mobile tasks, the spatial weighting coefficient... Set to 0.9, action weighting coefficient Setting it to 0.1 emphasizes the priority of path matching; in action-based tasks, the spatial weighting coefficient... Set to 0.3, action weighting coefficient Setting it to 0.7 emphasizes the accuracy of action semantics; in parallel action-type tasks, both are set to 0.5 to achieve a balance between spatial and action considerations. The purpose of this step is that when a subtask is marked as a movement task, the spatial distance alignment is considered when calculating intent alignment. It should account for a larger proportion, action semantic alignment. It should account for a smaller proportion; as long as the operator is controlling the robot to move towards the destination and shorten the distance, any movement of the upper body is permissible to a certain extent. Similarly, when a subtask is tagged as an action class, the semantic alignment of the action should be considered. The proportion is larger, and the spatial distance alignment is higher. The weighting is smaller; when a subtask is marked as a parallel action, the weights are equal. It is understandable that the above weighting configuration can be dynamically adjusted according to the task type to ensure that the intent alignment calculation is more context-adaptive. In other words, the values ​​listed above are illustrative examples of the solution, not limitations on it.

[0080] Alignment based on spatial distance Action semantic alignment Spatial weighting coefficient Action weighting coefficient Intent alignment can be calculated; the calculation process is as follows.

[0081] (9)

[0082] In the above formula, Intended alignment. Based on intended alignment. Next, calculate and determine the task alignment. The calculation process is as follows.

[0083] (10)

[0084] In the above formula, For task alignment; This is a preset numerical constraint function, which determines the task alignment. The value is limited to between 0 and 1. Task alignment. It is used to indicate the intensity and effectiveness of the intention to advance an operational task, and to measure whether the operator's actions are moving towards the task objective. The higher the value, the clearer the operational intention and the more effective the execution.

[0085] In one embodiment, the task type includes mobile tasks; determining the task alignment degree corresponding to the feature data according to the task type includes: when the task type is a mobile task, acquiring the robot's movement data; correcting the task alignment degree according to the movement data and a preset penalty coefficient, and using the corrected task alignment degree as the final task alignment degree under the mobile task.

[0086] In one implementation, as described above, there are three types of task types. For mobile tasks, the initially calculated task alignment degree... Further adjustments are needed. In this case, it's necessary to acquire the robot's movement data, which can be included in the feature data. Specifically, the movement data used for this calculation includes the robot's current chassis speed. Regarding task alignment... For details, please refer to the following formula.

[0087] (11)

[0088] In the above formula, The corrected task alignment; Preset penalty coefficient; This represents the current chassis speed. The initial threshold for punishment, This refers to the speed normalization scaling factor; both are preset values. In mobile tasks, this relates to task alignment. Based on the current chassis movement speed Making corrections can prevent operators from accidentally pushing the robot when it should be stationary, thus ensuring safety and accuracy.

[0089] In one embodiment, as described above, in a preferred embodiment of this application, regarding the semantic alignment of actions... The process involves matching motion data with motion templates, and then using both the motion data and template motion data for calculation. However, considering the diverse possibilities for achieving the motion objective, in reality, there are instances where motion data cannot match the motion template, or the matching result is very poor, leading to issues with the final task alignment. In cases where significant discrepancies occur, the existing action templates are generally unable to match the user-submitted action data. In other words, the existing action template system needs to be learned and expanded, thus requiring the use of the verification layer configured on the cloud server mentioned earlier.

[0090] When the real-time layer calculates the current action feature vector With template action feature vector If the similarity is lower than a preset threshold, or if there is significant uncertainty in the robot operation controlled by the motion data, the motion data can be sent to the real-time layer of the cloud server for processing. A wireless communication connection is established between the robot and the cloud server. The wireless communication technologies may include, but are not limited to: Global System for Mobile Communication (GSM), Enhanced Data GSM Environment (EDGE), Wideband Code Division Multiple Access (W-CDMA), Code Division Multiple Access (CDMA), Time Division Multiple Access (TDMA), Bluetooth, Wireless Fidelity (WiFi) (such as IEEE 802.11a, IEEE 802.11b, IEEE 802.11g and / or IEEE 802.11n standards), Voice over Internet Protocol (VoIP), Worldwide Interoperability for Microwave Access (Wi-Max), other protocols for email, instant messaging, and short messages, and any other suitable communication protocols, even those not yet developed.

[0091] It is worth noting that the processing of the verification layer and the real-time layer does not interfere with each other; the learning process of the verification layer will not affect the action semantic alignment of the real-time layer. The calculation means that although the real-time actions and action templates of the remote takeover process are not highly aligned, the robot can still be controlled to complete the corresponding tasks.

[0092] The validation layer employs dynamic time warping to rigorously align and precisely match entire segments of action data with template actions. The validation results can be used to subsequently correct the real-time layer's decisions and continuously update or optimize the action template to improve the robustness of subsequent recognition. The validation layer performs two calculations on the reported action sequences to comprehensively evaluate "why the real-time alignment is low but the task is successful," and then manually determines whether to update the action template.

[0093] In the verification layer, two metrics need to be calculated for the motion data: soft dynamic time warping distance and segment dynamic time adjustment local alignment. These two metrics are used to determine whether the motion data needs to be updated to a new motion template so that the control system can continue to learn.

[0094] The calculation method for soft dynamic time warping distance is as follows.

[0095] (12)

[0096] In the above formula, For soft dynamic time warping distance, For the action sequence corresponding to the action data, For the action sequence corresponding to the template action data, the soft dynamic time warping distance It can be simply referred to as ; These are the preset smoothing parameters; The set of all possible paths is directly determined by the template action data. for One of the possible paths; For is the path The total matching cost is directly determined by the action data and the template action data.

[0097] The calculation of local alignment for segment dynamic time adjustment requires the use of soft dynamic time warping distance. The calculation method, at its core, involves splitting the motion data and template motion data into separate parts. There are several key segments, each corresponding to a part of the robot's motion. For the first segment... For each segment, the soft dynamic time warping distance is calculated and mapped to similarity to ultimately determine the local alignment of the segment's dynamic time adjustment.

[0098] Therefore, the first step is to calculate the soft dynamic time warping distance of each segment. The calculation method is as follows.

[0099] (13)

[0100] (14)

[0101] In the above formula, For the first The dynamic time warp distance of the segment; That is, the dynamic time warping distance calculation function, which can be found in formula (12). The first action sequence in the action data split Short action segments; The first action sequence in the template action data A short action sequence. For the first The similarity between segments of action sequences; For the preset first Distance normalization scale of action segments.

[0102] The average of the segment similarity scores of all segments is used to obtain the local alignment score of the segment dynamic time adjustment. The calculation method is as follows.

[0103] (15)

[0104] In the above formula, Adjust local alignment for dynamic time of sub-segments; The total number of action segments to be broken down.

[0105] Soft dynamic time normalization distance Dynamic time adjustment of local alignment of sub-segments The learning score for this action data is obtained by performing a weighted summation. The calculation process is as follows.

[0106] + (16)

[0107] In the above formula, Score the learning of this action data; and These are the soft dynamic time warping distances. Dynamic time adjustment of local alignment of sub-segments The corresponding preset weighting coefficients are used to determine the learning score. If the threshold is exceeded, it means that the action data does not belong to any of the existing action templates. The action data corresponding to the operation instruction can be retained as a new action template in the database. Alternatively, the action data can be used to optimize the existing action templates in the database, thereby enabling the system to continuously learn and solve similar problems in the future. This reduces the reliance on remote control for users and improves the intelligence and availability of the system.

[0108] Step S140: Calculate the confidence level of the operation instruction based on the task alignment.

[0109] In one embodiment, step S140: calculating the confidence level of the operation command based on the task alignment includes: calculating the motion imbalance compensation coefficient based on the robot's motion imbalance data; calculating the environmental impact compensation coefficient based on the robot's environmental impact data; calculating the motion smoothing compensation coefficient based on the operation command; weighting and summing the motion imbalance compensation coefficient, the environmental impact compensation coefficient, and the motion smoothing compensation coefficient to obtain the comprehensive compensation degree; and calculating the confidence level based on the preset compensation coefficient, the comprehensive compensation degree, and the task alignment.

[0110] In one embodiment, the task alignment degree is calculated in step S130. This is used to describe the intensity and effectiveness of the intent to advance an operational task. However, task alignment alone... This is still insufficient as a final decision-making basis. In industrial scenarios, situations exist where the direction is correct but the process is unreasonable. For example, although the robot is moving towards the target point, its posture is severely distorted (the robot's movement exceeds normal limits, leading to collisions with the environment or violating robot movement constraints), or the operation is not smooth (repeated adjustments by the user for the same action cause unnecessary mechanical wear). Ignoring these problems will directly affect execution safety and result reliability. Therefore, it is also necessary to consider task alignment. The confidence level is calculated to determine whether and how to execute the user-issued operation command. The calculation of the confidence level requires utilizing not only task alignment... In addition, three additional aspects need to be considered: whether the current operation will cause physical imbalance of the robot, the impact of the operation on the environment, and whether the current operation is smooth. These three aspects correspond to the motion imbalance compensation coefficient, the environmental impact compensation coefficient, and the motion smoothness compensation coefficient, respectively.

[0111] The motion imbalance compensation coefficient is calculated based on the robot's motion imbalance data. This data may include the distance between the robot's center of mass and the supporting polygon, such as the ZMP distance (Zero Moment Point), or the center of mass deviation, thereby obtaining the deviation distance. Uneven pressure distribution on the support feet absolute values ​​of pitch / roll angles Joint overload / torque abnormality The above data is normalized and mapped, and then the motion imbalance compensation coefficient is calculated by weighted averaging. .

[0112] The environmental impact compensation coefficient is calculated based on the robot's environmental impact data. This environmental impact data may include the distance to the nearest obstacle. Personnel proximity Visual recognition confidence level Sensor health Collision or contact alarm signs already exist. The environmental impact compensation coefficient is calculated using the same method of normalization mapping followed by weighted averaging. For the environmental impact compensation coefficient You can also add a constraint to determine the proximity of the people involved. Is the distance less than a preset personnel safety threshold (e.g., set to 0.5m)? If the personnel proximity is... If it is less than the personnel safety threshold, the environmental impact compensation coefficient can be directly applied. Set it to 0 to ensure the safety of people nearby while the robot is operating.

[0113] Calculate motion smoothing compensation coefficients based on operation instructions. (Regarding task alignment...) The calculation process already extensively utilizes robot body data. Here, the operational smoothness only considers four parameters: joint angular velocity, angular acceleration, joint linear velocity, and joint linear acceleration. The calculated quantities include mean variance and mutation rate (the proportion of acceleration samples exceeding a threshold within a window). Then, the same normalized projection and weighted average calculations are performed to finally determine the smoothness compensation coefficient. .

[0114] It is worth noting that the calculation of the motion imbalance compensation coefficient... The environmental impact compensation coefficient was calculated using various motion imbalance data. All the environmental impact data used are directly included in the feature data. The corresponding data can be directly obtained from the feature data for calculation. The calculation process is all existing technology and will not be described in detail here.

[0115] The motion imbalance compensation coefficient Environmental impact compensation coefficient and motion smoothing compensation coefficient The weighted summation yields the overall compensation degree. The calculation process can be found in the following formula.

[0116] (17)

[0117] In the above formula, For comprehensive compensation degree; , and These are the motion imbalance compensation coefficients. Environmental impact compensation coefficient and motion smoothing compensation coefficient The corresponding preset weighting coefficients.

[0118] Determine the overall compensation degree Then, based on task alignment... As the dominant factor (gating nature), the comprehensive compensation degree corresponding to the other three factors. Used for correction and constraint. A "dominant multiplication + weighted compensation" approach is adopted to ensure that the intention is dominant while allowing the environment, attitude, and smoothness to influence the final result. The confidence level is determined through final calculation; the calculation process is as follows.

[0119] (18)

[0120] In the above formula, Confidence level; The preset compensation coefficient can be between 0.6 and 0.8, and can be arbitrarily set according to actual needs, without specific restrictions. The task alignment is achieved through the calculation method of formula (18). In very low cases, regardless of the overall compensation degree Even higher, the final calculated confidence level The values ​​remain very low, thus preventing erroneous responses that could cause necessary damage or risk. Similarly, if task alignment... The value is high, but due to the overall compensation degree A poor performance of one of the compensation coefficients leads to an inefficient overall compensation. The lower the value, the lower the final confidence level. Even low values ​​can affect or prevent the response to operational commands. For example, the personnel proximity mentioned earlier. The environmental impact compensation coefficient is lower than the personnel safety threshold. Set to 0 to reduce the overall compensation degree. The numerical value also reduces the confidence level. Numerical values ​​will influence the robot's response, thereby ensuring the safety of personnel and equipment to a certain extent, so that the robot no longer responds mechanically.

[0121] Step S150: Execute the operation instruction based on the confidence level.

[0122] In one embodiment, step S150: executing an operation instruction based on confidence level includes at least one of the following: if the confidence level is higher than a first threshold, then controlling the robot to perform the corresponding operation according to the operation instruction; if the confidence level is lower than or equal to the first threshold and greater than or equal to a second threshold, then issuing a warning to the user and controlling the robot to perform the corresponding operation according to the operation instruction; if the confidence level is lower than the second threshold, then pausing the execution of the operation instruction and sending a reconfirmation request to the user, and controlling the robot to execute or terminate the operation according to the user's feedback.

[0123] In one implementation, by confidence level The magnitude of the value determines whether and how the operation command's corresponding action should be executed. If the confidence level... If the value is higher than the first threshold, for example, if the first threshold is set to 0.85, it indicates that the user-issued operation instructions are reasonable and effective. They not only solve the problem the robot is currently facing but also achieve the robot's intended purpose on schedule. Furthermore, there are no extraneous actions during operation or any safety threats to surrounding equipment, the environment, or personnel. Therefore, the robot can be directly controlled to execute the actions corresponding to the operation instructions.

[0124] If confidence level The value is below or equal to the first threshold and greater than or equal to the second threshold, which can be set to 0.5, i.e., the confidence level. The values ​​are between 0.85 and 0.5. It's important to note that the specific values ​​of the first and second thresholds mentioned above are merely examples to describe the method and are not limitations on the technology; the specific values ​​depend on the actual situation. In this case, it indicates that compared to the previous situation, the complete response to the user's operation command may have certain problems, including but not limited to the following: the action exceeds the robot's maximum range of motion; the action is difficult to achieve or would cause damage to the equipment; continued movement may lead to collisions with surrounding equipment, the environment, or people, causing unnecessary damage; the user's operation command contains a large number of unnecessary additional actions, resulting in unnecessary energy waste and mechanical wear. In the above situations, the robot can be completed, but the process and result carry certain risks, requiring some prompting or warning to the user. Therefore, warning prompts can be issued to the user, such as suggestions or negotiated execution, through pop-up reminders in the remote control tool interface or VR glasses. After the reminder, the robot still executes the corresponding operation according to the operation command to ensure the response of the operation command itself.

[0125] If confidence level If the value is below the second threshold, it indicates that the user-issued operation command has an unavoidable problem. This could include, but is not limited to, the control action corresponding to the command exceeding the robot's reach, making the robot unable to complete the action. If the robot fully responds to the command, it will inevitably collide with surrounding equipment, the environment, or people. In other words, the operation command is neither reasonable nor effective; it either fails to achieve the robot's original goal, cannot resolve the robot's current predicament, and may even cause a collision with the external environment, leading to a safety accident. Therefore, this operation command should be prohibited from execution or require confirmation. The robot will refuse to execute the received teleoperation command and will send a reconfirmation request to the user. After user confirmation, the robot will be controlled to execute or terminate the operation based on the user's feedback. This ensures the robot's operational safety and prevents accidents.

[0126] Therefore, the method of this application includes the following steps: responding to the user-inputted operation command, processing the acquired robot data and operation command into feature data; determining the task type corresponding to the operation command; determining the task alignment degree corresponding to the feature data based on the task type; calculating the confidence degree of the operation command based on the task alignment degree; and executing the operation command based on the confidence degree. Therefore, this application can, upon receiving a user's operation command, enable the robot to understand the user's control intention through a comprehensive judgment of the operation command, task type, and robot data, thereby possessing the ability of "autonomous judgment + collaborative control" during remote takeover response. Ultimately, the response is based on the confidence degree of the operation command, rather than directly controlling according to the operation command, thereby reducing the safety and performance risks caused by human error. Furthermore, control through confidence degree also reduces reliance on highly skilled operators and improves the universality of takeover. The confidence degree mechanism achieves human-machine "consensus control," improving the success rate of complex tasks. Simultaneously, for the robot, it reduces reliance on the operator during remote takeover, realizing the robot's autonomy and intelligence.

[0127] Figure 2 An internal structural diagram of a computer device in one embodiment is shown. This computer device can specifically be a terminal, i.e., a robot, or a server. Figure 2 As shown, the device includes: a processor 310 and a memory 311 storing a computer program; wherein, Figure 2 The processor 310 shown in the diagram does not indicate that there is only one processor 310, but only indicates the positional relationship of the processor 310 relative to other devices. In practical applications, there can be one or more processors 310; similarly, Figure 2The memory 311 illustrated herein has the same meaning, that is, it is only used to indicate the positional relationship of memory 311 relative to other devices. In practical applications, there can be one or more memories 311. When the processor 310 runs the computer program, the method applied to the above-mentioned device is implemented.

[0128] The device may also include at least one network interface 312. The various components of the device are coupled together via a bus system 313. It is understood that the bus system 313 is used to implement communication between these components. In addition to a data bus, the bus system 313 also includes a power bus, a control bus, and a status signal bus. However, for clarity, in... Figure 2 The general designated all buses as Bus System 313.

[0129] The memory 311 can be volatile memory or non-volatile memory, or both. Non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), ferromagnetic random access memory (FRAM), flash memory, magnetic surface memory, optical disc, or compact disc read-only memory (CD-ROM); magnetic surface memory can be disk storage or magnetic tape storage. Volatile memory can be random access memory (RAM), used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as Static Random Access Memory (SRAM), Synchronous Static Random Access Memory (SSRAM), Dynamic Random Access Memory (DRAM), Synchronous Dynamic Random Access Memory (SDRAM), Double Data Rate Synchronous Dynamic Random Access Memory (DDRSDRAM), Enhanced Synchronous Dynamic Random Access Memory (ESDRAM), SyncLink Dynamic Random Access Memory (SLDRAM), and Direct Rambus Random Access Memory (DRRAM).The memory 311 described in the embodiments of the present invention is intended to include, but is not limited to, these and any other suitable types of memory.

[0130] The memory 311 in this embodiment of the invention is used to store various types of data to support the operation of the device. Examples of this data include: any computer programs used to operate on the device, such as operating systems and applications; contact data; phonebook data; messages; pictures; videos, etc. The operating system includes various system programs, such as the framework layer, core library layer, driver layer, etc., used to implement various basic services and handle hardware-based tasks. Applications can include various applications, such as media players, browsers, etc., used to implement various application services. Here, the program implementing the method of this embodiment of the invention can be included in the application.

[0131] Based on the same inventive concept as the foregoing embodiments, this embodiment also provides a computer-readable storage medium storing a computer program. The computer-readable storage medium can be a magnetic random access memory (FRAM), a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a flash memory, a magnetic surface memory, an optical disc, or a compact disc read-only memory (CD-ROM), etc.; it can also be various devices including one or any combination of the above-mentioned memories, such as mobile phones, computers, tablet devices, personal digital assistants, etc. When the computer program stored in the computer-readable storage medium is run by a processor, it implements the above method. For the specific steps implemented when the computer program is executed by the processor, please refer to [link to relevant documentation]. Figure 1 The description of the illustrated embodiments will not be repeated here.

[0132] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0133] In this document, the terms “comprising,” “including,” or any other variations thereof are intended to cover non-exclusive inclusion, which includes not only the elements listed but also other elements not expressly listed.

[0134] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A robot control method, characterized in that, Includes the following steps: In response to user input commands, the acquired robot data and the commands are processed into feature data. Determine the task type corresponding to the operation instruction; Determine the task alignment degree corresponding to the feature data according to the task type; the step of determining the task alignment degree corresponding to the feature data according to the task type includes: calculating spatial distance alignment degree and action semantic alignment degree respectively according to the feature data; determining the weighting coefficients corresponding to the spatial distance alignment degree and the action semantic alignment degree respectively according to the task type; and performing weighted fusion of spatial distance alignment degree and action semantic alignment degree according to the corresponding weighting coefficients to obtain the task alignment degree. Calculating spatial distance alignment based on the feature data includes: obtaining the target position and the current position from the feature data; and calculating and determining the spatial distance alignment based on the target position, the current position, and a preset maximum reference distance. Calculating the action semantic alignment degree based on the feature data includes: acquiring action data from the feature data; calculating a current action feature vector based on the action data; matching template action data corresponding to the action data; calculating a template action feature vector based on the template action data; and calculating the similarity between the current action feature vector and the template action feature vector as the action semantic alignment degree. The confidence level of the operation instruction is calculated based on the task alignment. This calculation includes: calculating a motion imbalance compensation coefficient based on the robot's motion imbalance data; calculating an environmental impact compensation coefficient based on the robot's environmental impact data; calculating a motion smoothing compensation coefficient based on the operation instruction; weighted summing of the motion imbalance compensation coefficient, the environmental impact compensation coefficient, and the motion smoothing compensation coefficient to obtain a comprehensive compensation degree; and calculating the confidence level based on a preset compensation coefficient, the comprehensive compensation degree, and the task alignment. The operation instruction is executed based on the confidence level.

2. The robot control method as described in claim 1, characterized in that, Determining the task type corresponding to the operation instruction includes: Obtain the robot's current task and break it down into at least one subtask; Based on the matching relationship between the subtask and the operation instruction, the task type corresponding to the operation instruction is determined.

3. The robot control method as described in claim 1, characterized in that, The task type includes mobile tasks; determining the task alignment degree corresponding to the feature data based on the task type includes: When the task type is a mobile task, acquire the robot's movement data; The task alignment is corrected based on the movement data and a preset penalty coefficient, and the corrected task alignment is taken as the final task alignment for the movement-type task.

4. The robot control method as described in claim 1, characterized in that, The execution of the operation instruction based on the confidence level includes at least one of the following: If the confidence level is higher than the first threshold, then the robot is controlled to perform the corresponding operation according to the operation instruction; If the confidence level is lower than or equal to the first threshold and greater than or equal to the second threshold, a warning is issued to the user, and the robot is controlled to perform the corresponding operation according to the operation instructions. If the confidence level is lower than the second threshold, the execution of the operation instruction is suspended, and a reconfirmation request is sent to the user. Based on the user's feedback, the robot is controlled to execute or terminate the operation.

5. A computer device, characterized in that, Including processor and memory; The processor is used to execute a computer program stored in the memory to implement the method as described in any one of claims 1 to 4.

6. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the method as described in any one of claims 1 to 4.

Images