A physical human-robot collaboration intention estimation method and related device

Abstract

The application belongs to an estimation method, aims at the technical problem that in the prior art, intention estimation mainly depends on short-term motion data, which influences the accuracy and safety of prediction, and the effectiveness of long-term collaborative human intention estimation is reduced, and provides a physical human-robot collaborative intention estimation method and related device, respectively acquiring an unobstructed past trajectory and an obstructed past trajectory, then respectively inputting the unobstructed past trajectory and the obstructed past trajectory into an unobstructed trajectory prediction model and an obstructed trajectory prediction model to obtain an unobstructed predicted trajectory and an obstructed predicted trajectory, and then performing role assignment according to the unobstructed predicted trajectory and the obstructed predicted trajectory. The application integrates human dynamics into long-term prediction, provides accurate intention understanding, realizes reasonable role assignment, and realizes the autonomy and operability of the robot.

Description

Technical Field

[0001] This application pertains to an estimation method, specifically a method and apparatus for estimating the intent of physical human-machine collaboration. Background Technology

[0002] In physical human-robot collaboration, effective strategies are needed to ensure seamless cooperation between robots and humans, accurately estimate intentions, dynamically adjust behavior, and assist humans with minimal effort. Therefore, accurate estimation of human intentions and reasonable allocation of human-robot roles are key challenges in improving collaborative performance.

[0003] In complex environments with potential hazards, such as unknown obstacles for robots, the rapid changes in human intent pose a significant challenge to intent estimation. Existing technologies primarily rely on short-term motion data, such as position and velocity, which limits the ability to detect changes in human intent and affects the accuracy and safety of predictions. Simultaneously, short-term data reduces the effectiveness of human intent estimation in long-term collaboration. Furthermore, human-robot role assignment involves complex task control allocation mechanisms between humans and robots. This process coordinates the human-robot relationship in real time, reduces discrepancies, and improves the robot's level of assistance. However, ensuring that the robot's behavior aligns with human intent remains crucial. Figure 1 Maintaining both autonomy and operability remains a significant challenge. Summary of the Invention

[0004] This application addresses the technical problem that prior art intention estimation mainly relies on short-term motion data, which affects the accuracy and security of prediction, and the effectiveness of human intention estimation in long-term collaboration is reduced. It provides an intention estimation method and related apparatus for physical human-machine collaboration.

[0005] To achieve the above objectives, this application adopts the following technical solution: Firstly, this application proposes a method for estimating the intent of physical human-computer collaboration, including: Obtain the unobstructed past trajectory and the obstructed past trajectory respectively; The accessible past trajectory and the obstructed past trajectory are respectively input into the accessible trajectory prediction model and the obstructed trajectory prediction model to obtain the accessible predicted trajectory and the obstructed predicted trajectory. The barrier-free trajectory prediction model and the obstacle-prone trajectory prediction model have the same structure, and the processing methods for both include: The past trajectory and its position encoding results are embedded and connected, then input into the encoder, and then sequentially passed through a multilayer perceptron and sampled to obtain the sampling results; After performing temporal convolution on the past trajectories, the result is combined with the sampling result and input into the decoder; The decoder output and the encoder output are processed through a future-past attention mechanism to obtain updated features; The updated features are sequentially passed through a multilayer perceptron, a Gaussian mixture model, and sampling to obtain the predicted trajectory; Roles are assigned based on the accessible and accessible predicted trajectories.

[0006] Furthermore, the method for assigning roles includes: Differential cooperative game theory is used to assign human and machine roles.

[0007] Furthermore, the method of allocating human-machine roles using differential cooperative game theory includes: Set up a reference model for the robot; By introducing state variables, the robot's reference model is rewritten as a linear state-space system; Treating humans and robots as two participants, define cost functions for both humans and robots; Based on differential cooperative game theory, a shared cost function is defined in cooperation; Substituting the cost functions of humans and robots into the shared cost function in cooperation yields the equation to be solved. Obtain the solution that minimizes the equation to be solved, and obtain the control signal to be input to the robot.

[0008] Furthermore, the expression for the reference model of the robot includes:

[0009] in, The inertia matrix, For the Coriolis / centrifugal force matrix, For gravity, For trajectory tracking error, For acceleration error, For speed error, The force exerted on humanity For the force of virtual robots; The expression for the linear state-space system includes:

[0010] in, , , ; For a linear state-space system, The state variables introduced.

[0011] Furthermore, the cost function for the human and the robot includes:

[0012] in, For human cost function, For human reference trajectory, The tracking error weights are calculated based on the human cost function compared to the human intention trajectory. For the robot's reference trajectory, To compare the tracking error weights with the human cost function for robot trajectories, The control input weights for the forces exerted on humans. Let the cost function of the robot be... The tracking error weights are relative to the robot cost function of the robot trajectory. The tracking error weights are used to calculate the robot's cost function compared to the trajectory of the human intention. The control input weights for the robot's control forces; The shared cost function in the cooperation includes:

[0013] in, For the shared cost function in cooperation, To control the role allocation coefficients of each cost weight, A weighted combination of human and robot reference trajectories. , Construct a diagonal matrix; The equations to be solved include:

[0014] in, For human intention trajectory weights, For robot trajectory weights.

[0015] Furthermore, the processing method in the encoder includes: Given end effector data over T time steps, features are extracted from the past trajectory and the position encoding results of the past trajectory after embedding and concatenation, and linear projection is performed to obtain the key, query, and value. The keys, queries, and values ​​are fed into a self-attention mechanism for learning to obtain the latent distribution.

[0016] Furthermore, the future-past attention mechanism includes: It combines keys and values ​​representing past data characteristics with queries for future data, following a standard feedforward and norm layer implementation.

[0017] Secondly, this application proposes an intention estimation system for physical human-computer collaboration, comprising: The data module is used to acquire the unobstructed past trajectory and the obstructed past trajectory, respectively; The prediction module is used to input the accessible past trajectory and the obstructed past trajectory into the accessible trajectory prediction model and the obstructed trajectory prediction model, respectively, to obtain the accessible predicted trajectory and the obstructed predicted trajectory. The barrier-free trajectory prediction model and the obstacle-prone trajectory prediction model have the same structure, and the processing methods for both include: The past trajectory and its position encoding results are embedded and connected, then input into the encoder, and then sequentially passed through a multilayer perceptron and sampled to obtain the sampling results; After performing temporal convolution on the past trajectories, the result is combined with the sampling result and input into the decoder; The decoder output and the encoder output are processed through a future-past attention mechanism to obtain updated features; The updated features are sequentially passed through a multilayer perceptron, a Gaussian mixture model, and sampling to obtain the predicted trajectory; The role assignment module is used to assign roles based on the accessible and obstructed predicted trajectories.

[0018] Thirdly, this application proposes an electronic device, including: a memory and one or more processors; the memory is coupled to the processors; wherein the memory stores computer program code, the computer program code including computer instructions, and when the computer instructions are executed by the processor, the electronic device performs the steps of the above-described physical human-machine collaboration intent estimation method.

[0019] Fourthly, this application proposes a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the above-described intention estimation method for physical human-computer collaboration.

[0020] Compared with the prior art, this application has the following beneficial effects: This application proposes a method for estimating intent in physical human-robot collaboration. It acquires both unobstructed and obstructed past trajectories, then inputs them into an unobstructed trajectory prediction model and an obstructed trajectory prediction model, respectively, to obtain unobstructed and obstructed predicted trajectories. Role assignment is then performed based on these predicted trajectories. Human intent estimation utilizes two transformer-based conditional variational autoencoders to combine the robot's motion data in unobstructed conditions with human-guided trajectories and obstacle avoidance forces. Furthermore, role assignment is performed based on the unobstructed and obstructed predicted trajectories to ensure that the robot's behavior aligns with human intent. Figure 1 To whom this application is made. This application incorporates human dynamics into long-term predictions, provides accurate intent understanding, enables reasonable role allocation, and achieves robot autonomy and operability.

[0021] This application also proposes a physical human-computer collaboration intent estimation system, an electronic device, and a computer-readable storage medium, which possess all the advantages of the aforementioned physical human-computer collaboration intent estimation methods. Attached Figure Description

[0022] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0023] Figure 1 This is a flowchart illustrating the intention estimation method for physical human-computer collaboration in this application. Figure 2 This is a schematic diagram illustrating one implementation of the intention estimation method for physical human-computer collaboration in this application; Figure 3 This is a schematic diagram of the overall structure of the barrier-free trajectory prediction model and the obstacle-prone trajectory prediction model in the embodiments of this application; Figure 4 This is a schematic diagram of the training structure of the barrier-free trajectory prediction model and the obstacle-prone trajectory prediction model in the embodiments of this application; Figure 5 This is a schematic diagram of the structure of the barrier-free trajectory prediction model and the obstacle-prone trajectory prediction model during testing / actual prediction in the embodiments of this application; Figure 6 This is a schematic diagram illustrating the principle of role allocation in the embodiments of this application; Figure 7 This is a schematic diagram of the intention estimation system for physical human-machine collaboration in this application. Detailed Implementation

[0024] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. The components of the embodiments of this application described and shown in the accompanying drawings can generally be arranged and designed in various different configurations.

[0025] Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely to illustrate selected embodiments of the application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.

[0026] It should be noted that similar labels and letters in the following figures indicate similar items. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures.

[0027] In the description of the embodiments of this application, it should be noted that if terms such as "upper," "lower," "horizontal," or "inner" indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings, or the orientation or positional relationship commonly used when the product of the invention is in use, they are only for the convenience of describing this application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation on this application. In addition, terms such as "first" and "second" are only used to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0028] Furthermore, the use of the term "horizontal" does not imply that the component must be absolutely horizontal, but rather that it can be slightly tilted. For example, "horizontal" simply means that its direction is more horizontal than "vertical," and does not mean that the structure must be completely horizontal, but can be slightly tilted.

[0029] In the description of the embodiments of this application, it should also be noted that, unless otherwise explicitly specified and limited, the terms "set," "install," "connect," and "link" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection of two components. Those skilled in the art can understand the specific meaning of the above terms in this application according to the specific circumstances.

[0030] In physical human-robot collaboration scenarios, achieving seamless collaboration between robots and humans requires effective strategies. This necessitates that robots accurately estimate human intentions and dynamically and flexibly adjust their behavior based on the estimation results, providing powerful assistance to humans with minimal resource input. In this process, accurate estimation of human intentions and reasonable allocation of human-robot roles become key and challenging tasks for improving collaborative efficiency.

[0031] In complex environments with potential dangers, numerous factors pose significant challenges to intention estimation. For example, robots may encounter unknown obstacles, and human intentions can change rapidly. Currently, most mainstream techniques rely on short-term motion data such as position and velocity for intention estimation. However, this approach has significant limitations, weakening the system's ability to detect changes in human intentions, thus negatively impacting the accuracy and safety of predictions. Furthermore, in long-term collaborative scenarios, relying solely on short-term data reduces the effectiveness of intention estimation.

[0032] Furthermore, the allocation of human-robot roles involves a complex task control mechanism between humans and robots. This mechanism requires real-time coordination of the human-robot relationship to minimize disagreements and enhance the robot's level of assistance to humans. However, ensuring that the robot's behavior both aligns with human intentions and maintains its autonomy and operability remains a major challenge that urgently needs to be overcome.

[0033] Based on the above, this application proposes a method and related apparatus for estimating the intent of physical human-machine collaboration. The following is a detailed description of this application in conjunction with embodiments and accompanying drawings.

[0034] like Figure 1 The diagram shown is a flowchart illustrating one method for estimating the intent of physical human-machine collaboration according to this application, which may include: S101, obtain the unobstructed past trajectory and the obstructed past trajectory respectively.

[0035] In human-robot collaboration scenarios, accurate estimation of human intent requires the collection of relevant data. Here, past trajectories refer to the motion trajectories generated by humans and robots during collaboration over a period of time. Unobstructed past trajectories utilize sensors (such as motion capture devices and inertial measurement units) to continuously monitor and record the movement of humans or robots when they encounter no significant obstacles. For example, in an open factory workshop, workers and collaborative robots work together to complete assembly tasks; the recorded motion trajectories of the workers and robots at this time constitute unobstructed past trajectories. This trajectory data is typically in time-series format, containing information such as the position and velocity of the moving objects at various points in time. Obstructed past trajectories also utilize the aforementioned sensors to collect data when obstacles exist during collaboration. For instance, if temporary stacks of goods suddenly appear in the workshop, the motion trajectories generated by workers and robots as they navigate around these goods to continue their tasks include information such as the position, velocity, and force of the moving objects at various points in time. Collecting obstructed past trajectories allows the system to learn the changing characteristics of motion patterns when facing different obstacles, which is crucial for accurately estimating human intent in complex environments.

[0036] S102, input the unobstructed past trajectory and the obstructed past trajectory into the unobstructed trajectory prediction model and the obstructed trajectory prediction model respectively to obtain the unobstructed predicted trajectory and the obstructed predicted trajectory.

[0037] Although the two models are designed for different scenarios, they have the same structure and consistent processing methods. The following section will detail their internal workflow to explain how the predicted trajectory is obtained.

[0038] The barrier-free trajectory prediction model and the obstacle-prone trajectory prediction model have the same structure, and the processing methods for both include: (1) The past trajectory and the position encoding result of the past trajectory are embedded and connected and then input into the encoder, and then passed through the multilayer perceptron and sampling in sequence to obtain the sampling result.

[0039] Since past trajectory data is a chronological sequence, positional encoding is needed to enable the model to capture the positional information (i.e., chronological order) within the data. For example, the positional information of each time point can be transformed into an encoded vector using a specific mathematical function. This vector not only contains positional information but also allows the model to distinguish elements at different positions when processing the sequence. Embedding the positionally encoded result with the original past trajectory data adds an extra dimension of positional information, allowing the model to better understand the positional relationships of the data within the time series during subsequent processing. The concatenated result is then input into the encoder. The encoder extracts and encodes features from the input trajectory sequence, transforming the original trajectory data into a more abstract and easily processed feature representation. The encoder's output features are then fed into a multilayer perceptron. The multilayer perceptron, a feedforward neural network composed of multiple fully connected layers, further nonlinearly transforms the encoder's output features. By adjusting the weight parameters between different layers, features are filtered and combined, highlighting useful features for subsequent predictions and suppressing noise and irrelevant information. The results processed by the multilayer perceptron undergo sampling. The purpose of sampling is to randomly extract some samples from the feature distribution output by the multilayer perceptron to increase the robustness and generalization ability of the model. For example, Monte Carlo sampling or other methods can be used to randomly select several sample points from the feature distribution. These sample points will serve as the basis for subsequent processing to capture the uncertainty and diversity in the feature distribution.

[0040] (2) After performing temporal convolution on the past trajectory, it is combined with the sampling result and input into the decoder.

[0041] Temporal convolutional neural networks (CNNs) can effectively capture both local and long-term dependencies in time-series data. By designing convolutional kernels of different sizes and the number of convolutional layers, convolution operations are performed on past trajectories along the time dimension to extract local and time-series features from the trajectory data. The result of the temporal convolution is combined with the previously obtained sampling results. This combination method fully utilizes the information obtained from the two different processing methods, including both a global abstract representation of trajectory features (sampling results) and the extraction of local and time-series features (temporal convolution results). The combined information is then input into a decoder, which also consists of multiple neural network layers. Its task is to transform the input feature information into trajectory data that can be used for prediction.

[0042] (3) The output of the decoder and the output of the encoder are processed through a future-past attention mechanism to obtain updated features.

[0043] Attention mechanisms are widely used in deep learning to allow models to focus on the importance of different parts of data when processing it. Future-past attention mechanisms are used to establish a correlation between the decoder and encoder outputs. By calculating the similarity between the decoder and encoder output features, it determines which parts of the encoder output are more important to the decoder's current prediction, thus assigning higher weights to these parts. For example, using this mechanism, the model can refer to the most relevant features from past trajectory encoding information to refine and improve the current prediction of the future trajectory, resulting in an updated feature representation. This updated feature representation, which integrates past trajectory information and current prediction information, is more conducive to accurately predicting future trajectories.

[0044] (4) The updated features are sequentially passed through a multilayer perceptron, a Gaussian mixture model, and sampling to obtain the predicted trajectory.

[0045] The updated features are then passed through a multilayer perceptron for further adjustment and transformation to better suit subsequent processing needs. This multilayer perceptron performs final feature optimization, making the updated features more consistent with the feature distribution of the predicted trajectory. The features processed by the multilayer perceptron are then input into a Gaussian mixture model (GMM). A GMM is a probabilistic model that assumes the data is a mixture of multiple Gaussian distributions. In trajectory prediction, the GMM is used to model the features, estimating the parameters (mean, covariance, etc.) of different Gaussian distributions. These parameters generate multiple possible trajectory predictions, as each Gaussian distribution represents a possible motion pattern. Finally, sampling is performed from the multiple possible trajectory predictions generated by the GMM to obtain the final predicted trajectory. Sampling allows the model to select one of several possible motion patterns as its output prediction of the future trajectory. This predicted trajectory is an estimate of the possible future trajectory based on the input past trajectory and the learned motion patterns.

[0046] S103, assign roles based on the accessible and obstructed predicted trajectories.

[0047] Once the obstacle-free and obstacle-prone predicted trajectories are obtained, human-robot roles can be assigned based on these predictions. One possible strategy for role assignment is to match the characteristics of the predicted trajectories with the requirements of the current collaborative task. For example, if the obstacle-free predicted trajectory shows that a human can complete a subtask more efficiently (e.g., a human can quickly reach the target location for fine manipulation when there are no obstacles), while the obstacle-prone predicted trajectory indicates that the robot has better motion planning capabilities when dealing with obstacles (e.g., the robot can more accurately calculate paths around obstacles), then in actual collaboration, tasks requiring rapid arrival at the target location can be assigned to the human, while obstacle-related path planning and movement tasks can be assigned to the robot.

[0048] The application explored scenarios where humans and robots cooperated in potentially hazardous environments, such as obstacles that the robot could not autonomously avoid. In these situations, the human's intention would change to avoid the danger, prompting the robot to adjust its behavior accordingly. These scenarios can be divided into two distinct cases: 1. Barrier-free: In this scenario, the force exerted by humans is close to zero, and the robot automatically assumes the leading role. As a guide, the robot avoids measurement errors and environmental interference inherent in human-applied forces.

[0049] 2. Obstacle Avoidance: Humans guide robots to adjust their trajectories to avoid hazards. Humans play a leading role, with robots providing assistance. By incorporating human-applied forces into predictions, robots can better understand human intentions and avoid dangers in the environment.

[0050] This application can effectively utilize past trajectory data to predict future motion trajectories and make reasonable human-machine role allocation based on these predictions, thereby improving the performance of physical human-machine collaboration.

[0051] The following is a more detailed embodiment of the intention estimation method for physical human-computer collaboration in this application, which will further illustrate this application in detail: like Figure 2 The diagram illustrates one implementation of the intention estimation method for physical human-robot collaboration proposed in this application. This application proposes a dual Transformer-based robot trajectory generator for physical human-robot collaboration. This generator features a hierarchical structure and utilizes human-guided motion and force data to rapidly capture changes in human intention, thereby achieving accurate trajectory prediction and dynamic robot behavior adjustment, ultimately enabling effective collaboration. Specifically, the human intention estimation in the trajectory generator uses two transformer-based conditional variational autoencoders to combine the robot's motion data in obstacle-free conditions with human-guided trajectories and obstacle avoidance forces. Furthermore, differential cooperative game theory is employed to synthesize predictions based on human-applied forces, ensuring that the robot's behavior aligns with human intentions. Figure 1To this end, a robot trajectory generator based on dual Transformer incorporates human dynamics into long-term predictions, providing accurate intent understanding, enabling reasonable role allocation, and achieving robot autonomy and operability. For example... Figure 3 The diagram shown illustrates the overall structure of the barrier-free trajectory prediction model and the obstacle-prone trajectory prediction model. Figure 4 The diagram shown illustrates the training structures for the obstacle-free trajectory prediction model and the obstacle-affected trajectory prediction model. Figure 5 The diagram shows the structural schematics of the barrier-free trajectory prediction model and the obstacle-prone trajectory prediction model during testing / actual prediction. It should be noted that during training, both past and future data are known, forming data pairs, while during testing, only past data is available. The overall structure is basically the same, with slight differences.

[0052] 1. Intent estimation.

[0053] This application aims to learn the multimodal probability distribution function. The trajectory is modeled as a past conditional distribution to predict future trajectories. The barrier-free trajectory prediction model and the barrier-prone trajectory prediction model in this application adopt the CVAE (Conditional Variational Autoencoder) framework, introducing Gaussian latent variables. The potential distribution of the data can be represented as: .

[0054] Through Transformer and linear layers, probability distribution ( Encode the data and minimize the loss. To optimize the probability distribution, loss It consists of the weighted negative variational lower bound and the prediction error.

[0055] Before learning features, a Transformer-based positional encoding method is used to represent the temporal information of the data. This encoding is concatenated with the data input embedding and fed into the encoder. The temporal dependencies of the data are learned through a multi-head self-attention mechanism. Specifically, given... The end effector data for each time step is first used to extract features from the input and then linearly projected to obtain the key. Query Sum The keys, queries, and values ​​of these mappings are fed into a self-attention mechanism to learn these characteristics.

[0056] The past encoder utilizes a multi-head self-attention mechanism combined with standard feedforward and norm layers to capture past features, encoding past data into a latent distribution. This is then fed into a multilayer perceptron to generate... The Gaussian parameters are used. Similarly, the future encoder encodes future data into a latent distribution to capture features. This application incorporates a future-past focus module that updates features by learning the relationship between past and future data. This is achieved by using features from past data. and With future data characteristics This is achieved by combining standard feedforward and norm layers. Finally, it is fed into a multilayer perceptron to generate... Gaussian parameters.

[0057] In the decoder, the sampling results The results of temporal convolutions with past data are combined as input. During training, Generated by the future encoder Obtain, such as Figure 4 As shown. During testing, it was generated by the previous encoder. Obtained, such as Figure 5 As shown, the decoder's structure is similar to that of the future encoder. Furthermore, an appropriate mask is used during prediction to prevent the model from influencing subsequent data. Finally, the output is used as parameters of a Gaussian mixture model, from which the prediction results are sampled.

[0058] In this process, this application introduces a hierarchical structure to predict future trajectories in the long term. Two branches handle manually guided obstacle avoidance and obstacle-free trajectories respectively, and integrate them into role assignment.

[0059] 2. Role allocation.

[0060] like Figure 6 The diagram shown illustrates the principle of role allocation. This application introduces differential cooperative game theory to rationally allocate human and machine roles, reducing discrepancies based on human intention estimation.

[0061] The robot's reference model can be designed as follows:

[0062] in, These are the required inertia matrix, Coriolis / centrifugal force matrix, and gravity, respectively. It is the trajectory tracking error. The reference trajectory is the result of the intention estimation. For the actual trajectory. It is virtual robot force. It is the force exerted by humans. For acceleration error, This represents the speed error.

[0063] First, introduce state variables. And rewrite (1) as a linear state-space system as .in, , Represents the identity matrix. It controls the input.

[0064] Next, we consider humans and robots as two participants and define their cost function as follows:

[0065] in, To calculate the tracking error weights for the human cost function compared to the human intention trajectory, To compare the tracking error weights with the human cost function of the robot trajectory, The tracking error weights are relative to the robot cost function of the robot trajectory. The tracking error weights are used to calculate the robot's cost function compared to the trajectory of the human intention. The control input weights for the forces exerted on humans. The control input weights are used to define the robot's control force. Based on differential cooperative game theory, the shared cost function in cooperation is defined as:

[0066] in, It is the role allocation coefficient that controls the weighting of each cost. Construct a diagonal matrix. It is a weighted combination of human and robot reference trajectories. When the prediction is accurate, substituting (2) into (3) yields:

[0067] in, For human intention trajectory weights, For robot trajectory weights.

[0068] Minimizing the solution of (3) is a classic linear quadratic optimal control (LQR) problem, with the optimal control input being... This is the final control signal input to the robot. This can be obtained by solving the algebraic Riccati equation. Clearly, the robot's assistance level can be adjusted by regulating the role allocation coefficient. To control. In this application, the force applied by humans can be adjusted based on environmental factors and task requirements. To optimize robot behavior.

[0069] This application proposes a robot trajectory generator based on a dual Transformer model, employing a hierarchical structure of two transformer-based conditional variational autoencoders. Human-applied forces are introduced into long-term predictions, achieving rational human-robot role allocation based on differential cooperative game theory and reducing human-robot disagreements. This application combines human intention estimation with role allocation, detecting intention changes and reducing disagreements, effectively improving collaborative performance in complex and prone-to-disagreement environments. The hierarchical structure for human intention estimation can simultaneously handle motion and force in physical human-robot collaboration, improving prediction accuracy and providing a precise understanding of human intentions. Role allocation based on differential cooperative game theory enables adaptive leader switching based on human-applied forces, ensuring that the robot's behavior aligns with human intentions. Figure 1 The goal is to reduce disagreements while maintaining the robot's autonomy.

[0070] like Figure 7 The diagram shown is a schematic of an intention estimation system for physical human-machine collaboration, which may include: The data module is used to acquire the unobstructed past trajectory and the obstructed past trajectory, respectively; The prediction module is used to input the accessible past trajectory and the obstructed past trajectory into the accessible trajectory prediction model and the obstructed trajectory prediction model, respectively, to obtain the accessible predicted trajectory and the obstructed predicted trajectory. The barrier-free trajectory prediction model and the obstacle-prone trajectory prediction model have the same structure, and the processing methods for both include: The past trajectory and its position encoding results are embedded and connected, then input into the encoder, and then sequentially passed through a multilayer perceptron and sampled to obtain the sampling results; After performing temporal convolution on the past trajectories, the result is combined with the sampling result and input into the decoder; The decoder output and the encoder output are processed through a future-past attention mechanism to obtain updated features; The updated features are sequentially passed through a multilayer perceptron, a Gaussian mixture model, and sampling to obtain the predicted trajectory; The role assignment module is used to assign roles based on the accessible and obstructed predicted trajectories.

[0071] It should be noted that, in the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the system embodiments described above are merely illustrative. For instance, the division of each module is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple modules may be combined or integrated into another device, or some features may be ignored or not executed. The modules described as separate components may or may not be physically separated. The components shown as modules may be one or more physical units, that is, they may be located in one place or distributed in multiple different places. Some or all of the modules can be selected to achieve the purpose of the solution in this embodiment according to actual needs.

[0072] Furthermore, in the various embodiments of the present invention, the modules can be integrated into one processing unit, or each module can exist physically separately, or two or more modules can be integrated into one unit. The integrated unit described above can be implemented in hardware or as a software functional unit.

[0073] This application also provides an electronic device, which may include one or more processors, memory and communication interfaces.

[0074] The memory, communication interface, and processor are coupled together. For example, the memory, communication interface, and processor can be coupled together via a bus.

[0075] The communication interface is used for data transmission with other devices. The memory stores computer program code. This computer program code includes computer instructions, which, when executed by the processor, cause the electronic device to perform the steps of the aforementioned physical human-machine collaboration intent estimation method.

[0076] The processor can be a processor or controller, such as a Central Processing Unit (CPU), a general-purpose processor, a Digital Signal Processor (DSP), an Application-Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. It can implement or execute the various exemplary logic blocks, modules, and circuits described in conjunction with this disclosure. The processor can also be a combination that implements computational functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, etc. The processor can be used to support an electronic device in performing the method steps provided in the above embodiments.

[0077] The bus can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. These buses can be categorized as address buses, data buses, control buses, etc.

[0078] This application provides a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, it implements the steps of the above-described intention estimation method for physical human-computer collaboration.

[0079] The computer-readable storage media involved in this application include random access memory (RAM), memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disks, removable disks, CD-ROMs, or any other form of storage media known in the art.

[0080] The above are merely preferred embodiments of this application and are not intended to limit this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A method for estimating intent in physical human-computer collaboration, characterized in that, include: Obtain the unobstructed past trajectory and the obstructed past trajectory respectively; The accessible past trajectory and the obstructed past trajectory are respectively input into the accessible trajectory prediction model and the obstructed trajectory prediction model to obtain the accessible predicted trajectory and the obstructed predicted trajectory. The barrier-free trajectory prediction model and the obstacle-prone trajectory prediction model have the same structure, and the processing methods for both include: The past trajectory and its position encoding results are embedded and connected, then input into the encoder, and then sequentially passed through a multilayer perceptron and sampled to obtain the sampling results; After performing temporal convolution on the past trajectories, the result is combined with the sampling result and input into the decoder; The decoder output and the encoder output are processed through a future-past attention mechanism to obtain updated features; The updated features are sequentially passed through a multilayer perceptron, a Gaussian mixture model, and sampling to obtain the predicted trajectory; Roles are assigned based on accessible and obstructed predicted trajectories; the role assignment strategy includes matching the characteristics of the predicted trajectories with the requirements of the current collaborative task. The method for assigning roles includes using differential cooperative game theory to assign human-machine roles: Set up a reference model for the robot; By introducing state variables, the robot's reference model is rewritten as a linear state-space system; Treating humans and robots as two participants, define cost functions for both humans and robots; Based on differential cooperative game theory, a shared cost function is defined in cooperation; Substituting the cost functions of humans and robots into the shared cost function in cooperation yields the equation to be solved. Obtain the solution that minimizes the equation to be solved, and obtain the control signal to be input to the robot.

2. The intention estimation method for physical human-machine collaboration according to claim 1, characterized in that, The expression for the reference model of the robot includes: in, The inertia matrix, For the Coriolis / centrifugal force matrix, For gravity, For trajectory tracking error, For acceleration error, For speed error, The force exerted on humanity For the force of virtual robots; The expression for the linear state-space system includes: in, , , ; For a linear state-space system, The state variables introduced.

3. The intention estimation method for physical human-machine collaboration according to claim 2, characterized in that, The cost functions for humans and robots include: in, For human cost function, For human reference trajectory, To calculate the tracking error weights for the human cost function compared to the human intention trajectory, For the robot's reference trajectory, To compare the tracking error weights with the human cost function of the robot trajectory, The control input weights for the forces exerted on humans. Let the cost function of the robot be... The tracking error weights are relative to the robot cost function of the robot trajectory. The tracking error weights are used to calculate the robot's cost function compared to the trajectory of the human intention. The control input weights for the robot's control forces; The shared cost function in the cooperation includes: in, For the shared cost function in cooperation, To control the role allocation coefficients of each cost weight, A weighted combination of human and robot reference trajectories. , , To construct a diagonal matrix; The equations to be solved include: in, For human intention trajectory weights, For robot trajectory weights.

4. The intention estimation method for physical human-machine collaboration according to claim 1, characterized in that, The processing method in the encoder includes: Given end effector data over T time steps, features are extracted from the past trajectory and the position encoding results of the past trajectory after embedding and concatenation, and linear projection is performed to obtain the key, query, and value. The keys, queries, and values ​​are fed into a self-attention mechanism for learning to obtain the latent distribution.

5. The intention estimation method for physical human-machine collaboration according to claim 4, characterized in that, The future-past attention mechanism includes: It combines keys and values ​​representing past data characteristics with queries for future data, following a standard feedforward and norm layer implementation.

6. An intention estimation system for physical human-computer collaboration, characterized in that, include: The data module is used to acquire the unobstructed past trajectory and the obstructed past trajectory, respectively; The prediction module is used to input the accessible past trajectory and the obstructed past trajectory into the accessible trajectory prediction model and the obstructed trajectory prediction model, respectively, to obtain the accessible predicted trajectory and the obstructed predicted trajectory. The barrier-free trajectory prediction model and the obstacle-prone trajectory prediction model have the same structure, and the processing methods for both include: The past trajectory and its position encoding results are embedded and connected, then input into the encoder, and then sequentially passed through a multilayer perceptron and sampled to obtain the sampling results; After performing temporal convolution on the past trajectories, the result is combined with the sampling result and input into the decoder; The decoder output and the encoder output are processed through a future-past attention mechanism to obtain updated features; The updated features are sequentially passed through a multilayer perceptron, a Gaussian mixture model, and sampling to obtain the predicted trajectory; The role assignment module is used to assign roles based on the accessible and obstructed predicted trajectories; the role assignment strategy includes matching the characteristics of the predicted trajectory with the requirements of the current collaborative task. The method for assigning roles includes using differential cooperative game theory to assign human-machine roles: Set up a reference model for the robot; By introducing state variables, the robot's reference model is rewritten as a linear state-space system; Treating humans and robots as two participants, define cost functions for both humans and robots; Based on differential cooperative game theory, a shared cost function is defined in cooperation; Substituting the cost functions of humans and robots into the shared cost function in cooperation yields the equation to be solved. Obtain the solution that minimizes the equation to be solved, and obtain the control signal to be input to the robot.

7. An electronic device, characterized in that, include: A memory, one or more processors; the memory is coupled to the processors; wherein the memory stores computer program code, the computer program code including computer instructions, and when the computer instructions are executed by the processor, the electronic device performs the steps of the physical human-machine collaboration intent estimation method as described in any one of claims 1-5.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the intention estimation method for physical human-computer collaboration as described in any one of claims 1-5.

Images