Human-computer interaction closed-loop feedback body model training method and system

By using the embodied large-scale model training method of human-computer interaction closed-loop feedback in human-computer robot training, virtual space is used for space-time superposition and Transformer architecture model training, the security problem in close collaboration and interaction of human-computers is solved, and a safe human-computer Interactive training and data set generation provide effective solutions for humanoid robot applications.

CN118916698AActive Publication Date: 2024-11-08TONGJI UNIV

Patent Information

Application Number
CN202411112403.5
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-14
Publication Date
2024-11-08
Estimated Expiration
2044-08-14

AI Technical Summary

Technical Problem

The prior art has failed to effectively solve the personal safety problems of humanoid robots in the process of close cooperation and interaction between humans and machines, and lacks training platforms and testing methods that can ensure safety.

Method used

Using the embodied large-scale model training method of human-computer interaction closed-loop feedback, by obtaining behavioral data in the first physical space and the second physical space, building virtual space for space-time superposition, and using the Transformer architecture model for tokenization processing and large-scale neural networks Training, realizes human-computer synchronous work, collaborative work and game interaction training, and defines corresponding reward functions to optimize model training.

Benefits of technology

Through the fusion of cross-time, space, virtual and real, security problems in the human-computer interaction training process are avoided, rich human-computer interaction data sets are generated, effective solutions are provided, and the promotion of human-shaped robot applications is promoted.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118916698A_ABST
    Figure CN118916698A_ABST
Patent Text Reader

Abstract

The invention relates to a human-computer interaction closed-loop feedback body-provided large model training method and system, and the training method relates to three space-time crossing exercise work spaces, which comprise a first physical space, a second physical space and a virtual space. The first physical space is used for training a robot body intelligent model and comprises a body sensing feedback system, a global motion capture system and a robot body; the second physical space is used for human behavior data collection and cognitive analysis and comprises a wearable multimode sensor, a global motion capture system and a natural person experiment object. And the virtual space is used for carrying out virtual-real space-time alignment fusion on the first physical space and the second physical space, so that operation scene construction rendering is realized, and human-computer interaction feedback body model training is realized. Compared with the prior art, the method has the advantages that the safety problem in the man-machine interaction training process is avoided through cross-space-time interaction learning training of the human and the robot, and the like.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the field of embodied intelligence, and in particular, relates to a large embodied model training method and system with closed-loop feedback of human-computer interaction. Background Art

[0002] The aging population, labor shortage, and the continuous pursuit of improving productivity and quality of life have driven the growth of demand in the humanoid robot market. In recent years, with the development of technologies such as artificial intelligence, machine learning, computer vision, and natural language processing, the intelligence level of humanoid robots has been significantly improved, and governments have issued a series of policies and guidelines to support the development of the humanoid robot industry. Humanoid robots have become a research hotspot at home and abroad, but their development still faces many major challenges: the need to improve perception, decision-making, and action capabilities to achieve more natural and efficient interactions; the need to integrate multimodal sensory information such as vision, hearing, and touch to achieve richer and more intuitive interaction methods; the need to improve emotional interaction capabilities so that robots can better understand and express emotions; the need to ensure that robot behavior complies with ethical and legal regulations and to ensure the safety of human-machine interactions in various social service scenarios.

[0003] In order to meet the above challenges, it is necessary to develop a basic platform for humanoid robot research and performance testing and verification, which can be used for cross-scenario humanoid robot skill learning and training. However, many safety issues often arise during the training of humanoid robots, especially the personal safety issues of algorithm training for humanoid robots in the process of close human-machine collaborative interaction. Currently, there is no training platform and testing method that can guarantee safety.

[0004] Chinese patent application CN112446962A discloses a method and system for generating an intelligent agent based on virtual-real evolution. The specific implementation of the method includes: constructing a physical space training environment to obtain an initial environment for intelligent agent training; constructing a virtual space based on the initial environment, and training the intelligent agent in the virtual space; verifying the trained intelligent agent in the physical space, eliminating the difference between the virtual space and the physical space, and generating an intelligent agent based on virtual-real evolution. However, this method does not solve the personal safety problem of humanoid robots performing algorithm training in the process of close human-machine collaborative interaction. Summary of the invention

[0005] The purpose of the present invention is to overcome the defects of the above-mentioned prior art and to provide an embodied large model training method and system with closed-loop feedback of human-computer interaction.

[0006] The purpose of the present invention can be achieved by the following technical solutions:

[0007] The present invention provides an embodied large model training method for closed-loop feedback of human-computer interaction, comprising the following steps:

[0008] Step S1: In the first physical space (1), the behavior data of the embodied intelligent model for the training task is obtained through the multimodal sensor of the fuselage and the global motion capture system deployed in the first physical space (1), and the behavior data of the embodied intelligent model is constructed as a multidimensional vector of time series.

[0009] Step S2: In the second physical space (2), the wearable multimodal sensor arranged on the natural human experimental subject and the global motion capture system deployed in the second physical space (2) are used to obtain the behavioral data of the natural human experimental subject performing the task corresponding to the embodied intelligent model training task in step S1, and the behavioral data of the natural human experimental subject is modeled as a multidimensional vector of a time series. The first physical space (1) and the second physical space (2) are in different time and space;

[0010] Step S3: construct a virtual space (3) that realizes the virtual-real fusion and spatiotemporal superposition of the first physical space (1) and the second physical space (2), and model the digital environment data of the virtual space (3) as a multidimensional vector of a time series.

[0011] Step S4: Multidimensional vector Multidimensional vector With multidimensional vectors Four-dimensional ternary space-time mapping and fusion are realized through space-time affine transformation. The multidimensional vector of space-time data after mapping and fusion is defined as

[0012] Step S5: Transform the spatiotemporal data into multidimensional vectors The training data set is used as a training data set for the embodied big model to carry out the embodied big model algorithm training, and guide the embodied intelligent model of the first physical space (1) to perform corresponding tasks.

[0013] The behavioral data for training the embodied intelligence model includes posture data, sensor data, and interaction data.

[0014] The virtual space (3) realizes a metaverse digital environment for skill learning and model training in different operation task scenarios, and the metaverse digital environment includes terrain, weather, lighting, static objects, and dynamic objects.

[0015] The spatiotemporal data multidimensional vector The solution expression is:

[0016] in, is the spatiotemporal mapping function between the first physical space (1) and the virtual space scene elements, It is the spatiotemporal mapping function between the second physical space (2) and the virtual space scene elements.

[0017] The step S5 carries out the embodied large model algorithm training, and the training steps are as follows: the spatiotemporal multidimensional vector of the long time series The data is tokenized and imported into the embodied big model algorithm for learning and training to achieve embodied intelligent task skill training of human-like perception-positioning-cognition-decision-making-planning-execution. The embodied big model is based on the Transformer architecture model.

[0018] The algorithm categories of the embodied large model training include human-machine synchronous operation skill training, human-machine collaborative operation skill training, and human-machine game interaction training.

[0019] The human-machine synchronous operation skill training comprises the following steps:

[0020] Step A1: The embodied model is Perform tokenization and large-scale neural network training based on the Transformer architecture model to guide the embodied intelligent model in the first physical space (1) to perform synchronous operation tasks;

[0021] Step A2: Mapping the behavior process data of the natural person experimental subject in the second physical space (2) to the virtual space (3) to generate the operation data as the Teacher data set, and mapping the behavior process data of the embodied intelligent model in the first physical space (1) to the virtual space (3) to generate the data as the Student data set;

[0022] Step A3: Define the consistency-based reward function as

[0023]

[0024] Among them, r t (π,μ) is the immediate reward calculated at time step t according to the Student’s strategy π and the Teacher’s strategy μ; γ is the discount factor; T is the time step;

[0025] R consist Measures the difference between the Student's strategy π and the Teacher's strategy μ. The greater the difference between the Student's behavior and the Teacher's behavior, the greater the negative reward given. By maximizing the reward R consist Iterative learning based on consistency is achieved, the training of the embodied large model is continuously optimized, and the embodied intelligent model strategy in the first physical space (1) is updated to synchronize the work tasks of the natural human experimental subjects in the second physical space (2).

[0026] The human-machine collaborative operation skill training comprises the following steps:

[0027] Step B1: The first physical space (1) and the second physical space (2) are superimposed in time and space by the virtual space (3), so as to realize the interaction between the human and the embodied intelligent model in the virtual space;

[0028] Step B2: The embodied intelligent model in the first physical space and the natural person experimental subject in the second physical space perform the same collaborative task and are mapped and fused into a multidimensional vector of spatiotemporal data in the virtual space. As input for training embodied large models;

[0029] Step B3: The embodied large model is Perform tokenization and large-scale neural network training based on the Transformer architecture model to guide the embodied intelligent model in the first physical space to perform collaborative tasks;

[0030] Step B4: using the interactive collaborative operation data between the second physical space and the virtual space as a Teacher data set, and using the interactive collaborative operation data between the first physical space and the virtual space as a Student data set;

[0031] Step B5: When the Student plays a leading role and the Teacher provides assistance or supervision, the collaborative reward function is defined as

[0032]

[0033] Among them, r t (π,μ) is the immediate task completion reward calculated at time step t according to the Student's strategy π and the Teacher's strategy μ; γ is the discount factor; T is the time step; P t (μ)≤0 is the penalty term when the Teacher performs auxiliary supervision at time step t to prompt the Student to change the current unreasonable strategy;

[0034] On the contrary, when the teacher plays a leading role and the student provides assistance or supervision, the collaborative reward is defined as

[0035]

[0036] Among them, r t (π,μ) is the immediate task completion reward calculated at time step t according to the Student's strategy π and the Teacher's strategy μ; γ is the discount factor; T is the time step; -P t (π) ≥ 0 is the reward for the Student to execute the correct auxiliary supervision strategy at time step t;

[0037] By maximizing the reward R collabThrough dynamic collaboration, we realize reinforcement learning training based on human feedback, continuously optimize the training of the embodied large model, and guide the embodied intelligent model in the first physical space and the natural human experimental subjects in the second physical space to perform collaborative tasks.

[0038] The human-computer game interactive training comprises the following steps:

[0039] Step C1: realizing the interaction between the human and the embodied intelligent model in the virtual space by superimposing the first physical space (1) and the second physical space (2) in time and space through the virtual space (3);

[0040] Step C2: The embodied intelligent model in the first physical space and the natural person experimental subject in the second physical space perform a confrontational game task, and are mapped and fused into a multi-dimensional vector of spatiotemporal data in the virtual space. As input for training embodied large models;

[0041] Step C3: The embodied large model is Perform tokenization and large-scale neural network training based on the Transformer architecture model to guide the embodied intelligent model of the first physical space to perform adversarial game tasks;

[0042] Step C4: The competition data between the natural person experimental subject in the second physical space (2) and the twin embodied intelligent model in the virtual space (3) is used as the Teacher data set, and the competition data between the embodied intelligent model in the first physical space (1) and the twin natural person in the virtual space (3) is used as the Student data set;

[0043] Step C5: Define the adversarial reward as

[0044]

[0045] Among them, r t (π,μ) is the immediate reward calculated at time step t according to the Student's strategy π and the Teacher's strategy μ, γ is the discount factor, and T is the time step;

[0046] The adversarial loss is defined as

[0047]

[0048] Among them, τ represents the execution task (a series of states and action sequences), π is the strategy adopted to execute the task (a series of control output instructions), p(τ|π) is the probability of executing task τ under strategy π, and μ(τ) is the probability distribution of the Teacher's evaluation of the performance of the task;

[0049] By maximizing adversarial rewards and minimizing adversarial losses, we can achieve reinforcement learning training based on human feedback, continuously optimize the training of the embodied large model, and guide the embodied intelligent model in the first physical space to play adversarial games with natural persons in the second physical space.

[0050] The present invention also provides an embodied large model training system with closed-loop feedback of human-computer interaction, characterized in that it includes: a first physical space, a second physical space and a virtual space, the first physical space includes an embodied sensor feedback system, a global motion capture system and an embodied intelligent model, the second physical space includes a wearable multi-modal sensor, a global motion capture system and a natural human experimental subject, and the virtual space is used to align and fuse the first physical space and the second physical space in virtual and real time and space, realize the construction and rendering of the operation scene, and realize the embodied model training of human-computer interaction feedback.

[0051] Compared with the prior art, the present invention has the following advantages:

[0052] (1) Avoid safety issues in the human-robot interaction training process through cross-temporal and spatial interactive learning and training between humans and robots.

[0053] (2) By integrating the virtual and the real, the generation of human-computer interaction datasets in different scenarios is realized, solving the problem of the lack of big data on human-computer interaction with embodied large models.

[0054] (3) A human feedback reinforcement learning training method based on virtual-real fusion and spatiotemporal superposition was proposed to realize different types of experiments such as human-machine synchronous operation skill training, human-machine collaborative operation skill training, and human-machine game interaction training.

[0055] (4) It provides effective solutions for skill learning in different practical embodied intelligence application scenarios, which is expected to promote the application and promotion of humanoid robots. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] Figure 1 This is a diagram of the architecture of the large model training platform of the method of the present invention;

[0057] Figure 2 This is a diagram of the large model training framework;

[0058] Figure numerals: 1. first physical space, 2. second physical space, 3. virtual space. DETAILED DESCRIPTION

[0059] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0060] The present invention discloses a human-machine interaction feedback embodied model training method that integrates virtuality and reality across time and space to solve the safety problem of human-machine interaction training for humanoid robots. The present invention is applicable to large-scale data collection, scene generation, and safe human-machine interaction feedback reinforcement learning training for embodied intelligence such as humanoid robots, providing an effective solution for skill learning in different actual embodied intelligence application scenarios, and is expected to promote the application and promotion of humanoid robots.

[0061] The present invention provides a training method for a large embodied model based on virtual-real fusion space-time superposition human-computer interaction closed-loop feedback. The training method involves three cross-space and time motion operation spaces, including the first physical space, i.e., the robot training field, the second physical space, i.e., the human behavior cognition field, and the virtual space, i.e., the metaverse training field. Figure 1 As shown. The first physical space is used for robot embodied intelligent model training, including embodied sensor feedback system, global motion capture system and robot body; the second physical space is used for human behavior data collection and cognitive analysis, including wearable multi-modal sensors, global motion capture system and natural human experimental subjects. The virtual space is used to align and fuse the first physical space with the second physical space in virtual and real time and space, realize the construction and rendering of the operation scene, and realize the embodied model training of human-computer interaction feedback.

[0062] A large model training method based on virtual-real fusion space-time superposition human-computer interaction closed-loop feedback, including:

[0063] Step S1: construct the first physical space (1) for training the embodied intelligence model of the humanoid robot, arrange the multimodal sensors of the body and the global motion capture system deployed in the indoor environment of the physical space, and obtain the behavioral data (including posture data, sensor data, interaction data, etc.) for training the embodied intelligence model of the humanoid robot. The data set is constructed as a multidimensional vector of time series.

[0064] Step S2: construct a second physical space (2) for human behavior data collection and cognitive analysis, and the second physical space (2) and the first physical space (1) are in different time and space, and the second physical space (2) arranges wearable multi-modal sensors on natural human experimental subjects and a global motion capture system deployed in the indoor environment of the physical space, and obtains natural human behavior data for analysis, and models the natural human behavior data as a multi-dimensional vector of a time series

[0065] Step S3: Based on the computing power and rendering computing power platform, a virtual space is constructed to realize the virtual-real fusion and spatiotemporal superposition of the first physical space (1) and the second physical space (2), and a metaverse digital environment (including terrain, weather, lighting, static objects, dynamic objects, etc.) is realized for skill learning and model training in different task scenarios. The digital environment data of the virtual space is modeled as a multidimensional vector of time series.

[0066] Step S4: The first physical space (1) and the second physical space (2) are mapped and fused in the virtual space through space-time affine transformation. The multidimensional vector of the space-time data after mapping and fusion is defined as The solution expression is as follows:

[0067]

[0068] Step S5: Map the fused spatiotemporal data multidimensional vector As the training data set of the big model, the robot embodied big model algorithm training is carried out. The big model training framework is as follows Figure 2 As shown. Transform the spatiotemporal data into multidimensional vectors As the input for large model training, the long-term spatiotemporal multi-dimensional vector data is first tokenized and imported into a large-scale neural network based on the Transformer architecture model for learning and training, to achieve human-like perception-positioning-cognition-decision-making-planning-execution embodied intelligent task skill training.

[0069] Step S6: The algorithm categories for embodied large model training include three types: human-machine synchronous operation skill training, human-machine collaborative operation skill training, and human-machine game interaction training. Different data set classifications and training strategies are assigned to different types.

[0070] The step S6 of human-machine synchronous operation skill training, taking the skill of cutting vegetables in the kitchen scene as an example, specifically includes:

[0071] Step S61: The robot in the first physical space (1) and the natural person in the second physical space (2) perform the same vegetable cutting task, and map and fuse them into a multidimensional vector of spatiotemporal data in the virtual space. As the input for training the embodied large model,

[0072] Step S62: The embodied large model is Tokenization and large-scale neural network training based on the Transformer architecture model are performed to guide the robot in the first physical space (1) to perform the vegetable cutting task;

[0073] Step S63: Map the process data of the natural person cutting vegetables in the second physical space (2) to the virtual space to generate operation data as the Teacher data set, that is,

[0074] Step S64: Map the process data of the robot cutting vegetables in the first physical space (1) to the production data in the virtual space as the Student data set, i.e.

[0075] Step S65: Define the consistency-based reward function as

[0076]

[0077] Among them, r t (π,μ) is the immediate reward calculated at time step t according to the Student’s strategy π and the Teacher’s strategy μ; γ is the discount factor; T is the time step.

[0078] R consist Measures the difference between the Student's strategy π and the Teacher's strategy μ. The greater the difference between the Student's behavior and the Teacher's behavior, the greater the negative reward given. By maximizing the reward R consist Iterative learning based on consistency is achieved, and the embodied large model training is continuously optimized to update the robot strategy in the first physical space (1) to synchronize the vegetable-cutting actions of a natural person in the second physical space (2).

[0079] The step S6 specifically includes: human-machine collaborative operation skill training, taking the coordination of steps and movements required for a duet dance and completing a dance performance as an example, specifically including:

[0080] Step S61: The first physical space (1) and the second physical space (2) are superimposed in time and space by the virtual space to realize the interaction between the human and the robot in the virtual space, that is,

[0081] Step S62: The robot in the first physical space (1) and the natural person in the second physical space (2) perform the same duet dance performance task, and map and fuse them into a multidimensional vector of spatiotemporal data in the virtual space. As the input for training the embodied large model,

[0082] Step S63: The embodied large model is Tokenization and large-scale neural network training based on the Transformer architecture model are performed to guide the robot in the first physical space (1) to perform a duet dance performance task;

[0083] Step S64: Map the process data of the natural person's dance performance in the second physical space (2) to the virtual space to generate operation data as the Teacher data set, that is,

[0084] Step S65: Map the process data of the robot dance performance in the first physical space (1) to the production data in the virtual space as the Student data set, i.e.

[0085] Step S66: In the duet task, the student and the teacher can play different roles to promote the student's independent learning ability. For example, when the student plays a leading role and the teacher provides assistance or supervision, the collaborative reward function is defined as

[0086]

[0087] Among them, r t (π,μ) is the immediate task completion reward calculated at time step t according to the Student's strategy π and the Teacher's strategy μ; γ is the discount factor; T is the time step; P t (μ)≤0 is the penalty term when the Teacher performs auxiliary supervision at time step t to prompt the Student to change the current unreasonable strategy.

[0088] On the contrary, when the teacher plays a leading role and the student provides assistance or supervision, the collaborative reward is defined as

[0089]

[0090] Among them, r t (π,μ) is the immediate task completion reward calculated at time step t according to the Student's strategy π and the Teacher's strategy μ; γ is the discount factor; T is the time step; -P t (π) ≥ 0 is the reward for the Student executing the correct auxiliary supervision strategy at time step t.

[0091] By maximizing the reward R collab With dynamic collaboration, reinforcement learning based on human feedback (RLHF) training is implemented to continuously optimize the embodied large model training to guide the robot in the first physical space (1) and the natural person in the second physical space (2) to collaborate to complete a duet dance performance.

[0092] The step S6 specifically includes: human-computer game interactive training, taking two-person martial arts competition as an example, specifically includes:

[0093] Step S61: The first physical space (1) and the second physical space (2) are superimposed in time and space by the virtual space to realize the interaction between the human and the robot in the virtual space, that is,

[0094] Step S62: The robot in the first physical space (1) and the natural person in the second physical space (2) perform a martial arts competition task, and map and fuse them into a multi-dimensional vector of spatiotemporal data in the virtual space. As the input for training the embodied large model,

[0095] Step S63: The embodied large model is Tokenization and large-scale neural network training based on the Transformer architecture model are performed to guide the robot in the first physical space (1) to perform martial arts competition tasks;

[0096] Step S64: The martial arts competition data between the natural person in the second physical space (2) and the twin robot in the virtual space is used as the Teacher data set, that is,

[0097] Step S65: The martial arts competition data between the robot in the first physical space (1) and the twin natural person in the virtual space is used as the Student data set, that is,

[0098] Step S66: define the confrontation reward as

[0099]

[0100] Among them, r t (π,μ) is the immediate reward calculated at time step t according to the Student’s strategy π and the Teacher’s strategy μ, γ is the discount factor, and T is the time step.

[0101] The adversarial loss is defined as

[0102]

[0103] Among them, τ represents the execution task (a series of states and action sequences), π is the strategy adopted to execute the task (a series of control output instructions), p(τ|π) is the probability of executing task τ under strategy π, and μ(τ) is the probability distribution of the Teacher's evaluation of the quality of task execution.

[0104] By maximizing the adversarial reward and minimizing the adversarial loss, reinforcement learning based on human feedback (RLHF) training is implemented, and the embodied large model training is continuously optimized to guide the robot in the first physical space (1) and the natural person in the second physical space (2) to compete in martial arts.

[0105] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed by the present invention, and these modifications or replacements should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be based on the protection scope of the claims.

Claims

1. A method for training a large embodied model with closed-loop feedback of human-computer interaction, characterized in that: The following steps are involved: Step S1: In the first physical space (1), the behavior data of the embodied intelligent model for the training task is obtained through the multimodal sensor of the fuselage and the global motion capture system deployed in the first physical space (1), and the behavior data of the embodied intelligent model is constructed as a multidimensional vector of time series. Step S2: In the second physical space (2), the wearable multimodal sensor arranged on the natural human experimental subject and the global motion capture system deployed in the second physical space (2) are used to obtain the behavioral data of the natural human experimental subject performing the task corresponding to the embodied intelligent model training task in step S1, and the behavioral data of the natural human experimental subject is modeled as a multidimensional vector of a time series. The first physical space (1) and the second physical space (2) are in different time and space; Step S3: construct a virtual space (3) that realizes the virtual-real fusion and spatiotemporal superposition of the first physical space (1) and the second physical space (2), and model the digital environment data of the virtual space (3) as a multidimensional vector of a time series. Step S4: Multidimensional vector Multidimensional vector With multidimensional vectors Four-dimensional ternary space-time mapping and fusion are realized through space-time affine transformation. The multidimensional vector of space-time data after mapping and fusion is defined as Step S5: Transform the spatiotemporal data into multidimensional vectors The training data set is used as a training data set for the embodied big model to carry out the embodied big model algorithm training, and guide the embodied intelligent model of the first physical space (1) to perform corresponding tasks.

2. The embodied large model training method for closed-loop feedback of human-computer interaction according to claim 1 is characterized in that: The behavioral data for training the embodied intelligence model includes posture data, sensor data, and interaction data.

3. The embodied large model training method for closed-loop feedback of human-computer interaction according to claim 1 is characterized in that: The virtual space (3) realizes a metaverse digital environment for skill learning and model training in different operation task scenarios, and the metaverse digital environment includes terrain, weather, lighting, static objects, and dynamic objects.

4. The embodied large model training method for closed-loop feedback of human-computer interaction according to claim 1 is characterized in that: The spatiotemporal data multidimensional vector The solution expression is: in, is the spatiotemporal mapping function of the scene elements between the first physical space (1) and the virtual space (3), It is the spatiotemporal mapping function of the scene elements between the second physical space (2) and the virtual space (3).

5. The embodied large model training method for closed-loop feedback of human-computer interaction according to claim 1 is characterized in that: The step S5 carries out the embodied large model algorithm training, and the training steps are as follows: the spatiotemporal multidimensional vector of the long time series The data is tokenized and imported into the embodied big model algorithm for learning and training to achieve embodied intelligent task skill training of human-like perception-positioning-cognition-decision-making-planning-execution. The embodied big model is based on the Transformer architecture model.

6. The embodied large model training method for closed-loop feedback of human-computer interaction according to claim 5, characterized in that: The algorithm categories of the embodied large model training include human-machine synchronous operation skill training, human-machine collaborative operation skill training, and human-machine game interaction training.

7. The embodied large model training method for closed-loop feedback of human-computer interaction according to claim 6 is characterized in that: The human-machine synchronous operation skill training comprises the following steps: Step A1: The embodied model is Perform tokenization and large-scale neural network training based on the Transformer architecture model to guide the embodied intelligent model in the first physical space (1) to perform synchronous operation tasks; Step A2: Mapping the behavior process data of the natural person experimental subject in the second physical space (2) to the virtual space (3) to generate the operation data as the Teacher data set, and mapping the behavior process data of the embodied intelligent model in the first physical space (1) to the virtual space (3) to generate the data as the Student data set; Step A3: Define the consistency-based reward function as Among them, r t (π,μ) is the immediate reward calculated at time step t according to the Student’s strategy π and the Teacher’s strategy μ; γ is the discount factor; T is the time step; R consist Measures the difference between the Student's strategy π and the Teacher's strategy μ. The greater the difference between the Student's behavior and the Teacher's behavior, the greater the negative reward given. By maximizing the reward R consist Iterative learning based on consistency is achieved, the training of the embodied large model is continuously optimized, and the embodied intelligent model strategy in the first physical space (1) is updated to synchronize the work tasks of the natural human experimental subjects in the second physical space (2).

8. The embodied large model training method for closed-loop feedback of human-computer interaction according to claim 6 is characterized in that: The human-machine collaborative operation skill training comprises the following steps: Step B1: The first physical space (1) and the second physical space (2) are superimposed in time and space by the virtual space (3), so as to realize the interaction between the person and the embodied intelligent model in the virtual space (3); Step B2: The embodied intelligent model in the first physical space (1) and the natural person experimental subject in the second physical space (2) perform the same collaborative task, and are mapped and fused into a multidimensional vector of spatiotemporal data in the virtual space (3). As input for training the embodied large model; Step B3: The embodied large model is Tokenization and large-scale neural network training based on the Transformer architecture model are performed to guide the embodied intelligent model in the first physical space (1) to perform collaborative tasks; Step B4: using the interactive collaborative work data between the second physical space (2) and the virtual space (3) as a Teacher data set, and using the interactive collaborative work data between the first physical space (1) and the virtual space (3) as a Student data set; Step B5: When the Student plays a leading role and the Teacher provides assistance or supervision, the collaborative reward function is defined as Among them, r t (π,μ) is the immediate task completion reward calculated at time step t according to the Student's strategy π and the Teacher's strategy μ; γ is the discount factor; T is the time step; P t (μ)≤0 is the penalty term when the Teacher performs auxiliary supervision at time step t to prompt the Student to change the current unreasonable strategy; On the contrary, when the teacher plays a leading role and the student provides assistance or supervision, the collaborative reward is defined as Among them, r t (π,μ) is the immediate task completion reward calculated at time step t according to the Student's strategy π and the Teacher's strategy μ; γ is the discount factor; T is the time step; -P t (π) ≥ 0 is the reward for the Student to execute the correct auxiliary supervision strategy at time step t; By maximizing the reward R collab With dynamic collaboration, reinforcement learning training based on human feedback is realized, the training of the embodied large model is continuously optimized, and the embodied intelligent model in the first physical space (1) and the natural human experimental subject in the second physical space (2) are guided to perform collaborative tasks.

9. The embodied large model training method for closed-loop feedback of human-computer interaction according to claim 6, characterized in that: The human-computer game interactive training comprises the following steps: Step C1: realizing the interaction between the human and the embodied intelligent model in the virtual space (3) by superimposing the first physical space (1) and the second physical space (2) in time and space through the virtual space (3); Step C2: The embodied intelligent model in the first physical space (1) and the natural person experimental subject in the second physical space (2) perform a confrontational game task, and are mapped and fused into a multidimensional vector of spatiotemporal data in the virtual space (3). As input for training the embodied large model; Step C3: The embodied large model is Tokenization and large-scale neural network training based on the Transformer architecture model are performed to guide the embodied intelligent model in the first physical space (1) to perform adversarial game tasks; Step C4: The competition data between the natural person experimental subject in the second physical space (2) and the twin embodied intelligent model in the virtual space (3) is used as the Teacher data set, and the competition data between the embodied intelligent model in the first physical space (1) and the twin natural person in the virtual space (3) is used as the Student data set; Step C5: Define the adversarial reward as Among them, r t (π,μ) is the immediate reward calculated at time step t according to the Student's strategy π and the Teacher's strategy μ, γ is the discount factor, and T is the time step; The adversarial loss is defined as Among them, τ represents the execution task (a series of states and action sequences), π is the strategy adopted to execute the task (a series of control output instructions), p(τ|π) is the probability of executing task τ under strategy π, and μ(τ) is the probability distribution of the Teacher's evaluation of the performance of the task; By maximizing the adversarial reward and minimizing the adversarial loss, reinforcement learning training based on human feedback is achieved, and the embodied large model training is continuously optimized to guide the embodied intelligent model in the first physical space (1) to engage in adversarial game with the natural human experimental subject in the second physical space (2).

10. A human-computer interactive closed-loop feedback embodied large model training system, characterized in that: include: A first physical space (1), a second physical space (2) and a virtual space (3), wherein the first physical space (1) includes an embodied sensor feedback system, a global motion capture system and an embodied intelligent model, and the second physical space (2) includes a wearable multi-modal sensor, a global motion capture system and a natural human experimental subject, and the virtual space (3) is used to align and fuse the first physical space (1) and the second physical space (2) in virtual and real time and space, realize the construction and rendering of the operation scene, and realize the training of the embodied model of human-computer interaction feedback.

Citation Information

Patent Citations

  • Large-space multi-person virtual reality interaction system and method

    CN111796670A

  • Universal system of intelligent robot with body, construction method and use method

    CN117549310A

Cited By

  • Home-based care autonomous intelligent robot system and construction method

    CN119188785A

  • Construction method of robot teleoperation body model and robot teleoperation system

    CN120706217A