A somatotropic large model training method and system for human-computer interaction closed-loop feedback

Abstract

The present application relates to a kind of human-computer interaction closed loop feedback somatotype big model training method and system, the training method involves 3 cross-time and space movement operation space, including first physical space, second physical space, virtual space.First physical space is used for robot somatic intelligent model training, including somatic sensing feedback system, global motion capture system and robot ontology;Second physical space is used for human behavior data acquisition and cognitive analysis, including wearable multi-mode sensor, global motion capture system and natural person experimental object.Virtual space is used to carry out virtual-real space alignment fusion between first physical space and second physical space, realizes operation scene construction rendering, realizes human-computer interaction feedback somatic model training.Compared with prior art, the present application has the advantages of learning training through human and robot cross-time and space interaction, avoids safety problems in human-computer interaction training process and the like.

Description

Technical Field

[0001] This invention belongs to the field of embodied intelligence, and in particular relates to a method and system for training embodied large models with closed-loop feedback in human-computer interaction. Background Technology

[0002] Population aging, labor shortages, and the ongoing pursuit of improved productivity and quality of life have driven the growth in demand for humanoid robots. In recent years, advancements in technologies such as artificial intelligence, machine learning, computer vision, and natural language processing have significantly improved the intelligence level of humanoid robots. Governments worldwide have introduced a series of policies and guidelines to support the development of the humanoid robot industry. Humanoid robots have become a research hotspot both domestically and internationally, but their development still faces several major challenges: the need to enhance perception, decision-making, and action capabilities to achieve more natural and efficient interaction; the need to integrate multimodal sensory information, including vision, hearing, and touch, to achieve richer and more intuitive interaction methods; the need to improve emotional interaction capabilities, enabling robots to better understand and express emotions; and the need to ensure that robot behavior complies with ethical and legal regulations, guaranteeing the safety of human-robot interaction in various social service scenarios.

[0003] To address these challenges, a foundational platform is needed for humanoid robot research and performance testing, enabling cross-scenario skill learning and training. However, humanoid robot training often raises numerous safety concerns, particularly regarding the personal safety of personnel involved in algorithm training during close human-robot collaboration. Currently, there are no training platforms or testing methods that can guarantee safety.

[0004] Chinese patent application CN112446962A discloses a method and system for generating intelligent agents based on virtual-real evolution. The specific implementation of this method includes: constructing a physical space training environment to obtain the initial environment for intelligent agent training; constructing a virtual space based on the initial environment, and training the intelligent agent in the virtual space; verifying the trained intelligent agent in the physical space to eliminate the differences between the virtual space and the physical space, thereby generating an intelligent agent based on virtual-real evolution. However, this method does not address the safety issues related to human safety during algorithm training of humanoid robots in close human-machine collaborative interaction. Summary of the Invention

[0005] The purpose of this invention is to overcome the shortcomings of the existing technology and provide a method and system for training embodied large models with closed-loop feedback in human-computer interaction.

[0006] The objective of this invention can be achieved through the following technical solutions:

[0007] This invention provides a method for training an embodied large model with closed-loop feedback in human-computer interaction, comprising the following steps:

[0008] Step S1: In the first physical space (1), acquire behavioral data of the embodied intelligent model for training tasks through the multimodal sensors on the fuselage and the full-domain motion capture system deployed in the first physical space (1), and construct the behavioral data of the embodied intelligent model into a time-series multidimensional vector.

[0009] Step S2: In the second physical space (2), the wearable multi-mode sensors deployed on the natural human experimental subjects and the full-domain motion capture system deployed in the second physical space (2) are used to acquire the behavioral data of the natural human experimental subjects performing the tasks corresponding to the embodied intelligent model training task in step S1. The behavioral data of the natural human experimental subjects is then modeled as a time-series multi-dimensional vector. The first physical space (1) and the second physical space (2) are in different time and space.

[0010] Step S3: Construct a virtual space (3) that integrates the real and virtual spaces (1) and the second physical space (2) in a spatiotemporal superposition, and model the digital environment data of the virtual space (3) as a multidimensional vector of time series.

[0011] Step S4: Multidimensional Vector Multidimensional vector With multidimensional vectors Four-dimensional ternary spatiotemporal mapping and fusion are achieved through spatiotemporal affine transformation. The multidimensional vector of the spatiotemporal data after mapping and fusion is defined as follows:

[0012] Step S5: Multi-dimensional vectorize spatiotemporal data The training dataset is used to train the embodied large model algorithm and guide the embodied intelligent model in the first physical space (1) to perform corresponding tasks.

[0013] The behavioral data used in training the embodied intelligent model includes pose data, sensor data, and interaction data.

[0014] The virtual space (3) realizes a metaverse digital environment for skill learning and model training in different task scenarios. The metaverse digital environment includes terrain, weather, lighting, static objects, and dynamic objects.

[0015] The spatiotemporal data multidimensional vector The solution expression is:

[0016] in, This is the spatiotemporal mapping function between the first physical space (1) and the virtual space scene elements. The spatiotemporal mapping function between the second physical space (2) and the virtual space scene elements.

[0017] Step S5 involves training the embodied large model algorithm. The training steps are as follows: The spatiotemporal multidimensional vector of the long-term series is... The tokenized data is imported into the embodied big model algorithm for learning and training, thereby achieving embodied intelligent operation skills training of human-like perception, positioning, cognition, decision-making, planning and execution. The embodied big model is based on the Transformer architecture model.

[0018] The algorithms for training the embodied large model include human-machine synchronous operation skills training, human-machine collaborative operation skills training, and human-machine game interaction training.

[0019] The human-machine synchronous operation skills training includes the following steps:

[0020] Step A1: Embossed large model through... Tokenization and large-scale neural network training based on the Transformer architecture model are performed to guide the embodied intelligent model in the first physical space (1) to perform synchronous operation tasks.

[0021] Step A2: Map the behavioral process data of the natural human experimental object in the second physical space (2) to the virtual space (3) to generate the job data as the Teacher dataset, and map the behavioral process data of the embodied intelligent model in the first physical space (1) to the virtual space (3) to generate the Student dataset;

[0022] Step A3: Define the consistency-based reward function as follows:

[0023]

[0024] Where, r t (π,μ) is the immediate reward calculated at time step t based on the Student's policy π and the Teacher's policy μ; γ is the discount factor; T is the time step.

[0025] R consist We measure the difference between the Student's policy π and the Teacher's policy μ. The greater the difference between the Student's and Teacher's behaviors, the greater the negative reward given. This is achieved by maximizing the reward R. consist Iterative learning based on consistency is achieved, the training of the embodied large model is continuously optimized, and the embodied intelligent model strategy in the first physical space (1) is updated to synchronize the task of the natural human experimental subjects in the second physical space (2).

[0026] The human-machine collaborative work skills training includes the following steps:

[0027] Step B1: By overlaying the first physical space (1) and the second physical space (2) in the virtual space (3), the interaction between humans and the embodied intelligent model in the virtual space is realized;

[0028] Step B2: The embodied intelligent model in the first physical space and the natural human experimental subject in the second physical space perform the same collaborative task, and are mapped and fused into a spatiotemporal data multidimensional vector in the virtual space. As input for training the embodied large model;

[0029] Step B3: Embossed large model through... Tokenization and large-scale neural network training based on the Transformer architecture model are performed to guide the embodied intelligent model in the first physical space to perform collaborative tasks.

[0030] Step B4: Use the collaborative task data between the second physical space and the virtual space as the Teacher dataset, and use the collaborative task data between the first physical space and the virtual space as the Student dataset;

[0031] Step B5: When the Student plays a leading role and the Teacher provides assistance or supervision, define the collaboration reward function as follows:

[0032]

[0033] Where, r t (π,μ) is the immediate task completion reward calculated at time step t based on the Student's policy π and the Teacher's policy μ; γ is the discount factor; T is the time step; P t (μ)≤0 is the penalty term when the Teacher performs auxiliary supervision at time step t, to prompt the Student to change the current unreasonable strategy;

[0034] Conversely, when the Teacher plays a leading role and the Student provides assistance or supervision, the collaboration reward is defined as...

[0035]

[0036] Where, r t (π,μ) is the immediate task completion reward calculated at time step t based on the Student's policy π and the Teacher's policy μ; γ is the discount factor; T is the time step; -P t (π)≥0 is the reward for executing the correct auxiliary supervision strategy for Student at time step t;

[0037] By maximizing reward R collabThrough dynamic collaboration, reinforcement learning training based on human feedback is achieved, continuously optimizing the training of the embodied large model, and guiding the embodied intelligent model in the first physical space and the natural human experimental subjects in the second physical space to perform collaborative tasks.

[0038] The human-computer game interaction training includes the following steps:

[0039] Step C1: By overlaying the first physical space (1) and the second physical space (2) in the virtual space (3), the interaction between humans and the embodied intelligent model in the virtual space is realized;

[0040] Step C2: The embodied intelligent model in the first physical space and the natural human experimental subjects in the second physical space perform adversarial game tasks, and are mapped and fused into spatiotemporal data multidimensional vectors in the virtual space. As input for training the embodied large model;

[0041] Step C3: Embossed large model through... Tokenization and large-scale neural network training based on the Transformer architecture model are performed to guide the embodied intelligent model in the first physical space to perform adversarial game tasks.

[0042] Step C4: Use the competition data between the natural human experimental subjects in the second physical space (2) and the twin embodied intelligent model in the virtual space (3) as the Teacher dataset, and use the competition data between the embodied intelligent model in the first physical space (1) and the twin natural human in the virtual space (3) as the Student dataset;

[0043] Step C5: Define the adversarial reward as

[0044]

[0045] Where, r t (π,μ) is the immediate reward calculated at time step t based on the Student's policy π and the Teacher's policy μ, where γ is the discount factor and T is the time step.

[0046] Define adversarial loss as

[0047]

[0048] Where τ represents the task to be executed (a series of states and action sequences), π is the strategy adopted to execute the task (a series of control output instructions), p(τ|π) is the probability of executing task τ under strategy π, and μ(τ) is the probability distribution of the Teacher's evaluation of the quality of the task execution.

[0049] By maximizing adversarial rewards and minimizing adversarial losses, reinforcement learning training based on human feedback is achieved, continuously optimizing the training of the embodied large model, and guiding the embodied intelligent model in the first physical space to engage in adversarial games with natural persons in the second physical space.

[0050] This invention also provides a human-computer interaction closed-loop feedback embodied large model training system, characterized by comprising: a first physical space, a second physical space, and a virtual space. The first physical space includes an embodied sensing feedback system, a full-domain motion capture system, and an embodied intelligent model. The second physical space includes a wearable multi-mode sensor, a full-domain motion capture system, and a natural human experimental subject. The virtual space is used to perform real-time and virtual-space alignment and fusion of the first physical space and the second physical space to realize the construction and rendering of the work scene and to realize the training of the human-computer interaction feedback embodied model.

[0051] Compared with the prior art, the present invention has the following advantages:

[0052] (1) Learning and training through cross-temporal and spatial interaction between humans and robots avoids safety issues in the human-computer interaction training process.

[0053] (2) By integrating virtual and real data, the generation of human-computer interaction datasets in different scenarios was realized, which solved the problem of the lack of big data for human-computer interaction in embodied large models.

[0054] (3) A human feedback reinforcement learning training method based on virtual-real fusion and spatiotemporal superposition was proposed to realize different types of experiments such as human-machine synchronous operation skills training, human-machine collaborative operation skills training, and human-machine game interaction training.

[0055] (4) It provides effective solutions for skill learning in different real-world embodied intelligence application scenarios, and is expected to promote the application and promotion of humanoid robots. Attached Figure Description

[0056] Figure 1 This is a diagram of the large model training platform architecture of the method of this invention;

[0057] Figure 2 Diagram of the training framework for a large model;

[0058] Attached reference numerals: 1. First physical space, 2. Second physical space, 3. Virtual space. Detailed Implementation

[0059] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0060] This invention discloses a cross-temporal and spatial virtual-real fusion human-computer interaction feedback embodied model training method to solve the safety problems of human-computer interaction training for humanoid robots. This invention is applicable to large-scale data collection, scene generation, and safe human-computer interaction feedback reinforcement learning training for embodied intelligence such as humanoid robots, providing an effective solution for skill learning in different practical embodied intelligence application scenarios, and is expected to promote the application and promotion of humanoid robots.

[0061] This invention provides a training method for embodied large models based on virtual-real fusion, spatiotemporal overlay, and closed-loop feedback in human-computer interaction. This training method involves three spatiotemporal motion operation spaces, including a first physical space (robot training field), a second physical space (human behavioral cognition field), and a virtual space (metaverse training field), such as... Figure 1 As shown. The first physical space is used for training the robot's embodied intelligent model, including the embodied sensing feedback system, the full-domain motion capture system, and the robot itself; the second physical space is used for human behavior data collection and cognitive analysis, including wearable multi-modal sensors, the full-domain motion capture system, and natural human experimental subjects. The virtual space is used to perform real-time and virtual-space alignment and fusion of the first and second physical spaces to realize the construction and rendering of the work scene and to realize the training of the embodied model for human-computer interaction feedback.

[0062] A large-scale model training method based on virtual-real fusion, spatiotemporal overlay, and closed-loop feedback in human-computer interaction includes:

[0063] Step S1: Construct the first physical space (1) for training the humanoid robot's embodied intelligent model, deploy multimodal sensors on the robot body and a full-domain motion capture system deployed in the indoor environment of the physical space, and acquire behavioral data (including pose data, sensor data, interaction data, etc.) for training the humanoid robot's embodied intelligent model. The data set is constructed into a time-series multidimensional vector.

[0064] Step S2: Construct a second physical space (2) for human behavior data collection and cognitive analysis. The second physical space (2) is located in a different time and space from the first physical space (1). In the second physical space (2), wearable multi-modal sensors are deployed on the natural human experimental subjects, and a full-domain motion capture system is deployed in the indoor environment of the physical space. Data on the natural human behavior is acquired for analysis, and the natural human behavior data is modeled as a multi-dimensional vector of time series.

[0065] Step S3: Based on the computing power and rendering computing power platform, construct a virtual space to realize the spatiotemporal superposition of the first physical space (1) and the second physical space (2) to achieve virtual-real fusion. Realize the metaverse digital environment (including terrain, weather, lighting, static objects, dynamic objects, etc.) for skill learning and model training in different task scenarios. The digital environment data of the virtual space is modeled as a multi-dimensional vector of time series.

[0066] Step S4: The first physical space (1) and the second physical space (2) are transformed in the virtual space through a spatiotemporal affine transformation to achieve four-dimensional ternary spatiotemporal mapping and fusion. The multidimensional vector of the spatiotemporal data after mapping and fusion is defined as... The solution expression is as follows:

[0067]

[0068] Step S5: Mapping the fused spatiotemporal data multidimensional vector The training dataset for large-scale robot embodied model training is used as the training dataset for large-scale robot embodied model training. The large-scale model training framework is as follows: Figure 2 As shown. Multidimensional vectors of spatiotemporal data. As input for training large models, the spatiotemporal multidimensional vector data of long time series is first tokenized and then imported into a large-scale neural network based on the Transformer architecture model for learning and training, so as to achieve training of human-like embodied intelligent operation skills such as perception, localization, cognition, decision-making, planning and execution.

[0069] Step S6: The algorithm categories for training the embodied large model include three types: human-computer synchronous operation skill training, human-computer collaborative operation skill training, and human-computer game interaction training. Different dataset classifications and training strategies are assigned to different types.

[0070] Step S6, human-machine synchronized operation skill training, taking the skill of chopping vegetables in a kitchen scenario as an example, specifically includes:

[0071] Step S61: The robot in the first physical space (1) and the natural person in the second physical space (2) perform the same vegetable-cutting task, and are mapped and fused into a spatiotemporal data multidimensional vector in the virtual space. As input for training the embodied large model, i.e.

[0072] Step S62: Embossed large model through... Tokenization and large-scale neural network training based on the Transformer architecture model are performed to guide the robot in the first physical space (1) to perform the vegetable cutting task;

[0073] Step S63: Map the data of the natural person cutting vegetables in the second physical space (2) to the virtual space to generate job data as the Teacher dataset, i.e.

[0074] Step S64: Map the process data of the robot cutting vegetables in the first physical space (1) to the production data in the virtual space as the Student dataset, i.e.

[0075] Step S65: Define the consistency-based reward function as follows:

[0076]

[0077] Where, r t (π,μ) is the immediate reward calculated at time step t based on the Student's policy π and the Teacher's policy μ; γ is the discount factor; T is the time step.

[0078] R consist We measure the difference between the Student's policy π and the Teacher's policy μ. The greater the difference between the Student's and Teacher's behaviors, the greater the negative reward given. This is achieved by maximizing the reward R. consist Iterative learning based on consistency is achieved, continuously optimizing the training of the embodied large model and updating the robot policy in the first physical space (1) to synchronize the vegetable-cutting action of the natural person in the second physical space (2).

[0079] Step S6 specifically includes: human-machine collaborative work skills training, taking the coordination of steps and movements required for a duet dance as an example, specifically including:

[0080] Step S61: By spatiotemporally superimposing the first physical space (1) and the second physical space (2) in a virtual space, the interaction between humans and robots in the virtual space is realized, that is...

[0081] Step S62: The robot in the first physical space (1) and the natural person in the second physical space (2) perform the same duet dance performance task, and map and merge them into a spatiotemporal data multidimensional vector in the virtual space. As input for training the embodied large model, i.e.

[0082] Step S63: Embossed large model through... Tokenization and large-scale neural network training based on the Transformer architecture model are performed to guide the robot in the first physical space (1) to perform a duet dance performance task.

[0083] Step S64: Map the process data of the natural person's dance performance in the second physical space (2) to the virtual space to generate assignment data as the Teacher dataset, i.e.

[0084] Step S65: Map the process data of the robot dance performance in the first physical space (1) to the virtual space production data as the Student dataset, i.e.

[0085] Step S66: In the partner dance task, the Student and Teacher can play different roles to promote the Student's independent learning ability. For example, when the Student plays the leading role and the Teacher provides assistance or supervision, the collaborative reward function is defined as follows:

[0086]

[0087] Where, r t (π,μ) is the immediate task completion reward calculated at time step t based on the Student's policy π and the Teacher's policy μ; γ is the discount factor; T is the time step; P t (μ)≤0 is the penalty term when the Teacher performs auxiliary supervision at time step t, to prompt the Student to change the current unreasonable strategy.

[0088] Conversely, when the Teacher plays a leading role and the Student provides assistance or supervision, the collaboration reward is defined as...

[0089]

[0090] Where, r t (π,μ) is the immediate task completion reward calculated at time step t based on the Student's policy π and the Teacher's policy μ; γ is the discount factor; T is the time step; -P t (π)≥0 is the reward for executing the correct auxiliary supervision strategy for Student at time step t.

[0091] By maximizing reward R collab The training of the embodied large model is continuously optimized through dynamic collaboration to enable the robot in the first physical space (1) and the natural person in the second physical space (2) to collaborate in a duet performance.

[0092] Step S6 specifically includes: human-computer interactive training, taking a two-person martial arts competition as an example, specifically including:

[0093] Step S61: By spatiotemporally superimposing the first physical space (1) and the second physical space (2) in a virtual space, the interaction between humans and robots in the virtual space is realized, that is...

[0094] Step S62: The robot in the first physical space (1) and the natural person in the second physical space (2) perform a martial arts competition task, and are mapped and fused into a spatiotemporal data multidimensional vector in the virtual space. As input for training the embodied large model, i.e.

[0095] Step S63: Embossed large model through... Tokenization and large-scale neural network training based on the Transformer architecture model are performed to guide the robot in the first physical space (1) to perform martial arts competition tasks.

[0096] Step S64: Use the martial arts competition data between the natural person in the second physical space (2) and the virtual twin robot as the Teacher dataset, i.e.

[0097] Step S65: Use the martial arts competition data between the robot in the first physical space (1) and the twin natural person in the virtual space as the Student dataset, i.e.

[0098] Step S66: Define the adversarial reward as

[0099]

[0100] Where, r t (π,μ) is the immediate reward calculated at time step t based on the Student's policy π and the Teacher's policy μ, where γ is the discount factor and T is the time step.

[0101] Define adversarial loss as

[0102]

[0103] Where τ represents the task to be executed (a series of states and action sequences), π is the strategy adopted to execute the task (a series of control output instructions), p(τ|π) is the probability of executing task τ under strategy π, and μ(τ) is the probability distribution of the Teacher's evaluation of the quality of the task execution.

[0104] By maximizing adversarial rewards and minimizing adversarial losses, reinforcement learning (RLHF) training based on human feedback is achieved, continuously optimizing the training of the embodied large model to guide the robot in the first physical space (1) to engage in martial arts competition with the natural person in the second physical space (2).

[0105] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in the present invention, and these modifications or substitutions should all be covered within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A method for training an embodied large model with closed-loop feedback in human-computer interaction, characterized in that, Includes the following steps: Step S1: In the first physical space (1), acquire behavioral data of the embodied intelligent model for training tasks through the multimodal sensor of the fuselage and the full-domain motion capture system deployed in the first physical space (1), and construct the behavioral data of the embodied intelligent model into a time series multidimensional vector. ; Step S2: In the second physical space (2), the wearable multi-mode sensors deployed on the natural human experimental subjects and the full-domain motion capture system deployed in the second physical space (2) are used to acquire the behavioral data of the natural human experimental subjects performing the tasks corresponding to the embodied intelligent model training task in step S1. The behavioral data of the natural human experimental subjects is modeled as a time series multi-dimensional vector. The first physical space (1) and the second physical space (2) are in different time and space. Step S3: Construct a virtual space (3) that integrates the real and virtual spaces (1) and the second physical space (2) in a spatiotemporal superposition, and model the digital environment data of the virtual space (3) as a multidimensional vector of time series. ; Step S4: Multidimensional Vector Multidimensional vectors With multidimensional vectors Four-dimensional ternary spatiotemporal mapping and fusion are achieved through spatiotemporal affine transformation. The multidimensional vector of the spatiotemporal data after mapping and fusion is defined as follows: ; Step S5: Multi-dimensional vectorize spatiotemporal data The training dataset is used as the training dataset for training the embodied large model to carry out the training of the embodied large model algorithm and guide the embodied intelligent model in the first physical space (1) to perform the corresponding task. Step S5 involves training the embodied large model algorithm. The training steps are as follows: The spatiotemporal multidimensional vector of the long-term series is... The tokenized data is then imported into the embodied big model algorithm for learning and training, thereby achieving embodied intelligent operational skills training for human-like perception, positioning, cognition, decision-making, planning, and execution. The embodied big model is based on the Transformer architecture model. The algorithm categories for training the embodied large model include human-machine synchronous operation skills training, human-machine collaborative operation skills training, and human-machine game interaction training. The human-machine synchronous operation skills training includes the following steps: Step A1: Embossed large model through... Tokenization and large-scale neural network training based on the Transformer architecture model are performed to guide the embodied intelligent model in the first physical space (1) to perform synchronous operation tasks; Step A2: Map the behavioral process data of the natural human experimental subjects in the second physical space (2) to the virtual space (3) to generate the homework data as the Teacher dataset, and map the behavioral process data of the embodied intelligent model in the first physical space (1) to the virtual space (3) to generate the data as the Student dataset; Step A3: Define the consistency-based reward function as follows: in, In time step t Below, based on the Student's strategy π Strategies with Teachers μ Calculated instant rewards; γ It is a discount factor; T It is the time step; Strategies for measuring students π Strategies with Teachers μ The greater the difference between the student's and teacher's behavior, the greater the negative reward is given, thus maximizing the reward. Achieving consistency-based iterative learning, continuously optimizing the training of the embodied large model, and updating the embodied intelligent model strategy in the first physical space (1) to synchronize the task of the natural human experimental subjects in the second physical space (2).

2. The embodied large model training method with closed-loop feedback in human-computer interaction according to claim 1, characterized in that, The behavioral data used in training the embodied intelligent model includes pose data, sensor data, and interaction data.

3. The embodied large model training method with closed-loop feedback in human-computer interaction according to claim 1, characterized in that, The virtual space (3) is a metaverse digital environment for learning skills and training models in different job task scenarios. The metaverse digital environment includes terrain, weather, lighting, static objects, and dynamic objects.

4. The embodied large model training method with closed-loop feedback in human-computer interaction according to claim 1, characterized in that, The spatiotemporal data multidimensional vector The solution expression is: in, The spatiotemporal mapping function for scene elements in the first physical space (1) and the virtual space (3) is given. The spatiotemporal mapping function for scene elements in the second physical space (2) and the virtual space (3).

5. The embodied large model training method with closed-loop feedback in human-computer interaction according to claim 1, characterized in that, The human-machine collaborative work skills training includes the following steps: Step B1: By overlaying the first physical space (1) and the second physical space (2) in the virtual space (3), the interaction between humans and the embodied intelligent model in the virtual space (3) is realized; Step B2: The embodied intelligent model in the first physical space (1) and the natural human experimental subject in the second physical space (2) perform the same collaborative task, and are mapped and fused into a spatiotemporal data multidimensional vector in the virtual space (3). , as input for training the embodied large model; Step B3: Embossed large model through... Tokenization and large-scale neural network training based on the Transformer architecture model are performed to guide the embodied intelligent model in the first physical space (1) to perform collaborative tasks. Step B4: Use the interactive collaborative task data between the second physical space (2) and the virtual space (3) as the Teacher dataset, and use the interactive collaborative task data between the first physical space (1) and the virtual space (3) as the Student dataset; Step B5: When the Student plays a leading role and the Teacher provides assistance or supervision, define the collaboration reward function as follows: in, In time step t Below, based on the Student's strategy π Strategies with Teachers μ Calculate the instant reward for task completion; γ It is a discount factor; T It is the time step; In time step t Below, the teacher implements penalty items when performing auxiliary supervision to prompt the student to change the current unreasonable strategy; Conversely, when the Teacher plays a leading role and the Student provides assistance or supervision, the collaboration reward is defined as... in, In time step t Below, based on the Student's strategy π Strategies with Teachers μ Calculate the instant reward for task completion; γ It is a discount factor; T It is the time step; In time step t The following are rewards for students who execute correct auxiliary supervision strategies; By maximizing rewards Through dynamic collaboration, reinforcement learning training based on human feedback is achieved, continuously optimizing the training of the embodied large model, and guiding the embodied intelligent model in the first physical space (1) and the natural human experimental subjects in the second physical space (2) to perform collaborative tasks.

6. The embodied large model training method with closed-loop feedback in human-computer interaction according to claim 1, characterized in that, The human-computer game interaction training includes the following steps: Step C1: By overlaying the first physical space (1) and the second physical space (2) in the virtual space (3), the interaction between humans and the embodied intelligent model in the virtual space (3) is realized; Step C2: The embodied intelligent model in the first physical space (1) and the natural human experimental subjects in the second physical space (2) perform adversarial game tasks, and are mapped and fused into spatiotemporal data multidimensional vectors in the virtual space (3). , as input for training the embodied large model; Step C3: Embossed large model through... Tokenization and large-scale neural network training based on the Transformer architecture model are performed to guide the embodied intelligent model in the first physical space (1) to perform adversarial game tasks; Step C4: Use the competition data between the natural human experimental subjects in the second physical space (2) and the twin embodied intelligent model in the virtual space (3) as the Teacher dataset, and use the competition data between the embodied intelligent model in the first physical space (1) and the twin natural human in the virtual space (3) as the Student dataset. Step C5: Define the adversarial reward as in, In time step t Below, based on the Student's strategy π Strategies with Teachers μ Calculated instant reward γ It is a discount factor. T It is the time step; Define adversarial loss as in, τ This represents the execution of a task, a sequence of states and actions. π The strategy adopted to execute the task, a series of control output instructions, p ( τ | π ) is in the strategy π Execute the task τ The probability, μ ( τ ) is the probability distribution by which the teacher evaluates the quality of task execution; By maximizing adversarial rewards and minimizing adversarial losses, reinforcement learning training based on human feedback is achieved, continuously optimizing the training of the embodied large model, and guiding the embodied intelligent model in the first physical space (1) to engage in adversarial games with the natural human experimental subjects in the second physical space (2).

7. A training system for the embodied large model training method of human-computer interaction closed-loop feedback as described in any one of claims 1 to 6, characterized in that, include: The first physical space (1), the second physical space (2), and the virtual space (3) are: the first physical space (1) includes an embodied sensing feedback system, a full-domain motion capture system, and an embodied intelligent model; the second physical space (2) includes a wearable multi-mode sensor, a full-domain motion capture system, and a natural human experimental subject; and the virtual space (3) is used to perform virtual-real-time alignment and fusion of the first physical space (1) and the second physical space (2) to realize the construction and rendering of the work scene and to realize the training of the human-computer interaction feedback embodied model.

Images