Teaching method and device based on physical and social joint simulation and medium

Abstract

The invention provides a teaching method and device based on physical and social joint simulation and a medium, and relates to the field of teaching learning of agents. The method comprises the following steps: creating a virtual substitute in a pre-created virtual environment; accessing a user behavior into the pre-created virtual environment by using a preset interaction device so as to drive the virtual substitute to execute a teaching task through the user behavior; in the process of executing the teaching task by the virtual substitute, extracting physical behaviors and social behaviors of a user from the virtual substitute to obtain teaching data; and enabling the intelligent agent to learn the teaching data through observation type learning and / or interactive learning so as to update the model parameters of the intelligent agent. According to the scheme, physical and social task joint analog simulation is realized, practical operation of a user is accessed to the platform for teaching demonstration of various tasks to generate corresponding data for an intelligent agent to learn, and the skill and knowledge level of the intelligent agent is remarkably improved.

Description

Technical Field

[0001] This invention relates to the field of teaching and learning of intelligent agents, and more particularly to a teaching method, device and medium based on physical and social joint simulation. Background Technology

[0002] Most current simulation platforms focus on physical simulation, scene simulation, and interaction logic simulation, such as MUJOCO, AI2THOR, and Habitat. Additionally, platforms like VirtualHome, while including multiple characters, are limited to physical interactions between characters and scene objects, lacking in-depth development in social simulation (including agent mental states and social behaviors). Therefore, they are ill-suited for simulating tasks involving complex social interactions. Unifying the simulation of physical and social spaces is a highly complex problem and is crucial for constructing effective learning environments for intelligent agents.

[0003] In platforms that include complex task simulations, enabling agents to learn autonomously is a crucial research question. Typically, even though humans can generate teaching data using various methods, agents require special design to utilize this data efficiently. The most common teaching-based learning paradigms are learn from demonstration (LfD) or imitation learning, which can extract information from teaching data and train the agent's skills. However, this type of learning usually involves collecting data beforehand, i.e., offline learning, and cannot provide targeted demonstrations based on the agent's learning problems, lacking interactivity. Therefore, achieving highly efficient interactive learning is also a critical issue. Summary of the Invention

[0005] This invention provides a teaching method, device, and medium based on physical and social joint simulation to address the problems in existing technologies that lack simulation of the mental state and social behavior of intelligent agents, and that the single offline learning method results in poor interactivity. It realizes a simulation platform that can jointly simulate physical and social tasks. Human users can access the platform to demonstrate various tasks, generate corresponding data, and provide intelligent agents with learning opportunities to improve their skills and knowledge.

[0006] According to a first aspect of the present invention, the present invention provides a teaching method based on physical and social co-simulation, the method comprising: Create virtual avatars within a pre-created virtual environment; By using a preset interactive device, user behavior is integrated into the pre-created virtual environment, so that the virtual avatar can be driven to perform teaching tasks through user behavior; During the process of the virtual avatar performing the teaching task, the user's physical and social behaviors are extracted from the virtual avatar to obtain teaching data; The agent learns from the teaching data through observational learning and / or interactive learning to update the agent's model parameters.

[0007] According to the teaching method based on physical and social co-simulation of the present invention, user behavior is integrated into the pre-created virtual environment using a preset interactive device, so as to drive the virtual avatar to perform teaching tasks through user behavior, including: The VR device worn by the user records the user's full-body posture, hand movements, and eye gaze direction, and transmits the recorded data to the virtual avatar so that the virtual avatar can perform the same full-body posture, hand movements, and eye gaze direction as the user in the virtual environment; and / or

[0008] The virtual avatar performs tasks in the virtual environment through user control of the mouse and keyboard, wherein different mouse and keyboard operations correspond to different preset task behaviors; and / or

[0009] The user's natural language commands are decomposed into a sequence of executable atomic tasks that drive the virtual avatar using a pre-defined language understanding tool, and the virtual avatar is then driven to perform tasks in the virtual environment using the atomic task sequence.

[0010] According to the teaching method based on physical and social co-simulation of the present invention, the user's physical and social behaviors are extracted from the virtual avatar to obtain teaching data, including: A pre-built physics engine is used to provide feedback on the physical behavior of the virtual avatar, thereby obtaining physical interaction data; A social engine is formed by using a pre-set social interaction logic to provide feedback on the social behavior of virtual avatars and obtain social behavior data.

[0011] According to the teaching method based on physical and social co-simulation of the present invention, the user's physical and social behaviors are extracted from the virtual avatar to obtain teaching data, including: Unified representation: The physics engine updates the state of the physical simulation space, and the social engine updates the state of the social simulation space. The state of the physical simulation space is vectorized, and the state of the social simulation space is also vectorized. The vectors of the physical simulation space and the vectors of the social simulation space are concatenated to form a vector with a higher dimension. The dimension is equal to the sum of the dimensions of the physical simulation space vector and the social simulation space vector, resulting in a unified space as the physical-social joint simulation space. State Update: Vectors in the physical-social co-simulation space are used to represent the state of the space. The transition between states is accomplished through a state transition matrix, which realizes the transition from one state vector to another. Any behavior of the virtual avatar is equivalent to a state transition matrix, which will update the state of the virtual space containing physical and social simulation.

[0012] According to the teaching method based on physical and social co-simulation of the present invention, the observational learning includes: Learning objectives are set by the intelligent agent, which sets the learning objectives and maps them to changes in the state of the scene. Extract effective information from observation data, and use mature visual semantic extraction methods or real data from simulation platforms to extract events with significant semantic information during task execution; Statistical learning is performed on a specified learning task. The maximum likelihood estimation method is used to fit the execution process of the specified task, and the fitted model can effectively represent the existing data.

[0013] According to the teaching method based on physical and social co-simulation of the present invention, the interactive learning includes: An intelligent agent works with a user to perform a task, wherein the intelligent agent has the ability to perform the task and can interact with the user during the task; In response to the task execution process, the intelligent agent raises questions. The intelligent agent's questioning ability requires pre-learning a questioner. After the intelligent agent receives visual and linguistic information input, it analyzes the input information to extract effective knowledge and compares it with its own stored knowledge to determine whether to ask a question. If the intelligent agent determines that a question needs to be asked, it will ask a question to the human user based on the knowledge in the input information and wait for the user's answer. User feedback is used as answers or feedback to the questions raised by the agent and incorporated into the agent's optimization process, serving as new data to assist the agent in updating the model.

[0014] According to the teaching method based on physical and social co-simulation of the present invention, the pre-created virtual environment is obtained through the following steps: Digital assets are created based on environmental requirements, including buildings, streets, houses, and interior furniture. The digital assets are adjusted according to the layout of objects that conform to the living habits of general users to form a universal 3D virtual environment; or

[0015] The digital assets are adjusted according to the user's personal preferences for object layout to create a personalized 3D virtual environment.

[0016] According to a second aspect of the present invention, the present invention also provides a teaching device based on physical and social co-simulation, the device comprising: A creation module is used to create virtual avatars within a pre-created virtual environment; The access module is used to access user behavior into the pre-created virtual environment using a preset interactive device, so as to drive the virtual avatar to perform teaching tasks through user behavior; The extraction module is used to extract the user's physical and social behaviors from the virtual avatar during the virtual avatar's performance of teaching tasks, thereby obtaining teaching data. The learning module is used to enable the agent to learn from the teaching data through observational learning and / or interactive learning in order to update the agent's model parameters.

[0017] According to a third aspect of the invention, the invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the program, implements the teaching method based on physical and social co-simulation as described above.

[0018] According to a fourth aspect of the invention, the invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the teaching method based on physical and social co-simulation as described above.

[0019] This invention provides a teaching method based on physical and social co-simulation. First, a virtual avatar is created in a pre-created virtual environment. Then, user behavior is integrated into the pre-created virtual environment using a preset interactive device. The user's behavior drives the virtual avatar to perform teaching tasks. Next, the user's physical and social behaviors are extracted from the virtual avatar. Finally, a specific learning method is used to enable the intelligent agent to learn from the extracted physical and social behaviors, thereby updating the model parameters. This achieves joint simulation of physical and social tasks. The user's actual operations are integrated into the platform for teaching demonstrations of various tasks, generating corresponding data for the intelligent agent to learn from, significantly improving the agent's skills and knowledge level.

[0020] In addition, the teaching device based on physical and social co-simulation, an electronic device, and a non-transitory computer-readable storage medium provided by the present invention can also achieve the above-mentioned technical effects, which will not be elaborated here. Attached Figure Description

[0021] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0022] Figure 1 This is one of the flowcharts of the teaching method based on physical and social joint simulation provided by the present invention.

[0023] Figure 2 This is a hardware schematic diagram of a teaching method for implementing a physical and social co-simulation method provided by the present invention.

[0024] Figure 3 This is a schematic diagram of data interaction in the teaching method based on physical and social joint simulation provided by the present invention.

[0025] Figure 4 This is the second flowchart of the teaching method based on joint physical and social simulation provided by this invention.

[0026] Figure 5 This is a schematic diagram of the teaching device based on physical and social joint simulation provided by the present invention.

[0027] Figure 6 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0028] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0029] The following is combined with Figures 1 to 6 The present invention describes a teaching method based on physical and social co-simulation, a teaching device based on physical and social co-simulation, an electronic device, and a non-transitory computer-readable storage medium.

[0030] Figure 1 This is a flowchart illustrating the teaching method based on physical and social co-simulation provided in this embodiment of the invention. Please refer to it. Figure 1 As shown, this embodiment provides a teaching method based on physical and social co-simulation, which can be implemented through steps S101 to S104. Each step will be described in detail below: Step S101: Create a virtual avatar in a pre-created virtual environment.

[0031] In this embodiment, the virtual environment is a computer-generated simulation environment. It can be completely fictional or a simulation of part or all of the real world. The virtual environment in this embodiment is pre-created. There are many ways to create a virtual environment, and existing virtual environment tools and platforms can be used. A virtual avatar is a digital representative, typically used to represent a user in a virtual environment. Virtual avatars can be used for various purposes, including social interaction, games, education, presentation tasks, etc.

[0032] Step S102: Use a preset interactive device to connect user behavior to the pre-created virtual environment, so as to drive the virtual avatar to perform teaching tasks through user behavior.

[0033] In this embodiment, the preset interaction device refers to a hardware tool that allows users to interact with a computer system, software application, or virtual environment. The preset interaction device may be a virtual reality headset, eye tracker, wearable device, AR glasses, mouse and keyboard of a personal computer, gesture recognition device, motion controller, and mobile phone, etc.

[0034] Step S103: During the process of the virtual avatar performing the teaching task, the user's physical and social behaviors are extracted from the virtual avatar to obtain teaching data.

[0035] In this embodiment, there are many ways to extract physical and social behaviors from virtual enhancements. User behavior data can be extracted based on independent physical and social engines, or it can be extracted based on a unified physical-social joint engine. In specific implementation, to facilitate the use of teaching data by the intelligent agent, the extracted physical and social behavior data can be stored first, and subsequent learning can be carried out after the data scale reaches a certain level.

[0036] Step S104: The agent learns from the teaching data through observational learning and / or interactive learning to update the agent's model parameters.

[0037] In this embodiment, observational learning refers to human users directly demonstrating the execution methods of various tasks, generating effective teaching data, which is stored in the background for the intelligent agent to learn from. This learning method does not require the intelligent agent to participate in real-time interaction. Interactive learning refers to the intelligent agent, in a pre-trained version, interacting with human users to execute various tasks and generate task data. The intelligent agent can be updated online or offline. The intelligent agent can use the teaching data generated by observational learning alone, or the teaching data generated by interactive learning alone, or it can use both of the above learning methods simultaneously to learn from the teaching data.

[0038] This embodiment of the teaching method based on physical and social co-simulation first creates a virtual avatar in a pre-created virtual environment, then uses a preset interactive device to connect user behavior to the pre-created virtual environment, so that the virtual avatar can perform teaching tasks through user behavior. Next, the user's physical and social behaviors are extracted from the virtual avatar. Finally, through a specific learning method, the intelligent agent learns from the extracted physical and social behaviors to update the model parameters, realizing the joint simulation of physical and social tasks. The actual operation of the user is connected to the platform to demonstrate various tasks and generate corresponding data for the intelligent agent to learn, which significantly improves the skill and knowledge level of the intelligent agent.

[0039] In some possible implementations, step S102, which utilizes a preset interactive device to integrate user behavior into the pre-created virtual environment, drives the virtual avatar to perform teaching tasks through user behavior, specifically includes: The VR device worn by the user records the user's full-body posture, hand movements, and eye gaze direction, and transmits the recorded data to the virtual avatar, enabling the virtual avatar to perform the same full-body posture, hand movements, and eye gaze direction as the user in the virtual environment; for example, the user can wear the VR device to walk, and the VR device can drive the virtual avatar to perform the same full-body posture, hand movements, and eye gaze direction as the user; and / or

[0040] The virtual avatar performs tasks in the virtual environment through user control of the mouse and keyboard, wherein different mouse and keyboard operations correspond to different preset task behaviors; for example, mouse movement can control the virtual avatar's head orientation and forward direction, keyboard arrow keys can control the virtual avatar's forward, backward, left, and right movement, and other keyboard and mouse keys can trigger preset interactive functions; and / or

[0041] A pre-defined language understanding tool decomposes the user's natural language commands into a sequence of executable atomic tasks that drive the virtual avatar. This sequence of atomic tasks then drives the virtual avatar to perform tasks within the virtual environment. The pre-defined language understanding tool can be a large model such as GPT4. For example, a user saying "It's raining outside, I need to go out and tidy up the things in the yard" might be broken down into "stand up," "walk to the door," "pick up the umbrella," "open the door," and "walk outside," thus enabling the user to control the virtual avatar to perform tasks.

[0042] The teaching method based on physical and social co-simulation in this embodiment allows users to drive the virtual avatar through various means such as wearing VR devices, controlling the mouse and keyboard, and using natural language commands. This greatly enriches the user's interaction methods, has a high degree of intelligence, and improves the interaction between the user and the virtual environment.

[0043] In some possible implementations, in the aforementioned step S103, the user's physical and social behaviors are extracted from the virtual avatar to obtain teaching data, specifically including: A pre-built physics engine is used to provide feedback on the physical behavior of the virtual avatar, obtaining physical interaction data. The pre-built physics engine refers to a pre-set software component used to simulate real-world physical phenomena. This software component includes various computational rules, each calculating motion, rotation, and collision responses by assigning realistic physical properties to objects. The physics engine can simulate the motion and interactions of various objects in the real world. This physics engine can be Unity Physics Engine, Unreal Engine Physics, Unity3D, Bullet, etc. It should be noted that the physics engines listed above are for illustrative purposes only and should not be construed as limiting the invention. For example, if the virtual avatar knocks a cup off a table and onto the ground, the physics engine can extract the physically realistic effect of the cup breaking; if the virtual avatar pushes a window, the physics engine can extract the physical feedback of the window opening.

[0044] A social engine, based on pre-defined social interaction logic, provides feedback on the social behaviors of virtual avatars, generating social behavior data. For example, a user's social actions (such as waving or smiling at someone) will be evaluated by the social engine, which calculates corresponding responses. For instance, if a user waves to an AI-controlled virtual person on the street, the avatar will respond with a wave based on the engine's calculations.

[0045] The teaching method based on physical and social joint simulation in this embodiment extracts user behavior data through independent physical and social engines, and can accurately obtain the physical and social behaviors performed by the user-driven virtual avatar that can trigger corresponding events.

[0046] In some possible implementations, in the aforementioned step S103, the user's physical and social behaviors are extracted from the virtual avatar to obtain teaching data, including: Unified representation: The physics engine updates the state of the physical simulation space, and the social engine updates the state of the social simulation space. The state of the physical simulation space is vectorized, and the state of the social simulation space is also vectorized. The vectors of the physical simulation space and the vectors of the social simulation space are concatenated to form a vector with a higher dimension. The dimension is equal to the sum of the dimensions of the physical simulation space vector and the social simulation space vector, resulting in a unified space as the physical-social joint simulation space. State Update: Vectors in the physical-social co-simulation space are used to represent the state of the space. The transition between states is accomplished through a state transition matrix, which realizes the transition from one state vector to another. Any behavior of the virtual avatar is equivalent to a state transition matrix, which will update the state of the virtual space containing physical and social simulation.

[0047] The teaching method based on physical and social joint simulation in this embodiment extracts user behavior data through a unified physical-social joint engine, and can also accurately obtain the corresponding events caused by the physical and social behaviors of the user-driven virtual avatar.

[0048] In some possible implementations, the observational learning in step S104 specifically includes: Learning objectives are set by the intelligent agent, which sets the learning objectives and maps them to changes in the state of the scene. For example, if the learning objective is "cooking", the corresponding scene change is from the unprocessed state of the raw materials of various vegetables in the kitchen to the state of cooked food.

[0049] Extract effective information from observational data, and use mature visual semantic extraction methods or real data from simulation platforms to extract events with significant semantic information in task execution, such as "putting the food into the pot" or "mixing the ingredients into the container".

[0050] Statistical learning is performed on a specified learning task. The maximum likelihood estimation method is used to fit the execution process of the specified task. The fitted model can effectively represent the existing data. This model is the knowledge learned through observation.

[0051] The teaching method based on physical and social joint simulation in this embodiment enables the intelligent agent to learn how to perform various tasks by observing the user's direct demonstrations, thus ensuring that the intelligent agent has conventional teaching and learning capabilities.

[0052] In some possible implementations, the interactive learning described in step S104 specifically includes: An intelligent agent works with a user to perform a task, wherein the intelligent agent has the ability to perform the task and can interact with the user during the task; In response to the task execution process, the intelligent agent raises questions. The intelligent agent's questioning ability requires pre-learning a questioner. After the intelligent agent receives visual and linguistic information input, it analyzes the input information to extract effective knowledge and compares it with its own stored knowledge to determine whether to ask a question. If the intelligent agent determines that a question needs to be asked, it will ask a question to the human user based on the knowledge in the input information and wait for the user's answer. User feedback is used as answers or feedback to the questions raised by the agent and incorporated into the agent's optimization process, serving as new data to assist the agent in updating the model.

[0053] The teaching method based on physical and social joint simulation in this embodiment enables the intelligent agent to interact with human users and perform various tasks through interactive learning, thereby giving the intelligent agent the ability to learn actively and improving the learning effect of the intelligent agent.

[0054] In some possible implementations, in the aforementioned step S101, the pre-created virtual environment is obtained through the following steps: Digital assets are created based on environmental requirements, including buildings, streets, houses, and interior furniture. The digital assets are adjusted according to the layout of objects that conform to the living habits of general users to form a universal 3D virtual environment; or

[0055] The digital assets are adjusted according to the user's personal preferences for object layout to create a personalized 3D virtual environment.

[0056] The teaching method based on physical and social co-simulation in this embodiment utilizes digital town art assets to construct a 3D virtual environment. The simulation environment of the virtual environment can be more realistic, not only covering more scenarios such as art assets of various industrial production scenarios and indoor art assets of various home service scenarios, but also supporting the creation of general and personalized virtual environments, providing better support for the execution of complex teaching tasks and significantly improving the complexity of the simulation environment on which the teaching tasks depend.

[0057] The teaching device based on physical and social co-simulation provided by the present invention will be described below. The teaching device based on physical and social co-simulation described below and the teaching method based on physical and social co-simulation described above can be referred to in correspondence.

[0058] In yet another embodiment, for ease of understanding of the present invention, the following is used... Figure 2 Taking the illustrated hardware environment as an example, various interactive devices such as VR glasses, AR glasses, mobile phones, and personal computers that can be operated by human users are provided in advance. Server A and the above interactive devices support the teaching data generated during interactive tasks. A memory device supports the storage of the teaching data. Server B enables the intelligent agent to learn from the teaching data in the memory. Please refer to... Figure 3 As shown, the data interaction of this invention includes the following five parts: First, users access the system through a multimodal interface to drive the behavior of virtual avatars in the 3D virtual environment.

[0059] Second, the behaviors generated by virtual avatars are extracted through physical behavior extraction and social behavior extraction, thus producing effective teaching data.

[0060] Third, data recording can record effective teaching data, including various behavioral data of the virtual avatar and data on changes in the state of the scene.

[0061] Fourth, the agent learning mechanism is responsible for updating the agent's model parameters based on the taught data. The updated agent can then continue to enter the 3D virtual environment.

[0062] Fifth, in terms of scene construction, the art assets of the 3D virtual environment can consist of digital town art assets, or they can include a wider range of other categories, such as art assets of various industrial production scenes, interior art assets of various home service scenes, etc. There are no restrictions here, and they can be added according to the needs of the teaching task.

[0063] Please refer to the following: Figure 4 The software flow of the teaching method based on physical and social co-simulation of the present invention is described as follows: Step S401: Construct a 3D virtual environment based on digital town art assets.

[0064] The environment can be built manually by professional artists or users.

[0065] (1) Design and create digital assets according to requirements, including buildings, streets, houses, interior furniture, etc.

[0066] (2) Professional artists can manually arrange objects in a layout that conforms to human living habits to form common 3D virtual environments. Alternatively, human users can manually arrange objects in a layout that conforms to their personal habits (such as their own home environment) to form personalized 3D virtual environments.

[0067] Step S402: Connect the human user to the 3D virtual environment, drive the virtual avatar, and start the teaching task.

[0068] Interaction Method 1: Using VR devices, drive a virtual avatar to perform human actions.

[0069] When a user wears a VR device, the device records the user's full-body posture, hand movements, and eye gaze direction, and transmits this information to a virtual avatar. The virtual avatar can then mimic the human user's full-body posture, hand movements, and eye gaze direction in the virtual environment.

[0070] Interaction Method 2: Control the virtual avatar using a computer mouse and keyboard.

[0071] Users can use common interactive devices such as a mouse and keyboard to control a virtual avatar to perform tasks in a virtual environment. The tasks that the virtual avatar can perform depend on pre-set behaviors. For example, moving the mouse can control the virtual avatar's head direction and forward movement, the keyboard arrow keys can control the virtual avatar's forward, backward, left, and right movement, and other keyboard and mouse keys can trigger preset interactive functions.

[0072] Interaction Method 3: Control the virtual avatar through natural language commands.

[0073] The user's natural language commands (or human speech converted into written language) are parsed by a language understanding unit (such as using a large model like GPT4) and decomposed into a sequence of executable atomic tasks that drive the virtual avatar (for example, the user saying "It's raining outside, I need to go out and tidy up the things in the yard" may be decomposed into "stand up", "walk to the door", "pick up the umbrella", "open the door", "walk outside", etc.), thereby achieving the purpose of controlling the virtual avatar to perform tasks.

[0074] Step S403: In the teaching task, extract the physical and social behaviors of human users and complete the data recording.

[0075] Extraction Method 1: Extracting user behavior data based on independent physics and social engines

[0076] (1) Use the built-in physics engine of game development software such as Unreal Engine and Unity3D to provide feedback on the behavior of the virtual avatar and provide effects that conform to the principles of physics. For example, if a cup on the table is knocked to the ground, the cup will break.

[0077] (2) A preset social interaction logic is used to form an engine that responds to social interaction behaviors. The social behaviors performed by users (such as waving to someone, smiling at someone, etc.) will be calculated by the social engine to provide corresponding feedback. For example, if you wave to a virtual person controlled by AI on the street, it will respond with a waving action based on the engine's calculation results.

[0078] In summary, both physical and social behaviors performed by user-driven virtual avatars can trigger corresponding events.

[0079] Extraction Method 2: Extracting User Behavior Data Based on a Unified Physical-Social Joint Engine

[0080] (1) Unified Representation. The physics engine updates the state of the physical simulation space, while the social engine updates the state of the social simulation space. The states of the physical simulation space and the social simulation space are both vectorized. Then, the vectors of the physical simulation space and the social simulation space are concatenated to form a vector with a higher dimension, the size of which is equal to the sum of the dimensions of the physical simulation space vector and the social simulation space vector. This unified space is the physics-social joint simulation space.

[0081] (2) State Update. Vectors in the physics-social co-simulation space are used to represent the state of the space, and the transition between states is accomplished through a state transition matrix. The state transition matrix can realize the transition from one state vector to another. Therefore, any behavior of the virtual avatar can be equivalent to a state transition matrix, which will update the state of the 3D virtual space (containing physical simulation and social simulation).

[0082] Step S404: Based on the recorded data, the agent learns and updates the model parameters.

[0083] The physical interaction data (mainly changes in the object's state) and social behavior data (mainly posture, gaze direction, and user-labeled mental states) of the virtual avatar are recorded and stored in a database for later use. The intelligent agent learns through observational learning and interactive learning, which will be explained separately below.

[0084] Observational learning refers to human users directly demonstrating how to perform various tasks, generating effective teaching data, which is then stored in the background for the intelligent agent to learn from. This learning method does not require real-time interaction from the intelligent agent. The specific learning method is as follows: (1) Setting learning objectives. The agent sets the learning objectives and maps them to changes in the state of the scene. For example, if the learning objective is "cooking", the corresponding scene change is from the unprocessed state of the raw materials of various vegetables in the kitchen to the state of cooked food.

[0085] (2) Extract effective information from the observation data. Observe the data of each time someone performs the "cooking" task, and use mature visual semantic extraction methods (such as visual language large model) or real data from the simulation platform (obtain GroundTruth data from the background) to extract events with significant semantic information in the task execution, such as "putting the food into the pot" and "mixing the ingredients into the container".

[0086] (3) Perform statistical learning for the specified learning task. The most commonly used method is maximum likelihood estimation (MLE), which fits the execution process of the specified task to ensure that the fitted model can effectively represent the existing data. This model is the knowledge learned through observation.

[0087] Interactive learning involves an intelligent agent, pre-trained in a specific version, interacting with human users to perform various tasks and generate task data. The agent can be updated online or offline. The specific learning method is as follows: (1) An intelligent agent model performs a task together with a human user. The intelligent agent needs to have certain task execution capabilities and be able to interact with the user during the task.

[0088] (2) The agent raises questions during the task execution process. The agent's questioning ability requires pre-learning a questioner. After the agent receives visual, linguistic, and other information input, it can analyze the input information, extract effective knowledge, and compare it with its own stored knowledge to determine whether to ask a question. Once the agent determines that a question needs to be asked, it will ask the human user a question based on the knowledge in the input information and wait for the user's answer.

[0089] (3) User feedback, as the answer or feedback to the questions raised by the agent, enters the agent's optimization process and serves as new data to assist the agent in updating the model. Data obtained through interactive learning has a clearer focus, which can help the agent quickly solve problems and improve model performance.

[0090] The teaching method based on physical and social joint simulation in this embodiment has at least the following beneficial technical effects: First, in terms of driving the virtual avatar, it provides interactive methods including VR, keyboard and mouse, and natural language. Among them, natural language driving the virtual avatar breaks through the limitations of traditional virtual-real mapping devices, allowing the virtual avatar to generate more diverse behaviors when executing user commands, reducing the user's control operation burden. Second, it proposes to unify the modeling of the physical simulation space (for scene state and physical behavior) and the social simulation space (for intelligent agent mental state and social behavior), providing physical-social behavior simulation support for teaching tasks and improving the complexity of teachable tasks. Third, it can provide observational learning and interactive learning environments. Complex task simulation environments that support interactive learning are necessary support for embodied intelligent agents to actively learn from human society, overcoming the problems of low task complexity and insufficient interactivity in the original teaching system.

[0091] Please refer to Figure 5 As shown, this embodiment provides a teaching device based on physical and social co-simulation. Specifically, the device includes: a creation module 510, an access module 520, an extraction module 530, and a learning module 540. The following is a detailed description of each module: Create module 510 to create virtual avatars in a pre-created virtual environment; Access module 520 is used to access user behavior into the pre-created virtual environment using a preset interactive device, so as to drive the virtual avatar to perform teaching tasks through user behavior; Extraction module 530 is used to extract the user's physical and social behaviors from the virtual avatar during the virtual avatar's performance of the teaching task, and obtain teaching data. The learning module 540 is used to enable the agent to learn from the teaching data through observational learning and / or interactive learning in order to update the agent's model parameters.

[0092] This embodiment of the teaching device based on physical and social co-simulation first creates a virtual avatar in a pre-created virtual environment, then uses a preset interactive device to connect user behavior to the pre-created virtual environment, so that the virtual avatar can perform teaching tasks through user behavior. Next, the user's physical and social behaviors are extracted from the virtual avatar. Finally, through a specific learning method, the intelligent agent learns from the extracted physical and social behaviors to update the model parameters, realizing the joint simulation of physical and social tasks. The actual operation of the user is connected to the platform to demonstrate various tasks and generate corresponding data for the intelligent agent to learn, which significantly improves the skill and knowledge level of the intelligent agent.

[0093] It should be noted that the modules in the aforementioned teaching device based on physical and social co-simulation can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in the processor of the electronic device in hardware form or independently of it, or stored in the memory of the electronic device in software form, so that the processor can call and execute the corresponding operations of each module.

[0094] Figure 6 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 6 As shown, the electronic device may include: a processor 610, a communications interface 620, a memory 630, and a communication bus 640, wherein the processor 610, the communications interface 620, and the memory 630 communicate with each other through the communication bus 640. The processor 610 can call logical instructions in the memory 630 to execute a teaching method based on physical and social co-simulation. This method includes: creating a virtual avatar in a pre-created virtual environment; using a preset interactive device to access user behavior into the pre-created virtual environment, so as to drive the virtual avatar to perform teaching tasks through user behavior; extracting the user's physical and social behaviors from the virtual avatar during the performance of the teaching tasks, and obtaining teaching data; and enabling the agent to learn from the teaching data through observational learning and / or interactive learning to update the agent's model parameters.

[0095] Furthermore, the logical instructions in the aforementioned memory 630 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0096] On the other hand, the present invention also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the teaching method based on physical and social co-simulation provided by the above methods. The method includes: creating a virtual avatar in a pre-created virtual environment; using a preset interactive device to access user behavior into the pre-created virtual environment, so as to drive the virtual avatar to perform teaching tasks through user behavior; extracting the user's physical and social behavior from the virtual avatar during the performance of the teaching tasks to obtain teaching data; and enabling the agent to learn from the teaching data through observational learning and / or interactive learning to update the model parameters of the agent.

[0097] In another aspect, the present invention also provides a non-transitory computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the teaching method based on physical and social co-simulation provided by the above methods. This method includes: creating a virtual avatar in a pre-created virtual environment; using a preset interactive device to access user behavior into the pre-created virtual environment, so as to drive the virtual avatar to perform a teaching task through user behavior; extracting the user's physical and social behavior from the virtual avatar during the performance of the teaching task, obtaining teaching data; and enabling an agent to learn from the teaching data through observational learning and / or interactive learning to update the agent's model parameters.

[0098] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0099] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0100] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A teaching method based on joint physical and social simulation, characterized in that, The method includes: Create virtual avatars within a pre-created virtual environment; By using a preset interactive device, user behavior is integrated into the pre-created virtual environment, so that the virtual avatar can be driven to perform teaching tasks through user behavior; During the process of the virtual avatar performing the teaching task, the user's physical and social behaviors are extracted from the virtual avatar to obtain teaching data; The agent learns from the teaching data through observational learning and / or interactive learning to update the agent's model parameters.

2. The teaching method based on physical and social co-simulation according to claim 1, characterized in that, Using a preset interactive device to integrate user behavior into the pre-created virtual environment, so as to drive the virtual avatar to perform teaching tasks through user behavior, including: The VR device worn by the user records the user's full-body posture, hand movements, and eye gaze direction, and transmits the recorded data to the virtual avatar so that the virtual avatar can perform the same full-body posture, hand movements, and eye gaze direction as the user in the virtual environment; and / or The virtual avatar performs tasks in the virtual environment through user control of the mouse and keyboard, wherein different mouse and keyboard operations correspond to different preset task behaviors; and / or The user's natural language commands are decomposed into a sequence of executable atomic tasks that drive the virtual avatar using a pre-defined language understanding tool, and the virtual avatar is then driven to perform tasks in the virtual environment using the atomic task sequence.

3. The teaching method based on physical and social co-simulation according to claim 1, characterized in that, The user's physical and social behaviors are extracted from the virtual avatar to obtain teaching data, including: A pre-built physics engine is used to provide feedback on the physical behavior of the virtual avatar, thereby obtaining physical interaction data; A social engine is formed by using a pre-set social interaction logic to provide feedback on the social behavior of virtual avatars and obtain social behavior data.

4. The teaching method based on physical and social co-simulation according to claim 1, characterized in that, The user's physical and social behaviors are extracted from the virtual avatar to obtain teaching data, including: Unified representation: The physics engine updates the state of the physical simulation space, and the social engine updates the state of the social simulation space. The state of the physical simulation space is vectorized, and the state of the social simulation space is also vectorized. The vectors of the physical simulation space and the vectors of the social simulation space are concatenated to form a vector with a higher dimension. The dimension is equal to the sum of the dimensions of the physical simulation space vector and the social simulation space vector, resulting in a unified space as the physical-social joint simulation space. State Update: Vectors in the physical-social co-simulation space are used to represent the state of the space. The transition between states is accomplished through a state transition matrix, which realizes the transition from one state vector to another. Any behavior of the virtual avatar is equivalent to a state transition matrix, which will update the state of the virtual space containing physical and social simulation.

5. The teaching method based on physical and social co-simulation according to claim 1, characterized in that, The observational learning includes: Learning objectives are set by the intelligent agent, which sets the learning objectives and maps them to changes in the state of the scene. Extract effective information from observation data, and use mature visual semantic extraction methods or real data from simulation platforms to extract events with significant semantic information during task execution; Statistical learning is performed on a specified learning task. The maximum likelihood estimation method is used to fit the execution process of the specified task, and the fitted model can effectively represent the existing data.

6. The teaching method based on physical and social co-simulation according to claim 1, characterized in that, The interactive learning includes: An intelligent agent works with a user to perform a task, wherein the intelligent agent has the ability to perform the task and can interact with the user during the task; In response to the task execution process, the intelligent agent raises questions. The intelligent agent's questioning ability requires pre-learning a questioner. After the intelligent agent receives visual and linguistic information input, it analyzes the input information to extract effective knowledge and compares it with its own stored knowledge to determine whether to ask a question. If the intelligent agent determines that a question needs to be asked, it will ask a question to the human user based on the knowledge in the input information and wait for the user's answer. User feedback is used as answers or feedback to the questions raised by the agent and incorporated into the agent's optimization process, serving as new data to assist the agent in updating the model.

7. The teaching method based on physical and social co-simulation according to claim 1, characterized in that, The pre-created virtual environment is obtained through the following steps: Digital assets are created based on environmental requirements, including buildings, streets, houses, and interior furniture. The digital assets are adjusted according to the layout of objects that conform to the living habits of general users to form a universal 3D virtual environment; or The digital assets are adjusted according to the user's personal preferences for object layout to create a personalized 3D virtual environment.

8. A teaching device based on physical and social co-simulation, characterized in that, The device includes: A creation module is used to create virtual avatars within a pre-created virtual environment; The access module is used to access user behavior into the pre-created virtual environment using a preset interactive device, so as to drive the virtual avatar to perform teaching tasks through user behavior; The extraction module is used to extract the user's physical and social behaviors from the virtual avatar during the virtual avatar's performance of teaching tasks, thereby obtaining teaching data. The learning module is used to enable the agent to learn from the teaching data through observational learning and / or interactive learning in order to update the agent's model parameters.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the teaching method based on physical and social co-simulation as described in any one of claims 1 to 7.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the teaching method based on physical and social co-simulation as described in any one of claims 1 to 7.

Images