Teaching method and device based on intelligent agent system with body

Abstract

The invention discloses a teaching method and device based on an intelligent agent system, and the method comprises the steps: receiving a to-be-learned knowledge theme, expanding to-be-demonstrated knowledge content according to the to-be-learned knowledge theme, generating a demonstration mode corresponding to the to-be-demonstrated knowledge content, and demonstrating the to-be-demonstrated knowledge content through a knowledge demonstration script, the demonstration mode comprises a knowledge demonstration script generated based on the to-be-demonstrated knowledge content and the type and the number of bodies of the intelligent agent system. The method solves the problems that in the prior art, teaching content is high in presetting performance, flexibility is lacked, and knowledge innovation is not supported; the robot depends on a single type of robot, and the body language is mainly used for social contact rather than knowledge transfer; in order to solve the problems of low flexibility caused by high matching degree of a demonstration method and contents and incompatibility with robot cluster multi-modal fusion demonstration, the teaching method provided by the invention can change along with interaction between an agent with a body and a user, and the teaching diversity is increased.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to a teaching method, device, electronic device, non-transient computer-readable storage medium and computer program product based on an embodied intelligent body system. Background Art

[0002] In the prior art, research and exploration of using artificial intelligence technology to carry out education or assist education can be carried out. It can carry out knowledge education, such as teaching math problems, foreign language pronunciation and word spelling, and can also carry out action knowledge education, such as guiding the fitness movements of the elderly, sign language teaching, etc. At present, common teaching carriers are voice, tablet computers, and desktop computers with screens. Most of these works only involve knowledge teaching in a specific field (for example, only teaching math knowledge, or only teaching word knowledge). In order to ensure the accuracy and stability of teaching, the knowledge content is mostly given in advance. In recent years, there has been a gradual emergence of research on the application of social robots in the field of education. Most of this kind of work uses a single type of robot, and most of them are finished robots (for example, Nao robots are used very frequently). Robots can make some movements, but most of these movements are used for social effects, and do not bear the function of transferring knowledge.

[0003] The current technology has defects, mainly manifested in the fact that the teaching content is too preset, lacks flexibility, and does not support knowledge innovation; it relies on a single type of robot and body language is mainly used for social interaction rather than knowledge transfer; the demonstration method is highly matched with the content, resulting in low flexibility, and it is incompatible with multimodal fusion demonstration of robot clusters. Summary of the invention

[0004] In view of this, the present invention provides a teaching method, device, electronic device, non-transitory computer-readable storage medium and computer program product based on an embodied intelligent body system, so as to at least solve the problems in the prior art that the teaching content is too preset, lacks flexibility, and does not support knowledge innovation; relies on a single type of robot and body language is mainly used for social interaction rather than knowledge transfer; the demonstration method has a high degree of match with the content, resulting in low flexibility, and is incompatible with multi-modal fusion demonstration of robot clusters.

[0005] According to one aspect of the present invention, a teaching method based on an embodied intelligent agent system is provided, the method comprising: Receive the knowledge topic to be learned; Expand the knowledge content to be demonstrated according to the knowledge subject to be learned; Generate a demonstration method corresponding to the knowledge content to be demonstrated, and demonstrate the knowledge content to be demonstrated through a knowledge demonstration script; wherein the demonstration method includes a knowledge demonstration script generated based on the knowledge content to be demonstrated and the type and number of the intelligent system's own embodiments.

[0006] Optionally, expanding the knowledge content to be demonstrated according to the knowledge subject to be learned includes: Expanding the knowledge subject to be learned to form expanded knowledge content; Receive adjustment instructions; Locating the knowledge content referred to by the adjustment instruction through an intelligent agent positioning module; wherein the intelligent agent positioning module is implemented by at least one of the following methods: a single-head self-attention Transformer network, a keyword matching algorithm, and a bag-of-words model search algorithm; Using the adjustment instruction to adjust the indicated knowledge content until the adjusted knowledge content meets the predetermined requirement; wherein the adjusted knowledge content includes the complete knowledge content consisting of the adjusted content part and the content part that does not need to be adjusted; The adjusted knowledge content is input into the intelligent body memory module, and the adjusted knowledge content is selected and compressed and stored in the knowledge content library; wherein the intelligent body memory module is implemented by at least one of the following methods: language model, pre-trained transformer model, pre-trained BERT model.

[0007] Optionally, the embodied intelligent body system includes an intervention interface; wherein, the intervention interface is used for the user to directly access the adjusted knowledge content, the knowledge content library, input content improvement instructions, and insert knowledge content.

[0008] Optionally, generating a demonstration method corresponding to the knowledge content to be demonstrated, and demonstrating the knowledge content to be demonstrated through the knowledge demonstration script includes: Extracting roles from the knowledge content to be demonstrated and assigning a name to each role; Traversing the embodied bodies of the embodied agent system, and assigning a body to each role based on the type and number of the embodied bodies and the characteristics of the roles in the knowledge content to be demonstrated; Generate a portrait for each embodied character based on their characteristics; Combining the name of the physical character and the portrait of the physical character, generating a demonstration script line by line according to the knowledge content to be demonstrated; wherein each line of the script contains a character and an action, and contains at least one line of narration; Parsing the behavior of embodied characters into sequences of actions of embodied characters; Demonstrate action sequences with physical characters and play corresponding narration.

[0009] Optionally, combining the name of the embodied role and the portrait of the embodied role, generating a demonstration script line by line according to the knowledge content to be demonstrated includes: Decomposing the knowledge content to be demonstrated into semantic-level scripts and outputting semantic-level instructions; Determine whether the semantic-level instruction conforms to a preset expression format through a regular expression, and obtain a determination result; When the judgment result is negative, the instruction at the output semantic level is corrected according to a preset error correction table.

[0010] Optionally, in the case where the multiple embodiments are robot clusters, each line of the script indicates that a designated robot moves from a first coordinate position to a second coordinate position, and a narration is played; Based on the second coordinate position, combined with the first coordinate position when the cluster robot is initialized, the coordinate position is transformed into the actual coordinate system of the physical world, and the actual coordinate position and the line audio are sent to each cluster robot through the local area network; The built-in control board of the cluster robot controls its built-in motor to move the robot to the actual coordinate position, and plays the line audio through the built-in speaker.

[0011] Optionally, the embodiment includes a physical embodiment and a virtual embodiment; when it is determined that the role is not suitable for interpretation by a physical embodiment, the role is assigned to a virtual embodiment for interpretation; and the virtual embodiment is selected from a preset three-dimensional model library.

[0012] In another aspect, the present invention further provides a teaching device based on an embodied intelligent system, the device comprising: A receiving module, used for receiving the knowledge subject to be learned; An expansion module, used to expand the knowledge content to be demonstrated according to the knowledge subject to be learned; A demonstration module is used to generate a demonstration method corresponding to the knowledge content to be demonstrated, and demonstrate the knowledge content to be demonstrated through a knowledge demonstration script; wherein the demonstration method includes a knowledge demonstration script generated based on the knowledge content to be demonstrated and the type and number of the intelligent system's own embodiments.

[0013] Optionally, the expansion module includes: An expansion unit, used for expanding the knowledge subject to be learned to form expanded knowledge content; A receiving unit, used for receiving an adjustment instruction; A positioning unit, used to locate the knowledge content referred to by the adjustment instruction through an intelligent agent positioning module; wherein the intelligent agent positioning module is implemented by at least one of the following methods: a single-head self-attention Transformer network, a keyword matching algorithm, and a bag-of-words model search algorithm; An adjustment unit, used to adjust the indicated knowledge content using the adjustment instruction until the adjusted knowledge content meets the predetermined requirement; wherein the adjusted knowledge content includes the complete knowledge content consisting of the adjusted content part and the content part that does not need to be adjusted; A memory unit is used to input the adjusted knowledge content into an intelligent body memory module, and select and compress the adjusted knowledge content and store it in a knowledge content library; wherein the intelligent body memory module is implemented in at least one of the following ways: a language model, a pre-trained transformer model, and a pre-trained BERT model.

[0014] Optionally, the demonstration module includes: An extraction unit, used for extracting roles from the knowledge content to be demonstrated and assigning a name to each role; A traversal unit, used to traverse the embodied bodies of the embodied agent system, and assign a body to each role in combination with the type and number of the embodied bodies and the characteristics of the roles in the knowledge content to be demonstrated; A first generating unit, for generating a portrait for each embodied character according to the characteristics of the character; A second generating unit is used to generate a demonstration script line by line according to the knowledge content to be demonstrated, combining the name of the embodied character and the portrait of the embodied character; wherein each line of the script contains a character and an action, and contains at least one line of narration; A parsing unit, for parsing the behavior of the embodied character into an action sequence of the embodied character; Demonstration unit, used to demonstrate the action sequences of physical characters and play corresponding narration.

[0015] Another aspect of the present invention further provides an electronic device, comprising: Processor; and Memory for storing programs, Wherein, the program includes instructions, which, when executed by the processor, enable the processor to execute the teaching method based on the embodied intelligent body system according to any one of the above embodiments.

[0016] Another aspect of the present invention further provides a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to enable a computer to execute a teaching method based on an embodied intelligent body system according to any one of the above embodiments.

[0017] Another aspect of the present invention provides a computer program product, characterized in that the computer program product includes instructions, which, when executed, enable a computer to execute the teaching method based on the embodied intelligent body system according to any one of the above embodiments.

[0018] Through this optional embodiment, the teaching method based on the embodied intelligent system receives the knowledge subject to be learned, expands the knowledge content to be demonstrated according to the knowledge subject to be learned, generates a demonstration method corresponding to the knowledge content to be demonstrated, and demonstrates the knowledge content to be demonstrated through a knowledge demonstration script, wherein the demonstration method includes a knowledge demonstration script generated based on the knowledge content to be demonstrated and the type and number of the embodied bodies of the intelligent system itself. At least the problems in the prior art that the teaching content is too preset, lacks flexibility, and does not support knowledge innovation; relies on a single type of robot and body language is mainly used for social interaction rather than knowledge transfer; the demonstration method has a high degree of match with the content, resulting in low flexibility, and is incompatible with multimodal fusion demonstration of robot clusters are solved. The knowledge content taught by this optional embodiment can change with the interaction between the embodied intelligent body and the user, which increases the diversity of teaching.

[0019] It should be understood that the above general description and the following detailed description are merely exemplary and illustrative and are not intended to limit the scope of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The accompanying drawings below are part of the specification of the present invention, and illustrate exemplary embodiments of the present invention. Together with the description of the specification, the accompanying drawings are used to explain the principles of the present invention.

[0021] Figure 1 is a flow chart of a teaching method based on an embodied intelligent agent system according to an embodiment of the present invention.

[0022] Figure 2 is another flow chart of the teaching method based on the embodied intelligent agent system according to an embodiment of the present invention.

[0023] Figure 3 is another flow chart of the teaching method based on the embodied intelligent agent system according to an embodiment of the present invention.

[0024] Figure 4 It is a structural block diagram of a teaching device based on an embodied intelligent body system according to an embodiment of the present invention.

[0025] Figure 5 is a structural block diagram of an expansion module according to an embodiment of the present invention.

[0026] Figure 6 is a structural block diagram of a demonstration module according to an embodiment of the present invention.

[0027] Figure 7 A structural block diagram of an exemplary electronic device that can be used to implement the embodiments of the present disclosure is shown. DETAILED DESCRIPTION

[0028] In order to make the objectives, technical solutions and advantages of the embodiments of the present invention more clearly understood, the spirit of the contents disclosed by the present invention will be clearly explained with the accompanying drawings and detailed descriptions below. After understanding the embodiments of the contents of the present invention, any technician in the relevant technical field can change and modify the techniques taught by the contents of the present invention without departing from the spirit and scope of the contents of the present invention.

[0029] The exemplary embodiments and descriptions of the present invention are used to explain the present invention, but are not intended to limit the present invention. In addition, elements / components with the same or similar reference numerals used in the drawings and embodiments are used to represent the same or similar parts.

[0030] The terms “first”, “second”, etc. used in this document do not particularly refer to an order or sequence, nor are they used to limit the present invention. They are only used to distinguish elements or operations described with the same technical terms.

[0031] The directional terms used in this document, such as up, down, left, right, front or back, etc., are only used to refer to the directions of the drawings. Therefore, the directional terms used are used to illustrate and not to limit the present invention.

[0032] The words “include,” “including,” “have,” “contain,” etc. used in this document are open-ended terms, meaning including but not limited to.

[0033] As used herein, "and / or" includes any or all combinations of the items described.

[0034] As used herein, “plurality” includes “two” and “more than two”; as used herein, “plurality groups” includes “two groups” and “more than two groups”.

[0035] The terms "substantially" and "approximately" used herein are used to modify any quantity or error that may vary slightly, but these slight changes or errors do not change their essence. Generally speaking, the range of slight changes or errors modified by such terms may be 20% in some embodiments, 10% in some embodiments, 5% in some embodiments, or other values. Those skilled in the art should understand that the aforementioned values ​​can be adjusted according to actual needs and are not limited thereto.

[0036] All terms (including technical and scientific terms) used herein have the meanings commonly understood by those skilled in the art unless otherwise defined. It should be noted that the terms used herein should be interpreted as having a meaning consistent with the context of this specification and should not be interpreted in an idealized or overly rigid manner.

[0037] In the case of using expressions such as "at least one of A, B, and C, etc.", it should generally be interpreted in accordance with the meaning of the expression generally understood by those skilled in the art (for example, "a system having at least one of A, B, and C" should include but not be limited to a system having A alone, B alone, C alone, A and B, A and C, B and C, and / or a system having A, B, C, etc.). In the case of using expressions such as "at least one of A, B, or C, etc.", it should generally be interpreted in accordance with the meaning of the expression generally understood by those skilled in the art (for example, "a system having at least one of A, B, or C" should include but not be limited to a system having A alone, B alone, C alone, A and B, A and C, B and C, and / or a system having A, B, C, etc.). Those skilled in the art should also understand that any transitional conjunctions and / or phrases that substantially represent two or more optional items, whether in the specification, claims, or drawings, should be understood to give the possibility of including one of these items, either of these items, or both of these items. For example, the phrase "A or B" should be understood to include the possibilities of "A" or "B", or "A and B".

[0038] In this embodiment, a teaching method based on an embodied intelligent agent system is provided. Figure 1 is a flow chart of a teaching method based on an embodied intelligent agent system according to an embodiment of the present invention. Figure 1 As shown, the process includes the following steps: Step S102: receiving the knowledge topic to be learned. In an optional embodiment, the embodied intelligent system enters the "knowledge preparation and iteration" mode, and the educated human user specifies a knowledge topic to be learned through interaction with the embodied intelligent body, and the interaction method includes but is not limited to text, voice, etc. The knowledge topic to be learned can not only support the education of factual knowledge, such as historical knowledge, but also support the education of social knowledge, such as how to deal with social problems.

[0039] Step S104: Expand the knowledge content to be demonstrated according to the knowledge subject to be learned. In an optional embodiment, by analyzing the knowledge subject to be learned, relevant content is retrieved from the pre-trained model or knowledge base, and then the knowledge points to be demonstrated are screened and sorted out according to user needs, and displayed through an appropriate interface or platform. Further, in an optional embodiment, in the process of perfecting knowledge, human educators can intervene and influence the formation of the knowledge to be demonstrated through prompts or direct modification of knowledge content, so as to enhance the stability and reliability of the knowledge formation process. For example, for example, a human user wants to learn the relevant knowledge of a certain war in World War II. After the user proposes this topic, the intelligent system outputs the knowledge related to the war. After reading, the user can ask the intelligent system to further provide the background of a general in the war, or ask geographical questions related to the place where the war took place, or ask the intelligent system to correct the unreasonable parts of the output content. Alternatively, a human user may want to learn how to ask a supervisor questions encountered in scientific research. After the user raises this topic, the intelligent system outputs a possible conversation between the user and the supervisor based on the background information provided by the user. The user can continue the conversation based on his or her own specific situation, or ask the intelligent system to correct certain content in the output.

[0040] Step S106: Generate a demonstration method corresponding to the knowledge content to be demonstrated, and demonstrate the knowledge content to be demonstrated through a knowledge demonstration script; wherein the demonstration method includes a knowledge demonstration script generated based on the knowledge content to be demonstrated and the type and number of the intelligent system's own embodied bodies. Specifically, after the knowledge content to be demonstrated, the embodied intelligent system switches to the "knowledge demonstration and iteration" mode. In this mode, the embodied intelligent agent parses the knowledge content to be demonstrated into a form similar to a stage play, and demonstrates it using its own embodied body. Its own embodied body categories include, but are not limited to: robotic arms, humanoid robots, and desktop-level movable small-sized car robots. There is no limit on the number of each type of embodied body, and it can be increased or decreased according to actual conditions.

[0041] Through the above steps, the complete link from knowledge generation, knowledge demonstration to knowledge review is opened up, which can support fully automated actions. This method does not require pre-given teaching content, nor does it require human developers to design each teaching content separately. It only needs to input a topic to autonomously complete the complete link from knowledge expansion, demonstration teaching to interactive consolidation. This method enables the intelligent agent to interact with the learners. Based on the given knowledge or the knowledge gradually formed in the interactive process, the intelligent agent uses its own body to present the knowledge to the learners in the form of a drama through a multimodal hybrid method, and iteratively presents details in the process of interaction with the learners to help the learners consolidate the knowledge they have learned.

[0042] Figure 2is another flow chart of the teaching method based on the embodied intelligent system according to an embodiment of the present invention. The above step S104 involves expanding the knowledge content to be demonstrated according to the knowledge subject to be learned. In an optional embodiment, Figure 2 As shown, the process includes the following steps: Step S202: Expand the knowledge subject to be learned to form expanded knowledge content. Specifically, by extracting information from multiple data sources, structuring it and integrating it into the knowledge base, and then using reasoning and pattern recognition technology to generate new knowledge, the content of the specific knowledge subject is expanded. At the same time, through continuous learning and user feedback, the knowledge base is continuously optimized and updated. Based on the expanded content, human users can further ask questions, add a piece of knowledge content, or ask the intelligent system to modify the existing knowledge content.

[0043] Step S204: receiving adjustment instructions. In an optional embodiment, the feedback from the human user is converted into text data through a speech recognition algorithm and input into the intelligent agent system.

[0044] Step S206: Locate the knowledge content referred to by the intelligent agent positioning module according to the adjustment instruction; wherein the intelligent agent positioning module is implemented by at least one of the following methods: a single-head self-attention Transformer network, a keyword matching algorithm, and a bag-of-words model search algorithm.

[0045] Step S208: using the adjustment instruction to adjust the knowledge content, until the adjusted knowledge content meets the predetermined requirements; wherein the adjusted knowledge content includes the complete knowledge content consisting of the adjusted content part and the content part that does not need to be adjusted.

[0046] Step S210: input the adjusted knowledge content into the intelligent body memory module, select and compress the adjusted knowledge content and store it into the knowledge content library; wherein the intelligent body memory module is implemented by at least one of the following methods: language model, pre-trained transformer model, pre-trained BERT model.

[0047] In this optional embodiment, the agent system switches to the "knowledge preparation and iteration" mode, in which the agent system is set to give priority to the integrity and rationality of the knowledge content. This process not only enables the agent system to automatically generate knowledge, but also enhances the stability of the knowledge it generates, alleviates the problem of the existing system requiring a large amount of manual design of knowledge content, and improves the flexibility of the system.

[0048] In order to further ensure the security of the knowledge content and enhance the reliability of the entire system, in an optional embodiment, the embodied intelligent body system includes an intervention interface; wherein, the intervention interface is used for users to directly access the adjusted knowledge content, knowledge content library, input content improvement instructions, and insert knowledge content. In this optional embodiment, human educators can intervene in this process at any time, including directly accessing the knowledge content cache, knowledge content library, giving improvement suggestions / inserting knowledge content on behalf of human users, etc., to ensure the security of the knowledge content and enhance the reliability of the entire system. It should be noted that the system described in this optional embodiment only provides an intervention interface, but the intervention of human educators is not a necessary part. Only the embodied intelligent body system and the educated human users can complete the knowledge formation. The following situations involving human educators are similar to this, and are not necessary components of the system.

[0049] The above step S106 involves generating a demonstration method corresponding to the knowledge content to be demonstrated, and demonstrating the knowledge content to be demonstrated through a knowledge demonstration script. Specifically, in an optional embodiment, Figure 3 As shown, the following steps are included: Step S302: extract roles from the knowledge content to be demonstrated and assign names to each role. For example, if the knowledge content to be demonstrated is related to a war, the extracted roles are the countries / forces involved in the war; if the knowledge to be demonstrated is related to social issues, the extracted roles are the people involved, such as users and mentors. This part can be implemented through language models.

[0050] Step S304: traverse the embodied bodies of the embodied intelligent system, and assign an embodied body to each role in combination with the type and number of embodied bodies and the characteristics of the role in the knowledge content to be demonstrated. This part can be implemented by a language model. In an optional embodiment, the embodied body includes a physical embodied body and a virtual embodied body. When it is determined that the role is not suitable for interpretation by a physical embodied body, the role is assigned to a virtual embodied body for interpretation, and the virtual embodied body is selected from a pre-set three-dimensional model library.

[0051] Step S306: Generate a portrait for each concrete character according to the character's characteristics. Specifically, this step is implemented using a generation model such as DALLE3, stable diffusion, etc., and a unified prompt is used to keep the character style consistent in the same story.

[0052] Step S308: combining the name of the embodied character and the portrait of the embodied character, generating a demonstration script line by line according to the knowledge content to be demonstrated; wherein each line of the script contains a character and an action, and contains at least one line of narration.

[0053] Step S310: Parse the behavior of the embodied character into a sequence of actions of the embodied character. Specifically, the agent system includes a variety of different types of planners, each type of embodied body corresponds to a planner, which can parse the demonstration script generated by the agent in step S308 into control instructions for the embodied body. This step can be achieved through a simple planning algorithm (such as A*, etc.), or through reinforcement learning training to achieve the parsing from the character behavior to the embodied action sequence. For example, the character behavior is to move from xx to xx, or to grab xx object. The former can use the A* algorithm to plan the travel path, and the latter can output joint actions through reinforcement learning.

[0054] Step S312: Demonstrate the action sequence of the physical character and play the corresponding narration. For example, playing the narration can be achieved through TTS (text to speech) technology.

[0055] Step S314: During the process of the embodied intelligent system demonstrating knowledge, the embodied intelligent system switches to the interactive learning mode. The educated human user can interrupt the demonstration of the intelligent system at any time and ask the intelligent agent to modify the demonstration method or knowledge content in the form of natural language (text, voice, etc.). Human educators can intervene to influence the formation of knowledge demonstration by prompts or directly modifying the demonstration content to enhance the stability and reliability of the demonstration process. After determining the knowledge demonstration, the educated human user can watch the knowledge demonstration in its entirety. The user can ask questions to the intelligent system or ask the intelligent system to repeat a certain content. Human educators can intervene to influence the process of knowledge demonstration by prompts or directly modifying the starting position of the demonstration to enhance the stability and reliability of the demonstration process.

[0056] Furthermore, user feedback is collected, and some steps in the above S302-S314 are re-executed based on the user feedback.

[0057] Through this optional embodiment, compared to the existing technology that only supports a single type of robot, this system integrates various types of physical bodies (robot clusters, mechanical arms, robots, etc.), and virtual bodies (mainly 3D models). Through system design, they can work together to complete the knowledge demonstration. The demonstration of knowledge is a combination of physical scenes and virtual scenes. The embodied intelligent body system simultaneously dispatches its physical body and virtual body to complete the demonstration of knowledge. Among them, the virtual scene is often used as the background display of the demonstration, in which the scenery in the knowledge is mainly rendered, but the characters can also be rendered. The physical scene is often used as the main stage for the demonstration, and the physical body is mainly used as a role, but it can also assume the function of scenery. Similar to the first step, human educators can intervene in this process at any time to ensure the security of the knowledge content.

[0058] The embodied intelligent system switches to the "knowledge demonstration and iteration" mode, and comprehensively considers the knowledge content to be demonstrated, and the type and number of its own embodied bodies, to generate a set of scripts to demonstrate the knowledge. Compared with the existing technology, this optional embodiment can demonstrate knowledge in the form of dramatic performance, making the teaching process more interesting, and in addition to supporting knowledge teaching, it is also more suitable for social skills teaching.

[0059] The above-mentioned step S308 involves combining the name of the embodied character and the portrait of the embodied character, and generating a demonstration script line by line according to the knowledge content to be demonstrated. In an optional embodiment, the knowledge content to be demonstrated is decomposed into semantic-level scripts, and semantic-level instructions are output. Regular expressions are used to determine whether the semantic-level instructions conform to a preset expression format to obtain a judgment result. If the judgment result is no, the output semantic-level instructions are corrected according to a preset error correction table. If it is still unqualified after correction or the error is not in the error correction table, the instructions are regenerated.

[0060] In an optional embodiment, when multiple bodies are robot clusters, each line of the script indicates that a specified robot moves from a first coordinate position to a second coordinate position, and the narration played is transformed into the actual coordinate system of the physical world based on the second coordinate position, combined with the first coordinate position when the cluster robot is initialized, and the actual coordinate position and the line audio are sent to each cluster robot through the local area network. The built-in control board of the cluster robot controls its built-in motor to realize the robot moving to the actual coordinate position, and plays the line audio through the built-in speaker. Specifically, taking the robot cluster as an example, after each robot in the cluster is assigned a role, the role portrait is uploaded to the local storage of each robot through the local area network and displayed on the robot's display. In step S308, the intelligent system generates the script line by line, in which each line indicates which robot should move from where to where and read what lines. In step S310, the planner applicable to the cluster robot transforms the target coordinates in step S308 into the actual coordinate system of the physical world in combination with the posture when the cluster robot is initialized, and sends the actual coordinates and line audio data to each cluster robot through the local area network. In step S312, the built-in control panel of the cluster robot controls its built-in motor to move the robot to the specified position, and plays the audio of the lines through the built-in speaker. The same applies to robotic arms and humanoid robots. As for the virtual body, the role that the intelligent agent determines is not suitable for the physical body to perform will be assigned to the virtual body. The role portrait is presented on the surface of the virtual body in the form of a material map, and the model of the virtual body is selected from a pre-given basic 3D model library (a 2D to 3D generative model can also be used here). The intelligent agent system sends the control signal to the simulation environment through the local area network. After the planner in the simulation environment receives the control signal, it transforms the behavior of the virtual body into the coordinate system of the virtual world and controls the virtual body to perform corresponding behaviors (such as moving, playing action animations, etc.).

[0061] After the user determines the script for demonstrating knowledge, the embodied intelligent system switches to the "interactive learning consolidation" mode. The knowledge content to be demonstrated and the determined knowledge demonstration script are both the information library of this mode, and the embodied intelligent system can retrieve this information at any time. The user asks the embodied intelligent system questions through text, voice or gestures. The embodied intelligent system decides based on the user's requirements: (1) to re-demonstrate a certain content; (2) to directly answer the user's question. Human educators can intervene in this process at any time to ensure the security of the knowledge content.

[0062] In the present embodiment, a teaching device based on an embodied intelligent system is also provided, which is used to implement the above-mentioned embodiments and preferred implementation modes, and the descriptions that have been made will not be repeated. As used below, the term "module" can implement a combination of software and / or hardware of a predetermined function. Although the device described in the following embodiments is preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceived.

[0063] In summary, in order to address the shortcomings of existing technologies, the present invention proposes a new system design method, which makes: 1) There is no need to pre-set teaching content, nor is there a need for human developers to design each teaching content separately. Only a topic needs to be input to autonomously complete the complete link from knowledge expansion, demonstration teaching to interactive consolidation. 2) The system supports multiple types of embodied bodies. The intelligent body uses its own virtual embodied body or physical embodied body (which can be single or multiple) to show the knowledge to be learned to the educated in the form of a dramatic performance. The embodied body is directly responsible for the knowledge demonstration, and has the functions of knowledge transfer and social interaction, rather than just social functions. 3) The way of demonstrating knowledge can be adjusted at any time, and the adjustment cost is low, which can quickly support the demonstration of new knowledge without the need for redevelopment or redesign. 4) It supports multi-modal fusion demonstrations such as robot clusters, robotic arms, videos, and audio.

[0064] This optional embodiment provides a teaching device based on an embodied intelligent system, such as Figure 4 As shown, the device comprises: A receiving module 42, for receiving a knowledge subject to be learned; An expansion module 44, used to expand the knowledge content to be demonstrated according to the knowledge subject to be learned; The demonstration module 46 is used to generate a demonstration method corresponding to the knowledge content to be demonstrated, and demonstrate the knowledge content to be demonstrated through the knowledge demonstration script; wherein the demonstration method includes a knowledge demonstration script generated based on the knowledge content to be demonstrated and the type and number of the intelligent system's own embodiment.

[0065] Another optional embodiment also provides a structural block diagram of the expansion module 44, such as Figure 5 As shown, the device comprises: An expansion unit 52, used for expanding the knowledge subject to be learned to form expanded knowledge content; A receiving unit 54, configured to receive an adjustment instruction; A positioning unit 56, configured to locate the knowledge content referred to by the adjustment instruction through an agent positioning module; wherein the agent positioning module is implemented by at least one of the following methods: a single-head self-attention Transformer network, a keyword matching algorithm, and a bag-of-words model search algorithm; An adjustment unit 58, configured to adjust the indicated knowledge content using the adjustment instruction until the adjusted knowledge content meets the predetermined requirement; wherein the adjusted knowledge content includes the complete knowledge content consisting of the adjusted content part and the content part that does not need to be adjusted; The memory unit 60 is used to input the adjusted knowledge content into the intelligent body memory module, and select and compress the adjusted knowledge content and store it into the knowledge content library; wherein the intelligent body memory module is implemented by at least one of the following methods: language model, pre-trained transformer model, pre-trained BERT model.

[0066] like Figure 6 As shown, the demonstration module 46 includes: An extraction unit 462, used to extract roles from the knowledge content to be demonstrated and assign a name to each role; A traversal unit 464 is used to traverse the embodied bodies of the embodied agent system, and assign a body to each role in combination with the type and number of the embodied bodies and the characteristics of the roles in the knowledge content to be demonstrated; A first generating unit 466, for generating a portrait for each embodied character according to the character's characteristics; The second generating unit 468 is used to generate a demonstration script line by line according to the knowledge content to be demonstrated, combining the name of the embodied character and the portrait of the embodied character; wherein each line of the script contains a character and an action, and contains at least one line of narration; A parsing unit 470, for parsing the behavior of the embodied character into a sequence of actions of the embodied character; The demonstration unit 472 is used to demonstrate the action sequence of the physical character and play the corresponding narration.

[0067] The further functional description of each of the above modules is the same as that of the above corresponding embodiments and will not be repeated here.

[0068] The exemplary embodiment of the present invention further provides an electronic device, comprising: at least one processor; and a memory connected to the at least one processor in communication. The memory stores a computer program that can be executed by the at least one processor, and the computer program is used to enable the electronic device to perform a method according to an embodiment of the present invention when executed by the at least one processor.

[0069] Exemplary embodiments of the present invention also provide a non-transitory computer-readable storage medium storing a computer program, wherein the computer program, when executed by a processor of a computer, is used to cause the computer to perform a method according to an embodiment of the present invention.

[0070] An exemplary embodiment of the present invention further provides a computer program product, comprising a computer program, wherein when the computer program is executed by a processor of a computer, the computer is used to enable the computer to perform a method according to an embodiment of the present invention.

[0071] refer to Figure 7 , a block diagram of an electronic device that can be used as a server or client of the present invention will now be described, which is an example of a hardware device that can be applied to various aspects of the present invention. The electronic device is intended to represent various forms of digital electronic computer devices, such as laptop computers, desktop computers, workbenches, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or required herein.

[0072] like Figure 7 As shown, the electronic device 700 includes a computing unit 701, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 702 or a computer program loaded from a storage unit 708 into a random access memory (RAM) 703. In the RAM 703, various programs and data required for the operation of the device 700 can also be stored. The computing unit 701, the ROM 702, and the RAM 703 are connected to each other via a bus 704. An input / output (I / O) interface 705 is also connected to the bus 704.

[0073] A plurality of components in the electronic device 700 are connected to the I / O interface 705, including: an input unit 706, an output unit 707, a storage unit 708, and a communication unit 709. The input unit 706 may be any type of device capable of inputting information to the electronic device 700, and the input unit 706 may receive input digital or character information, and generate key signal inputs related to user settings and / or function control of the electronic device. The output unit 707 may be any type of device capable of presenting information, and may include but is not limited to a display, a speaker, a video / audio output terminal, a vibrator, and / or a printer. The storage unit 708 may include but is not limited to a disk, an optical disk. The communication unit 709 allows the electronic device 700 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks, and may include but is not limited to a modem, a network card, an infrared communication device, a wireless communication transceiver, and / or a chipset, such as a Bluetooth™ device, a WiFi device, a WiMax device, a cellular communication device, and / or the like.

[0074] The computing unit 701 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 701 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 701 performs the various methods and processes described above. For example, in some embodiments, the music data processing method may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit 708. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 700 via the ROM 702 and / or the communication unit 709. In some embodiments, the computing unit 701 may be configured to perform a method according to an embodiment of the present invention in any other appropriate manner (e.g., by means of firmware).

[0075] The program code for implementing the method of the present invention can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general-purpose computer, a special-purpose computer or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code can be executed entirely on the machine, partially on the machine, partially on the machine as a stand-alone software package and partially on a remote machine, or entirely on a remote machine or server.

[0076] In the context of the present invention, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0077] As used herein, the terms "machine-readable medium" and "computer-readable medium" refer to any computer program product, apparatus, and / or device (e.g., disk, optical disk, memory, programmable logic device (PLD)) for providing machine instructions and / or data to a programmable processor, including a machine-readable medium that receives machine instructions as a machine-readable signal. The term "machine-readable signal" refers to any signal for providing machine instructions and / or data to a programmable processor.

[0078] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0079] The systems and techniques described herein may be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.

[0080] A computer system may include clients and servers. Clients and servers are generally remote from each other and usually interact through a communication network. The relationship of client and server is generated by computer programs running on respective computers and having a client-server relationship to each other.

[0081] Obviously, the above embodiments are merely examples for the purpose of clear explanation, and are not intended to limit the implementation methods. For those skilled in the art, other different forms of changes or modifications can be made based on the above description. It is not necessary and impossible to list all the implementation methods here. The obvious changes or modifications derived therefrom are still within the scope of protection of the invention.

Claims

1. A teaching method based on an embodied intelligent agent system, characterized in that: The method comprises: Receive the knowledge topic to be learned; Expand the knowledge content to be demonstrated according to the knowledge subject to be learned; Generate a demonstration method corresponding to the knowledge content to be demonstrated, and demonstrate the knowledge content to be demonstrated through a knowledge demonstration script; wherein the demonstration method includes a knowledge demonstration script generated based on the knowledge content to be demonstrated and the type and number of the intelligent system's own embodiments.

2. The teaching method based on embodied intelligent agent system according to claim 1, characterized in that: The knowledge content to be demonstrated based on the knowledge subject to be learned includes: Expanding the knowledge subject to be learned to form expanded knowledge content; Receive adjustment instructions; Locating the knowledge content referred to by the adjustment instruction through an intelligent agent positioning module; wherein the intelligent agent positioning module is implemented by at least one of the following methods: a single-head self-attention Transformer network, a keyword matching algorithm, and a bag-of-words model search algorithm; Using the adjustment instruction to adjust the indicated knowledge content until the adjusted knowledge content meets the predetermined requirement; wherein the adjusted knowledge content includes the complete knowledge content consisting of the adjusted content part and the content part that does not need to be adjusted; The adjusted knowledge content is input into the intelligent body memory module, and the adjusted knowledge content is selected and compressed and stored in the knowledge content library; wherein the intelligent body memory module is implemented by at least one of the following methods: language model, pre-trained transformer model, pre-trained BERT model.

3. The teaching method based on embodied intelligent agent system according to claim 2, characterized in that: The embodied intelligent body system includes an intervention interface; wherein, the intervention interface is used for users to directly access the adjusted knowledge content, the knowledge content library, input content improvement instructions, and insert knowledge content.

4. The teaching method based on embodied intelligent agent system according to claim 1, characterized in that: Generating a demonstration method corresponding to the knowledge content to be demonstrated, and demonstrating the knowledge content to be demonstrated through the knowledge demonstration script includes: Extracting roles from the knowledge content to be demonstrated and assigning a name to each role; Traversing the embodied bodies of the embodied agent system, and assigning a body to each role based on the type and number of the embodied bodies and the characteristics of the roles in the knowledge content to be demonstrated; Generate a portrait for each embodied character based on their characteristics; Combining the name of the physical character and the portrait of the physical character, generating a demonstration script line by line according to the knowledge content to be demonstrated; wherein each line of the script contains a character and an action, and contains at least one line of narration; Parsing the behavior of embodied characters into sequences of actions of embodied characters; Demonstrate action sequences with physical characters and play corresponding narration.

5. The teaching method based on embodied intelligent agent system according to claim 4, characterized in that: Combining the name of the physical character and the portrait of the physical character, generating a demonstration script line by line according to the knowledge content to be demonstrated includes: Decomposing the knowledge content to be demonstrated into semantic-level scripts, and outputting semantic-level instructions; Determine whether the semantic-level instruction conforms to a preset expression format through a regular expression, and obtain a determination result; When the judgment result is negative, the instruction at the output semantic level is corrected according to a preset error correction table.

6. The teaching method based on embodied intelligent agent system according to claim 4, characterized in that: In the case where the multiple embodiments are robot clusters, each line of the script indicates that a designated robot moves from a first coordinate position to a second coordinate position, and a narration is played; Based on the second coordinate position, combined with the first coordinate position when the cluster robot is initialized, the coordinate position is transformed into the actual coordinate system of the physical world, and the actual coordinate position and the line audio are sent to each cluster robot through the local area network; The built-in control board of the cluster robot controls its built-in motor to move the robot to the actual coordinate position, and plays the line audio through the built-in speaker.

7. The teaching method based on embodied intelligent agent system according to claim 4, characterized in that: The embodiment includes a physical embodiment and a virtual embodiment; when it is determined that the role is not suitable for interpretation by a physical embodiment, the role is assigned to a virtual embodiment for interpretation; the virtual embodiment is selected from a pre-set three-dimensional model library.

8. A teaching device based on an embodied intelligent system, characterized in that: The device comprises: A receiving module, used for receiving the knowledge subject to be learned; An expansion module, used for acquiring knowledge content to be demonstrated according to the knowledge subject to be learned; A demonstration module is used to generate a demonstration method corresponding to the knowledge content to be demonstrated, and demonstrate the knowledge content to be demonstrated through a knowledge demonstration script; wherein the demonstration method includes a knowledge demonstration script generated based on the knowledge content to be demonstrated and the type and number of the intelligent system's own embodiments.

9. According to the teaching device based on the embodied intelligent system of claim 8, the expansion module comprises: An expansion unit, used for expanding the knowledge subject to be learned to form expanded knowledge content; A receiving unit, used for receiving an adjustment instruction; A positioning unit, used to locate the knowledge content referred to by the adjustment instruction through an intelligent agent positioning module; wherein the intelligent agent positioning module is implemented by at least one of the following methods: a single-head self-attention Transformer network, a keyword matching algorithm, and a bag-of-words model search algorithm; An adjustment unit, used to adjust the indicated knowledge content using the adjustment instruction until the adjusted knowledge content meets the predetermined requirement; wherein the adjusted knowledge content includes the complete knowledge content consisting of the adjusted content part and the content part that does not need to be adjusted; A memory unit is used to input the adjusted knowledge content into an intelligent body memory module, and select and compress the adjusted knowledge content and store it in a knowledge content library; wherein the intelligent body memory module is implemented in at least one of the following ways: a language model, a pre-trained transformer model, and a pre-trained BERT model.

10. The teaching device based on embodied intelligent system according to claim 8, characterized in that: The demo modules include: An extraction unit, used for extracting roles from the knowledge content to be demonstrated and assigning a name to each role; A traversal unit, used to traverse the embodied bodies of the embodied agent system, and assign a body to each role in combination with the type and number of the embodied bodies and the characteristics of the roles in the knowledge content to be demonstrated; A first generating unit, for generating a portrait for each embodied character according to the characteristics of the character; A second generating unit is used to generate a demonstration script line by line according to the knowledge content to be demonstrated, combining the name of the embodied character and the portrait of the embodied character; wherein each line of the script contains a character and an action, and contains at least one line of narration; A parsing unit, for parsing the behavior of the embodied character into an action sequence of the embodied character; Demonstration unit, used to demonstrate the action sequences of physical characters and play corresponding narration.

11. An electronic device comprising: processor; as well as Memory for storing programs, The program includes instructions, which, when executed by the processor, cause the processor to perform the method according to any one of claims 1 to 7.

12. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to make a computer execute the method according to any one of claims 1-7.

13. A computer program product, characterized in that The computer program product comprises instructions which, when executed, cause a computer to perform the method of any one of claims 1 to 7.

Images