Teaching game making method and device and electronic equipment

Abstract

The invention provides a teaching game making method and device and electronic equipment, and the method comprises the steps: recognizing a role description text and a virtual scene description text from voice media data collected by an intelligent terminal, and generating a game scene picture according to the virtual scene description text; obtaining visual media data collected by the intelligent terminal, and generating game role actions according to the visual media data and the role description text; and making a teaching game according to the game scene picture and the game role action. In the implementation process of the scheme, the game scene picture is generated through the virtual scene description text in the voice media data collected from the intelligent terminal, and the teaching game is made according to the game role action generated according to the visual media data, so that children participate in the role design and scene creation process, and the teaching effect is improved. The game content can be customized according to specific requirements and interests of each child, so that the individuation degree of teaching game making is effectively improved.

Description

Technical Field

[0001] This application relates to the technical fields of human-computer interaction, personalized teaching, artificial intelligence, and game production. Specifically, it relates to a method, device, and electronic device for producing teaching games. Background Art

[0002] In traditional early childhood education, the development of teaching resources is usually a time-consuming and resource-intensive process. Teachers and developers need to spend a lot of time and energy designing, producing, and testing teaching games suitable for children of different ages. In addition, most of these games adopt a "one-size-fits-all" model, which cannot be customized according to the unique needs and interests of each child, and it is difficult to adjust according to the interests and learning progress of each child. Children often just passively accept the designed teaching games. The design and production process of traditional teaching games is completely dominated by adults, and children have little chance to participate in the production of teaching games. Therefore, the current degree of personalization in teaching game production is relatively low. Summary of the Invention

[0003] The purpose of the embodiments of this application is to provide a method, device, and electronic device for producing teaching games, which are used to improve the problem of relatively low personalization in teaching game production.

[0004] The embodiments of this application provide a method for producing teaching games, including: identifying a character description text and a virtual scene description text from the voice media data collected by a smart terminal, and generating a game scene picture according to the virtual scene description text; obtaining the visual media data collected by the smart terminal, and generating a game character action according to the visual media data and the character description text; producing a teaching game according to the game scene picture and the game character action. In the implementation process of the above solution, by generating a game scene picture from the virtual scene description text in the voice media data collected by the smart terminal, and producing a teaching game based on the game character action generated from the visual media data, allowing children to participate in the process of character design and scene creation can not only improve their artistic creation ability, but also exercise their language expression ability and logical thinking ability, and can customize game content according to the specific needs and interests of each child, thus effectively improving the degree of personalization in teaching game production.

[0005] Optionally, in the embodiments of the present application, generating game character actions according to visual media data and character description text includes: identifying human postures from the visual media data; generating game character pictures according to the character description text; and determining game character actions according to the human postures and the game character pictures. In the implementation process of the above solution, by using human pose estimation technology to convert the actual actions of children into the actions of game characters, children can directly control the game characters to carry out various activities. This direct physical interaction greatly enhances the immersion and interactivity of the game, making children not only passive game participants, but also active creators and controllers, improving their participation and concentration, and thus effectively improving the personalization degree of educational game production.

[0006] Optionally, in the embodiments of the present application, identifying human postures from the visual media data includes: identifying human postures from the point cloud data in the visual media data, where the point cloud data is acquired by the scanning mechanism of the intelligent terminal; and / or, identifying human postures from the multi-view picture data and / or multi-view video data in the visual media data, where the multi-view picture data and / or multi-view video data are acquired by the camera of the intelligent terminal. In the implementation process of the above solution, by using point cloud data or picture and video data from multiple perspectives, more abundant information can be provided, enabling the system to still maintain good performance in complex environments (such as light changes, occlusions, etc.), effectively enhancing the accuracy and robustness of human posture recognition.

[0007] Optionally, in the embodiments of the present application, after identifying the character description text and the virtual scene description text from the voice media data collected by the intelligent terminal, it further includes: obtaining teaching object information text; filling the teaching object information text, the character description text, and the virtual scene description text into a preset prompt engineering template to obtain prompt engineering text; and inputting the prompt engineering text into a pre-trained large language model to obtain the teaching course content output by the pre-trained large language model. In the implementation process of the above solution, by combining teaching object information (such as age, interests, learning progress, etc.) with character description and virtual scene description, the system can generate highly personalized and targeted teaching course content. This customized content can not only better meet the unique needs of each child, but also significantly improve their learning motivation and participation, effectively realizing the personalization of the learning path of children.

[0008] Optionally, in the embodiments of the present application, after identifying the character description text and the virtual scene description text from the voice media data collected by the intelligent terminal, the method further includes: obtaining the teaching object information text; inputting the teaching object information text, the character description text, and the virtual scene description text into the game agent intelligent body to obtain the teaching course content output by the game agent intelligent body, where the game agent intelligent body is constructed by a pre-trained large language model. In the implementation process of the above solution, by combining the teaching object information such as the child's age, interest, learning progress, etc., and the character description and virtual scene description provided by the child, the game agent intelligent body can generate highly customized teaching course content, so that each child can obtain a unique learning experience, which can not only better match their personal needs, but also significantly improve their participation and learning motivation, thereby improving the personalization degree of the interaction experience.

[0009] Optionally, in the embodiments of the present application, the teaching course content includes: math calculation exercises, language and character learning tasks, art creation guidance tutorials, and / or handicraft production guidance tutorials. In the implementation process of the above solution, by integrating various types of educational content (such as mathematics, language, art, etc.), children can access interdisciplinary knowledge on a unified platform, enabling children to access knowledge in multiple fields in a single game environment, effectively improving the efficiency of education.

[0010] The embodiments of the present application also provide a teaching game production device, including: a game scene generation module, configured to identify the character description text and the virtual scene description text from the voice media data collected by the intelligent terminal, and generate game scene pictures according to the virtual scene description text; a character action generation module, configured to obtain the visual media data collected by the intelligent terminal, and generate game character actions according to the visual media data and the character description text; a teaching game production module, configured to produce a teaching game according to the game scene pictures and the game character actions.

[0011] Optionally, in the embodiments of the present application, the character action generation module includes: a human body posture recognition sub-module, configured to recognize the human body posture from the visual media data; a character picture generation sub-module, configured to generate game character pictures according to the character description text; a character action determination sub-module, configured to determine the game character actions according to the human body posture and the game character pictures.

[0012] Optionally, in the embodiments of the present application, the human body posture recognition sub-module includes: a first human body posture recognition unit for recognizing the human body posture from the point cloud data in the visual media data, where the point cloud data is acquired by the scanning mechanism of the intelligent terminal; and / or a second human body posture recognition unit for recognizing the human body posture from the multi-view picture data and / or multi-view video data in the visual media data, where the multi-view picture data and / or multi-view video data are acquired by the camera of the intelligent terminal.

[0013] Optionally, in the embodiments of the present application, the teaching game production device further includes: a first object information acquisition module for acquiring the teaching object information text; a prompt engineering text acquisition module for filling the teaching object information text, the character description text, and the virtual scene description text into a preset prompt engineering template to obtain the prompt engineering text; a first teaching course content output module for inputting the prompt engineering text into a pre-trained large language model to obtain the teaching course content output by the pre-trained large language model.

[0014] Optionally, in the embodiments of the present application, the teaching game production device further includes: a second object information acquisition module for acquiring the teaching object information text; a second teaching course content output module for inputting the teaching object information text, the character description text, and the virtual scene description text into a game agent intelligent body to obtain the teaching course content output by the game agent intelligent body, where the game agent intelligent body is constructed by a pre-trained large language model.

[0015] Optionally, in the embodiments of the present application, the teaching course content includes: math calculation exercises, language and character learning tasks, art creation guidance tutorials, and / or handicraft production guidance tutorials.

[0016] The embodiments of the present application further provide an electronic device, including: a processor and a memory, where the memory stores machine-readable instructions executable by the processor, and when the machine-readable instructions are run by the processor, the methods described above are executed.

[0017] The embodiments of the present application further provide a computer-readable storage medium, on which a computer program is stored, and when the computer program is run by a processor, the methods described above are executed.

[0018] The embodiments of the present application further provide a computer program product, including: a computer program or computer instructions, and when the computer program or computer instructions are run by a processor, the methods described above are executed. Description of the Drawings

[0019] To more clearly illustrate the technical solutions of the embodiments of the present application, the following will briefly introduce the accompanying drawings required for use in the embodiments of the present application. It should be understood that the following drawings only show some embodiments of the embodiments of the present application, and thus should not be regarded as limiting the scope. For those of ordinary skill in the art, without creative efforts, other related drawings can also be obtained based on these drawings.

[0020] Figure 1 Schematic flowchart of the teaching game production method provided by the embodiment of the present application shown;

[0021] Figure 2 Schematic diagram of the processing process of visual media data provided by the embodiment of the present application shown;

[0022] Figure 3 Schematic diagram of the teaching course content of the mathematics course provided by the embodiment of the present application on the graphical user interface shown;

[0023] Figure 4 Schematic diagram of the structure of the teaching game production device provided by the embodiment of the present application shown;

[0024] Figure 5 Schematic diagram of the structure of the electronic device provided by the embodiment of the present application shown. Specific embodiments

[0025] To make the objectives, technical solutions, and advantages of the embodiments of the present application clearer, the following will clearly and completely describe the technical solutions in the embodiments of the present application in conjunction with the accompanying drawings in the embodiments of the present application. It should be understood that the accompanying drawings in the embodiments of the present application only serve the purpose of illustration and description, and are not used to limit the protection scope of the embodiments of the present application. Additionally, it should be understood that the schematic drawings are not drawn to actual scale. The flowcharts used in the embodiments of the present application show the operations implemented according to some embodiments of the embodiments of the present application. It should be understood that the operations in the flowchart may not be implemented in sequence, and steps without logical context relationships may be reversed or implemented simultaneously. In addition, those skilled in the art can add one or more other operations to the flowchart or remove one or more operations from the flowchart under the guidance of the content of the embodiments of the present application.

[0026] In addition, the described embodiments are only a part of the embodiments of the present application, rather than all of the embodiments. The components of the embodiments of the present application usually described and shown in the accompanying drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present application provided in the accompanying drawings is not intended to limit the scope of the claimed embodiments of the present application, but only represents the selected embodiments of the present application.

[0027] It should be understood that in the embodiments of the present application, "first" and "second" are used to distinguish similar objects. Those skilled in the art can understand that words such as "first" and "second" do not limit the quantity and execution order, and words such as "first" and "second" do not necessarily limit being different. In the description of the embodiments of the present application, the term "and / or" is merely a description of the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B may represent three situations: A exists alone, A and B exist simultaneously, and B exists alone. In addition, the character " / " in this article generally represents an "or" relationship between the associated objects before and after. The term "plurality" refers to more than two (including two). Similarly, "multiple groups" refers to more than two groups (including two groups).

[0028] It should be noted that the teaching game production method provided in the embodiments of the present application can be executed by an electronic device. Here, the electronic device refers to a device terminal or a server with the function of executing a computer program. Examples of the device terminal include: smart phones, personal computers, tablet computers, personal digital assistants, or mobile Internet devices, etc. A server refers to a device that provides computing services through a network. Examples of the server include: x86 servers and non-x86 servers. Non-x86 servers include: mainframes, minicomputers, and UNIX servers.

[0029] The following introduces the application scenarios applicable to the teaching game production method. In traditional early childhood education, the development of teaching resources such as handicrafts, music, and art is usually a time-consuming and resource-intensive process, requiring teaching through entertainment and combining games to produce teaching content. Currently, teachers and developers need to spend a lot of time and energy designing, producing, and testing teaching games suitable for children of different ages. In addition, during the production process of teaching games, children hardly participate interactively in the production of teaching games. Most adopt a "one-size-fits-all" model and cannot be customized according to the unique needs and interests of each child. This lack of flexibility in design may cause some children to be uninterested in the teaching content or feel that the difficulty is inappropriate, thus affecting the learning effect. Since the design and production process of traditional teaching games is completely dominated by adults, children hardly have the opportunity to participate in this process, which not only limits their opportunities to exert creativity but also reduces their interest and sense of identity in the game content. Therefore, the current game production method lacks the ability of personalized customization and is difficult to adjust according to the interests and learning progress of each child. Children often just passively accept the designed teaching games.

[0030] Regarding the above problems, please refer to Figure 1Schematic flowchart of the teaching game production method provided by the embodiment of the present application shown; the main idea of this teaching game production method is to produce teaching games through the voice media data and visual media data collected by intelligent terminals, enabling children to participate in the process of character design and scene creation, and being able to customize game content according to the specific needs and interests of each child, thereby effectively improving the personalization degree of teaching game production. The implementation manners of the above teaching game production method may include:

[0031] Step S110: Identify the character description text and the virtual scene description text from the voice media data collected by the intelligent terminal, and generate game scene pictures according to the virtual scene description text.

[0032] An intelligent terminal refers to a device that can perform complex computing tasks and has data collection capabilities, such as a smart phone, a tablet computer, a smart watch or a dedicated educational device, etc. These devices can collect the voice media data of users through built-in or external microphones or other audio input devices, and can also collect images or video streams through cameras.

[0033] Voice media data is the sound signal data obtained through sound collection devices such as microphones, and can be voice data or audio data generated by users' speaking, singing or other forms of sound input.

[0034] The character description text refers to the text information extracted from the voice media data and used to describe the appearance, personality, abilities and other characteristics of game characters. This character description text can be the text information obtained after natural language processing of the voice media data. For example, the character is a Peashooter, and the text describing this character includes that the Peashooter can shoot pea bullets.

[0035] The virtual scene description text refers to the text information extracted from the voice media data and used to describe the environment, layout, objects and other characteristics of the game scene. This virtual scene description text can be the text information obtained after natural language processing of the voice media data. For example, fighting in a farmland garden or on the roof of a city.

[0036] The game scene pictures are game background pictures or scene pictures generated according to the virtual scene description text, such as: background pictures of farmland gardens, night gardens, or rooftops of cities. It can be understood that when generating game scene pictures with different styles but the same characters, the style transfer technology can be directly used for migration processing in game scenes with different styles. For example, in the case where a game scene picture of a farmland garden has been generated, a Generative Adversarial Network (GAN) model or a Diffusion Model can be used to generate a game scene picture of a night garden based on the game scene picture of the farmland garden. In the above implementation process, by using the style transfer technology for migration processing in game scenes with different styles, the visual effect of the game can be enhanced. In addition, images or videos that match the descriptions of young children can also be generated. For example, young children can generate game character pictures and scene pictures through voice descriptions. This interactive method is not only interesting but also helps young children better express their ideas and can effectively stimulate their imagination and creativity.

[0037] Step S120: Obtain the visual media data collected by the intelligent terminal, and generate game character actions according to the visual media data and the character description text.

[0038] Please refer to Figure 2 the schematic diagram of the processing process of the visual media data provided by the embodiment of the present application shown; the visual media data can be images or video streams collected by the intelligent terminal through the camera, such as image data of a new plant painting or handicraft drawn by the user captured by the camera, or real-time capture data such as the user's body movements and facial expressions. Of course, in the specific practice process, it can also be visual media data that the user has already collected through the intelligent terminal. At this time, the picture or video data directly uploaded by the intelligent terminal can be used. For example, a child drew a "new plant painting" and took a photo with the intelligent terminal and uploaded it to the server. This "new plant painting" is used to import into the teaching game, and this "new plant painting" is the visual media data.

[0039] The game character actions are the specific actions or behaviors of the character in the game generated based on the visual media data and the character description text. For example, according to paintings such as "sunflower" or "pea shooter" and the voice description of the character description text, an action animation of the sunflower growing taller and generating sunlight can be generated, or an action animation of the pea shooter shooting pea bullets can also be generated.

[0040] Step S130: Produce a teaching game according to the game scene picture and the game character actions.

[0041] It can be understood that, compared with the traditional teaching game production method, the development process of the traditional method is complex. From the creative concept to the final product launch, it needs to go through multiple links, including market research, concept design, prototype development, user testing, etc. The entire production cycle takes a long time and most links require human participation, resulting in low development efficiency. The above-mentioned teaching game production method combines multi-modal data (including voice media data and visual media data) collected by intelligent terminals to produce teaching games, thus realizing an automated process from user input to automatic generation of teaching game content. This not only simplifies the development process of teaching resources and reduces costs, but also gives children more creative freedom. In addition, the above-mentioned teaching game production method can quickly generate corresponding game scenes and character actions by automatically identifying voice descriptions and processing visual works, reducing the need for professionals and the number of reworks caused by human errors, thus reducing labor costs and greatly shortening the development cycle, and therefore effectively improving the development efficiency of teaching game production.

[0042] In the implementation process of the above solution, by generating game scene pictures from the virtual scene description text in the voice media data collected from the intelligent terminal, and making teaching games based on the game character actions generated from the visual media data, allowing children to participate in the process of character design and scene creation, can not only improve their artistic creation ability, but also exercise their language expression ability and logical thinking ability, and can customize game content according to the specific needs and interests of each child, thus effectively improving the personalization degree of teaching game production.

[0043] Optionally, the implementation manner of identifying the character description text and the virtual scene description text from the voice media data collected from the intelligent terminal in the above step S110 may include: using speech recognition technology (such as Speech-to-Text or Deep Speech) to convert the voice media data into text. Specifically, assume that a child says to the intelligent terminal: "I want to create a super sunflower that can glow at night and illuminate the whole garden". After using speech recognition technology (such as Speech-to-Text or Deep Speech) to recognize it into text, the pre-trained large language model (such as DeepSeek or ChatGPT) can also be used to identify "super sunflower" as the character description text and "glow at night and illuminate the whole garden" as the virtual scene description text from the text. It can be understood that before identifying from the text by the pre-trained large language model, the pre-trained large language model (such as DeepSeek or ChatGPT) can also be used to preprocess the conversations of young children to generate more teaching goal-compliant conversation content. This technology can help young children better express their ideas and at the same time provide more accurate teaching feedback for teachers.

[0044] Optionally, in addition to using the character description text and virtual scene description text identified from the voice media data, game sound effects can also be generated based on the voice media data collected by the smart terminal, so as to provide more audio materials for the teaching game, and also give children more opportunities to participate in the teaching game and personalized experiences, which can better stimulate children's learning interest and improve learning effects.

[0045] As an implementation manner of generating game scene pictures according to the virtual scene description text in the above step S110, it may include: First, a rule-based generation method: Using predefined rules and templates, corresponding game scene pictures are generated according to the keywords in the virtual scene description text. For example, if the description text mentions "sunflower", the pre-stored sunflower picture template can be called for generation. Second, image synthesis-based generation: Using image synthesis technology, each element in the virtual scene description text (such as "fresh sunflower", "green grassland") is synthesized into a complete game scene picture. Third, deep learning-based generation: Using image generation deep learning models such as the Stable Diffusion model or generative adversarial network (GAN), the virtual scene description text is directly converted into the corresponding picture, and this deep learning model can rely on a large amount of training data and complex neural network structures.

[0046] As an optional implementation manner of the above step S120, the implementation manner of generating game character actions according to the visual media data and character description text may include:

[0047] Step S121: Identify the human body posture from the visual media data.

[0048] It can be understood that specifically, the visual media data can be subjected to posture estimation through vision-based human body posture estimation technology to identify the human body posture from the visual media data. Since there are many vision-based human body posture estimation technologies, the implementation manner of this step will be described in detail below.

[0049] Step S122: Generate game character pictures according to the character description text.

[0050] For example, the implementation of the above step S122 is as follows: Fill the character description text into a preset prompt text template to obtain the prompt text content, and then input the prompt text content into a multi-modal generation model to obtain a game character picture. Among them, the above multi-modal generation model may include: the StableDiffusion model, the Midjourney model, etc. It can be understood that in the process of generating a game character picture according to the prompt text content by using multi-modal generation models such as Stable Diffusion and Midjourney, a game character video can also be generated simultaneously according to the prompt text content. This technology can automatically generate corresponding visual content according to the description or input of children, greatly reducing the threshold of content creation and enabling children to more intuitively see their ideas become reality.

[0051] Step S123: Determine the game character actions according to the human body posture and the game character picture.

[0052] For example, the implementation of the above step S123 is as follows: Use the 4DHumans prediction model in a 3D human action generation framework based on deep learning (such as the 3DHM framework) to process the human body posture and the game character picture, so as to obtain the game character actions generated by the 4DHumans prediction model. Or, a virtual engine such as Unreal Engine can be used to process the human body posture and the game character picture, so as to obtain the game character actions processed by the virtual engine. Optionally, the actual actions of the child can be converted into the actions of the game character by using human body posture estimation technology, so that the children can directly control the game character to carry out various activities. For example, when the child makes a jumping action, the game character will also jump accordingly. This direct physical interaction greatly enhances the immersion and interactivity of the game, making the children not only passive game participants, but also active creators and controllers, improving their participation and concentration, and thus effectively improving the personalization degree of the teaching game production.

[0053] As an alternative implementation of the above step S121, the above visual media data may include: point cloud data of the target object, multi-view picture data, and / or multi-view video data; the implementation of identifying the human body posture from the visual media data may include:

[0054] Step S121a: Identify the human body posture from the point cloud data, and the point cloud data is obtained by the scanning mechanism of the intelligent terminal.

[0055] For example, the implementation of the above step S121a is as follows: If the above terminal device is equipped with a scanning mechanism, then the target object can be scanned through the scanning mechanism of the terminal device to obtain a point cloud model composed of the point cloud data of the target object. Here, the target object can be handicrafts made by kindergarten children (such as new plants made by hand with clay) and / or paintings drawn, etc. Then, a heat map (HeatMap) is generated based on the point cloud data of the target object. The heat map represents the possible position distribution of each key point. Then, the heat map is input into a pre-trained multi-modal model so that the multi-modal model predicts the position coordinates of the human key points based on the input heat map, thereby obtaining the above human posture.

[0056] And / or, the implementation of identifying the human posture from the visual media data may include:

[0057] Step S121b: Identify the human posture from multi-view image data and / or multi-view video data, and the multi-view image data and / or multi-view video data can be collected by the camera of the intelligent terminal.

[0058] For example, the implementation of the above step S121b is as follows: After collecting multi-view image data and / or multi-view video data through the camera of the intelligent terminal, that is, after using the camera of the intelligent terminal to take pictures of the target object from multiple perspectives, computer vision techniques (such as OpenCV or MediaPipe) can be used to preprocess the multi-view image data and / or multi-view video data, including operations such as cropping, scaling, denoising, binarization, and / or contour extraction, to obtain the preprocessed image data. Then, a convolutional neural network (CNN) or other deep learning models (such as OpenPose, MediaPipe, AlphaPose, etc.) can be used to locate the key points of the human body (such as joint positions) in the multi-view image data and / or multi-view video data. These key points usually include parts such as the head, shoulders, elbows, wrists, hips, knees, and ankles. Finally, a regression model is used to perform regression calculations on the key points located in the multi-view image data and / or multi-view video data, thereby calculating the above human posture.

[0059] Optionally, after obtaining multi-view image data and / or multi-view video data, a heat map can also be directly generated based on the multi-view image data and / or multi-view video data. The heat map represents the possible position distribution of each key point. Then, the heat map is input into a pre-trained multi-modal model so that the multi-modal model can predict the position coordinates of the human key points based on the input heat map, thereby obtaining the above-mentioned human pose. In the implementation process of the above solution, by using point cloud data, images and / or video data from multiple views, more abundant information can be provided, enabling the system to still maintain good performance in complex environments (such as light changes, occlusions, etc.). For example, in outdoor teaching activities, even under different lighting conditions or with slight occlusions, the system can accurately capture the movements of children, effectively enhancing the accuracy and robustness of human pose recognition. In addition, by precisely identifying the human pose and converting it into the character actions in the game, children can experience a more realistic interactive effect. This highly realistic interaction method not only enhances the user's immersion, but also stimulates children's interest and curiosity in technology, further improving the realism and immersion of the user experience.

[0060] As the first optional implementation manner of the above step S130, for example: Integrate the generated game scene picture (night garden) and game character actions (super sunflower swaying and glowing) into a game development environment (such as Unity or Unreal Engine). Then, incorporate the game scene picture and game character actions into a pre-designed specific interactive task or storyline, thereby producing the teaching game "Plants vs. Insects". In this teaching game, children can plant super sunflowers to defend against the insects attacking at night, use their light to illuminate the path and discover hidden treasures. Optionally, after producing the teaching game "Plants vs. Insects", the game content can also be continuously adjusted and improved through user testing to ensure that it is both interesting and educational. For example, add guiding prompts to help children understand the game goals or complete specific tasks.

[0061] As a second alternative implementation of the above step S130, for example: digitize the generated game scene pictures (such as a night garden), three-dimensional objects (such as handicrafts made of clay by children), two-dimensional paintings, and game character actions. Specifically, input the game scene pictures, visual media data (such as images or videos collected for three-dimensional objects and / or two-dimensional paintings), and game character action data into a pre-trained large language model (such as DeepSeek or ChatGPT) to generate customized learning content according to the learning progress and needs of young children, and construct a mapping from the real world to the virtual world. This method of multi-modal data fusion can capture and transform various information in reality more comprehensively, provide richer materials for the generation of teaching games, and enable children to be not only passive game participants but also active creators and controllers, enhancing their participation and concentration, thereby effectively improving the personalization degree of teaching game production.

[0062] It can be understood that the above teaching games can be two-dimensional (2D) games or three-dimensional (3D) games, which can be set according to specific circumstances. For example, if the handicrafts made by children are clay handicrafts, this situation is more suitable for making three-dimensional teaching games. Another example is that if the children's works are new plant paintings, this situation is more suitable for making two-dimensional teaching games. Therefore, the specific form and dimension of the above teaching games should not be construed as a limitation of the embodiments of the present application.

[0063] Optionally, advanced technologies such as speech recognition and computer vision can also be incorporated into the game production process to create novel human-computer interaction methods. For example, children can directly interact with the game through speech to create characters or scenes, or directly affect the design of game elements through visual inputs such as painting. This way breaks through the limitations of traditional game interfaces and provides a brand-new interactive experience. In the process of participating in game design, children are not only players but also creators, which requires children to comprehensively apply knowledge in various aspects such as art (such as painting), language (such as describing characters and scenes), and science (such as understanding the biological characteristics in Plants vs. Insects), helping to cultivate interdisciplinary knowledge integration ability and problem-solving ability. In the implementation process of the above solution, by customizing game content according to the specific needs and interests of each child, this method can not only attract children to participate more deeply in the game but also better meet the learning styles and rhythms of different children, thereby promoting the development of personalized learning paths.

[0064] As an alternative implementation of the above teaching game production method, after identifying the character description text and virtual scene description text from the voice media data collected from the intelligent terminal, a pre-trained large language model can also be used to produce teaching course content. This implementation includes:

[0065] Step S140: Obtain the teaching object information text.

[0066] The implementation manner of the above step S140 is, for example: Read the teaching object information text from a pre-stored database or file system. The above teaching object information text refers to the demand information of the teaching object that needs to participate. For example, the demand information of children in an art course is that children draw pictures of new plants. After collecting visual media data (such as photos or videos) of the pictures of new plants, upload the visual media data to the server and import it into the above teaching game. Another example of the above teaching object information text is the demand information in a handicraft course, that is, children use clay to make models of new plants. After collecting visual media data of the models of new plants, upload the visual media data to the server and import it into the above teaching game. Another example of the above teaching object information text is the demand information in a mathematics course, that is, determine the toughness value (that is, how many values can be attacked before death) and attack power value of a specific plant, and these toughness values and attack power values can be set through a Graphical User Interface (GUI).

[0067] Step S150: Fill the teaching object information text, the character description text, and the virtual scene description text into a preset prompt engineering template to obtain the prompt engineering text.

[0068] The implementation manner of the above step S150 is, for example: Assume that a corresponding prompt engineering template (Prompt Project Template) is pre-designed for each course for children. Then, first obtain the course type corresponding to the teaching object information text, and obtain the prompt engineering template corresponding to this course type according to the course type corresponding to the teaching object information text. Then, fill the teaching object information text, the character description text, and the virtual scene description text into the preset prompt engineering template corresponding to this course type, so as to obtain the prompt engineering text.

[0069] Step S160: Input the prompt engineering text into a pre-trained large language model to obtain the teaching course content output by the pre-trained large language model.

[0070] For example, the implementation of step S160 is as follows: Input the prompt engineering text into the pre-trained large language model to obtain teaching course content such as art painting, writing, handicrafts, construction, language, picture books, mathematics, sports, etc. output by the pre-trained large language model. In the implementation process of the above solution, by combining teaching object information (such as age, interests, learning progress, etc.) with character descriptions and virtual scene descriptions, the system can generate highly personalized and targeted teaching course content. For example, a child who is particularly interested in dinosaurs may receive more dinosaur-themed games and learning tasks. Such customized content can not only better meet the unique needs of each child, but also significantly improve their learning motivation and engagement, effectively realizing the personalization of the child's learning path.

[0071] Optionally, by leveraging the powerful natural language processing capabilities of the pre-trained large language model, the system can dynamically generate teaching content based on real-time input information and provide instant feedback. This immediacy and flexibility enable teachers and parents to quickly respond to children's learning progress and adjust teaching strategies in a timely manner. For example, if a child encounters a problem in a game, the system can immediately generate relevant explanations or prompts to help them overcome the difficulty, thereby enhancing the flexibility of generating teaching course content.

[0072] Please refer to Figure 3 the schematic diagram of the teaching course content of the math class provided by the embodiment of the present application shown on the graphical user interface; the above-mentioned teaching course content is diverse, and only some examples are given here to illustrate. The teaching course content of art painting can be to draw paintings such as sunflowers, Peashooters, Fume-Shrooms, and / or Squash plants, etc. The teaching course content of the handicraft class can be to make Potato Mines using clay, etc. The teaching course content of the Chinese class can be the pronunciation of plants and the recognition and writing of Chinese characters. The teaching course content of the English class can be the listening, speaking, reading, and writing of plant English words. The teaching course content of the math class can be to calculate the sun value required to purchase plants (such as Peashooters), or calculate the number of plants required to defeat insects. For a specific example, if the toughness of an insect is 3000 and the attack power of a Squash is 1800, how many Squashes are needed to defeat it?

[0073] As an alternative implementation of the above teaching game production method, after identifying the character description text and virtual scene description text from the voice media data collected by the intelligent terminal, a game agent intelligent body can also be used to produce teaching course content. This implementation includes:

[0074] Step S170: Obtain the teaching object information text.

[0075] Among them, the implementation manner of the above step S170 is the same as that of step S140. Therefore, for unclear parts, reference can be made to the implementation manner of step S140, which will not be elaborated here.

[0076] Step S180: Input the teaching object information text, role description text, and virtual scene description text into the game agent intelligent body to obtain the teaching course content output by the game agent intelligent body. The game agent intelligent body is constructed through a pre-trained large language model.

[0077] For example, the implementation manner of the above step S180 is: Input the teaching object information text, role description text, and virtual scene description text into the game agent intelligent body (game intelligence agent) constructed by using LangChain technology, AnythingLLM, Dify technology, and / or Llama-Index to obtain the teaching course content output by the game agent intelligent body. The game agent intelligent body is constructed through a pre-trained large language model. The pre-trained large language model here is, for example, DeepSeekV3 and / or DeepSeek R1, etc. In the specific implementation process, the game agent intelligent body based on the pre-trained large language model can dynamically adjust the teaching content according to the real-time input information and provide instant feedback. This instantaneity and flexibility make the educational process more adaptable to individual differences. For example, if a child encounters difficulties in the game, the system can immediately provide targeted help or tips to help them overcome obstacles and continue. The above teaching game production method not only promotes the application of interdisciplinary knowledge, enabling children to access knowledge in multiple fields in a single game environment. For example, when designing a game about plants vs. insects, children can not only learn botany knowledge but also understand relevant content such as mathematical calculations and historical backgrounds.

[0078] In the implementation process of the above solution, by combining teaching object information such as the child's age, interest, and learning progress, as well as the role description and virtual scene description provided by the child, the game agent intelligent body can generate highly customized teaching course content, enabling each child to obtain a unique learning experience. This can not only better match their personal needs but also significantly improve their participation and learning motivation, thus enhancing the personalization degree of the interactive experience.

[0079] As an alternative implementation of the above teaching game production method, the teaching course content includes: math calculation exercises (such as calculating the sun value required to purchase plants), language and writing learning tasks (such as the pronunciation and recognition of plant pinyin and Chinese characters, as well as writing), art creation guidance tutorials (tutorials for guiding the drawing of sunflowers, pea shooters, hot peppers, and / or pumpkin plants, etc.) and / or handicraft production guidance tutorials (such as tutorials for making potato mines using clay). In the implementation process of the above solution, by integrating various types of educational content (such as math, language, art, etc.), children can access interdisciplinary knowledge on a unified platform, enabling them to come into contact with knowledge in multiple fields in a single game environment. This multi-disciplinary integration method can improve the efficiency of education and help young children develop comprehensively.

[0080] Please refer to Figure 4 the structural schematic diagram of the teaching game production device provided by the embodiment of the present application shown in; The embodiment of the present application provides a teaching game production device 200, including:

[0081] A game scene generation module 210, configured to identify a character description text and a virtual scene description text from the voice media data collected by the smart terminal, and generate a game scene picture according to the virtual scene description text.

[0082] A character action generation module 220, configured to obtain the visual media data collected by the smart terminal, and generate a game character action according to the visual media data and the character description text.

[0083] A teaching game production module 230, configured to produce a teaching game according to the game scene picture and the game character action.

[0084] As an alternative implementation of the above device, the character action generation module includes:

[0085] A human body posture recognition sub-module, configured to recognize the human body posture from the visual media data.

[0086] A character picture generation sub-module, configured to generate a game character picture according to the character description text.

[0087] A character action determination sub-module, configured to determine the game character action according to the human body posture and the game character picture.

[0088] As an alternative implementation of the above device, the human body posture recognition sub-module includes:

[0089] A first human body posture recognition unit, configured to recognize the human body posture from the point cloud data in the visual media data, and the point cloud data is obtained by the scanning mechanism of the smart terminal.

[0090] And / or, a second human posture recognition unit is used to recognize human posture from multi-view image data and / or multi-view video data in the visual media data, where the multi-view image data and / or multi-view video data are collected by a camera of the smart terminal.

[0091] As an optional implementation of the above device, the teaching game making device further includes:

[0092] The first object information acquisition module is used to acquire teaching object information text.

[0093] The prompt engineering text acquisition module is used to fill the teaching object information text, role description text and virtual scene description text into the preset prompt engineering template to obtain the prompt engineering text.

[0094] The first teaching course content output module is used to input the prompt engineering text into the pre-trained large language model to obtain the teaching course content output by the pre-trained large language model.

[0095] As an optional implementation of the above device, the teaching game making device further includes:

[0096] The second object information acquisition module is used to acquire teaching object information text.

[0097] The second teaching course content output module is used to input teaching object information text, role description text and virtual scene description text into the game agent intelligent body to obtain the teaching course content output by the game agent intelligent body. The game agent intelligent body is constructed by pre-training a large language model.

[0098] As an optional implementation of the above device, the teaching course content includes: mathematical calculation exercises, language and character learning tasks, art creation guidance tutorials and / or hand-made guidance tutorials.

[0099] It should be understood that the device corresponds to the above-mentioned teaching game production method embodiment and can execute the various steps involved in the above-mentioned method embodiment. The specific functions of the device can be referred to in the above description, and the detailed description is appropriately omitted here. The device includes at least one software function module that can be stored in a memory in the form of software or firmware or solidified in the operating system (OS) of the device.

[0100] See also Figure 5 The electronic device 300 provided in the embodiment of the present application includes: a processor 310 and a memory 320, wherein the memory 320 stores machine-readable instructions executable by the processor 310, and when the machine-readable instructions are executed by the processor 310, the above method is executed.

[0101] The embodiments of the present application further provide a computer-readable storage medium 330, on which a computer program is stored. When the computer program is run by the processor 310, the above method is executed. Among them, the computer-readable storage medium 330 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM for short), electrically erasable programmable read-only memory (EEPROM for short), erasable programmable read-only memory (EPROM for short), programmable read-only memory (PROM for short), read-only memory (ROM for short), magnetic memory, flash memory, magnetic disk or optical disk.

[0102] The embodiments of the present application further provide a computer program product, including: a computer program or computer instructions. When the computer program or computer instructions are run by the processor, the above-described method is executed.

[0103] It should be noted that the embodiments in this specification are all described in a progressive manner. Each embodiment focuses on the differences from other embodiments. The same or similar parts among the embodiments can be referred to each other. For device embodiments, since they are basically similar to method embodiments, the description is relatively simple. For the relevant parts, refer to the description of the method embodiments.

[0104] In several embodiments provided by the embodiments of the present application, it should be understood that the disclosed devices and methods can also be implemented in other ways. The device embodiments described above are only illustrative. For example, the flowcharts and block diagrams in the drawings show the possible architectures, functions and operations of devices, methods and computer program products according to multiple embodiments of the present application. In this regard, each block in the flowchart or block diagram can represent a module, a program segment or a part of code. A module, a program segment or a part of code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may also occur in a different order from that marked in the drawings. For example, two consecutive blocks can actually be executed substantially in parallel, and they can sometimes be executed in the reverse order, which mainly depends on the functions involved.

[0105] In addition, the functional modules in each embodiment of the present application can be integrated together to form an independent part, or each module can exist alone, or two or more modules can be integrated to form an independent part. Moreover, in the description of this specification, the descriptions with reference to the terms "one embodiment", "some embodiments", "example", "specific example", "some examples", etc. mean that the specific features, structures, materials or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the embodiments of the present application. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner. In addition, without contradiction, those skilled in the art can combine and combine the different embodiments or examples described in this specification and the features of different embodiments or examples.

[0106] The above description is only an optional implementation manner of the embodiments of the present application, but the protection scope of the embodiments of the present application is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed in the embodiments of the present application, and all of them should be covered within the protection scope of the embodiments of the present application.

Claims

1. A teaching game production method, characterized in that: include: Identify character description text and virtual scene description text from the voice media data collected by the smart terminal, and generate a game scene picture according to the virtual scene description text; Acquire visual media data collected by the intelligent terminal, and generate game character actions according to the visual media data and the character description text; A teaching game is produced based on the game scene pictures and the game character actions.

2. The method according to claim 1, characterized in that The step of generating a game character action according to the visual media data and the character description text comprises: recognizing a human body gesture from the visual media data; Generate a game character image according to the character description text; The game character action is determined according to the human body posture and the game character picture.

3. The method according to claim 2, characterized in that The identifying of a human body posture from the visual media data comprises: Recognizing the human body posture from point cloud data in the visual media data, wherein the point cloud data is acquired by a scanning mechanism of the smart terminal; And / or, the human body posture is identified from multi-view picture data and / or multi-view video data in the visual media data, and the multi-view picture data and / or the multi-view video data are collected by a camera of the smart terminal.

4. The method according to claim 1, characterized in that: After identifying the role description text and the virtual scene description text from the voice media data collected from the smart terminal, the method further includes: Get the teaching object information text; Filling the teaching object information text, the role description text and the virtual scene description text into a preset prompt engineering template to obtain a prompt engineering text; The prompt engineering text is input into a pre-trained large language model to obtain the teaching course content output by the pre-trained large language model.

5. The method according to claim 1, characterized in that After identifying the role description text and the virtual scene description text from the voice media data collected from the smart terminal, the method further includes: Get the teaching object information text; The teaching object information text, the role description text and the virtual scene description text are input into a game agent intelligent body to obtain the teaching course content output by the game agent intelligent body, and the game agent intelligent body is constructed by a pre-trained large language model.

6. The method according to any one of claims 4 or 5, characterized in that: The teaching course content includes: mathematical calculation exercises, language and character learning tasks, art creation guidance tutorials and / or hand-made guidance tutorials.

7. A teaching game making device, characterized in that: include: A game scene generation module, used to identify role description text and virtual scene description text from the voice media data collected by the smart terminal, and generate a game scene picture according to the virtual scene description text; A character action generation module, used to obtain the visual media data collected by the intelligent terminal, and generate game character actions according to the visual media data and the character description text; The teaching game production module is used to produce the teaching game according to the game scene pictures and the game character actions.

8. An electronic device, characterized in that: include: A processor and a memory, wherein the memory stores machine-readable instructions executable by the processor, and the machine-readable instructions are executed by the processor to perform any method according to claims 1 to 6.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 6 is executed.

10. A computer program product, characterized in that include: A computer program or a computer instruction, wherein when the computer program or the computer instruction is executed by a processor, the method according to any one of claims 1 to 6 is executed.

Images