Picture book interaction method for thinking enlightenment and related device

Abstract

The invention discloses a picture book interaction method for thinking enlightenment and a related device, and relates to the technical field of artificial intelligence, and the method comprises the steps: obtaining a target picture book resource which is obtained by picture book materials of all picture book pages, the picture book material is a material which is obtained according to the knowledge point to be enlighted and a preset picture book theme and is suitable for the target user or the user level where the target user is located, playing the target picture book resource, and performing at least one multi-mode interaction with the target user according to the picture book material in the playing process, and the to-be-enlightened knowledge point is taught to the target user and / or the target picture book resource is continuously played through multi-modal interaction. According to the method, the to-be-enlightenment knowledge points are fused into the multi-modal interaction, so that the target user can master the to-be-enlightenment knowledge points while performing the multi-modal interaction with the target picture book resource, the understanding and acceptance capabilities of the target user on the to-be-enlightenment knowledge points are improved, and the thinking enlightenment effect of the target user is further improved.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a picture book interaction method and related devices for thought enlightenment. Background Art

[0002] Cognitive development is a crucial component of children's early learning, gaining increasing attention and recognition from parents. Currently, online approaches to cognitive development primarily include online animated AI (Artificial Intelligence) classes, live animation classes, and dual-teacher video classes. However, these online animation and video classes typically feature pre-produced videos, and interaction with children is limited to simple selections from pre-existing options. This can lead to a lack of interest in viewing the videos, resulting in limited cognitive development. Summary of the Invention

[0003] In view of the above problems, this application provides a picture book interaction method and related devices for enlightenment of thinking, so as to achieve the purpose of enlightenment of thinking of users through diversified interactions with users during picture book playback. The specific scheme is as follows:

[0004] In a first aspect, the present application provides a picture book interaction method for enlightenment of thinking, comprising:

[0005] Obtaining target picture book resources, wherein the target picture book resources are obtained from picture book materials on each picture book page, wherein the picture book materials are obtained based on the knowledge points to be enlightened and the preset picture book themes, and are suitable for the target user or the user level of the target user;

[0006] Play the target picture book resource, and perform at least one multimodal interaction with the target user based on the picture book material during the playback process, so as to impart the knowledge points to be enlightened to the target user through the multimodal interaction and / or continue to play the target picture book resource.

[0007] In a possible implementation, imparting the knowledge point to be enlightened to the target user through the multimodal interaction includes:

[0008] Acquiring first interaction data output by the target user during the multimodal interaction;

[0009] If the first interaction data does not meet the preset interaction data requirements, guidance information related to the knowledge point to be enlightened is output to the target user to guide the target user to output second interaction data that meets the interaction data requirements.

[0010] In a possible implementation, performing at least one multimodal interaction with the target user according to the picture book material during the playback process includes:

[0011] If the picture book material being played is of the first tag type, triggering interaction is performed with the target user, wherein the triggering interaction refers to an interaction method in which the target user interacts with the target picture book resource by touching the terminal device where the target picture book resource is located;

[0012] If the picture book material being played is of the second tag type, performing voice interaction with the target user, wherein the voice interaction refers to an interaction mode in which the target user outputs voice data to interact with the target picture book resource;

[0013] If the picture book material being played is of the third tag type, a game-like interaction is performed with the target user, wherein the game-like interaction refers to an interaction method in which the target user interacts with the target picture book resource by triggering game elements that incorporate the knowledge points to be enlightened.

[0014] In a possible implementation, the target picture book resource is a picture book resource based on a picture book story, and the picture book story includes at least one of a scenario introduction stage, a problem elicitation stage, an independent exploration stage, a step-by-step guidance stage, a scenario escalation stage, a problem closure stage, and a method transfer stage;

[0015] The scenario introduction stage is used to introduce the story synopsis of the picture book story, wherein the story synopsis refers to information including the story overview of the picture book story but not including the ending of the story;

[0016] And / or, the question elicitation stage is used to introduce the story background based on the story summary and elicit a first question related to the knowledge point to be enlightened, wherein the interaction data is required to be a standard answer to the first question;

[0017] And / or, the autonomous exploration phase is used to enable the target user to conduct preliminary exploration of the first question through the game-like interaction;

[0018] And / or, the step-by-step guidance stage is used to output the guidance information to the target user when the second interaction data is not obtained in the preliminary exploration;

[0019] And / or, the scenario upgrade stage is used to, after obtaining the second interaction data based on the guidance information, continue playing the picture book story and raise a second question during the playing process, wherein the second question involves the same knowledge point to be enlightened as the first question but is equal to or more difficult than the first question;

[0020] And / or, the closed-loop question stage is used to raise the first question again based on the second question, and to review and summarize the solution ideas for the first question;

[0021] And / or, the method migration stage is used to enable the target user to answer the second question through the game-like interaction after the review and summary, and after the answer is correct, continue to play the picture book story until the end of the story.

[0022] In a possible implementation, the outputting the guidance information to the target user includes:

[0023] Outputting guidance information to the target user at least once, wherein the amount of guidance information included in the latter guidance information is greater than the amount of guidance information included in the former guidance information.

[0024] In one possible implementation, the picture book material includes audio material and visual material, the audio material includes vocal material for telling the picture book story, the vocal material includes sound material recorded by a real person and / or sound material reproduced based on a real person's voice, and the speaking speed, sentence structure, and language style of the vocal material are all adapted to the target user.

[0025] In a possible implementation, obtaining the target picture book resource includes:

[0026] Obtaining picture book materials for each picture book page;

[0027] The target picture book resource is obtained by configuring the picture book materials of each picture book page on the time track through a pre-built editor, wherein the editor is built using the Tauri framework.

[0028] A second aspect of the present application provides a picture book interactive device for enlightenment of thinking, comprising:

[0029] A picture book resource acquisition module is used to acquire target picture book resources, wherein the target picture book resources are obtained from picture book materials on each picture book page, and the picture book materials are obtained based on the knowledge points to be enlightened and the preset picture book themes, and are suitable for the target user or the user level of the target user;

[0030] The picture book resource playback module is used to play the target picture book resource, and during the playback process, perform at least one multimodal interaction with the target user based on the picture book material, so as to impart the knowledge points to be enlightened to the target user through the multimodal interaction and / or continue to play the target picture book resource.

[0031] A third aspect of the present application provides a computer program product comprising computer-readable instructions. When the computer-readable instructions are executed on an electronic device, the electronic device implements the picture book interaction method for thinking enlightenment according to the first aspect or any implementation of the first aspect.

[0032] A fourth aspect of the present application provides an electronic device, comprising at least one processor and a memory connected to the processor, wherein:

[0033] The memory is used to store computer programs;

[0034] The processor is used to execute the computer program so that the electronic device can implement the picture book interaction method for thinking enlightenment described in the first aspect or any implementation of the first aspect.

[0035] In a fifth aspect, the present application provides a computer storage medium carrying one or more computer programs. When the one or more computer programs are executed by an electronic device, the electronic device can use the picture book interaction method for thinking enlightenment according to the above-mentioned first aspect or any implementation of the first aspect.

[0036] By means of the above technical solution, the picture book interaction method for thinking enlightenment provided by the present application obtains the target picture book resource, and the picture book material in each picture book page contained in the target picture book resource is obtained based on the knowledge point to be enlightened and the preset picture book theme, and is suitable for the target user or the user level of the target user. Since the picture book material is more suitable for the target user, the target user can easily understand the target picture book resource without the assistance of others, thereby improving the target user's reading interest in the target picture book resource, thereby improving the target user's thinking enlightenment effect. Further, in the process of playing the target picture book resource, the present application can perform at least one multimodal interaction with the target user based on the played picture book material, and impart the knowledge point to be enlightened to the target user during the multimodal interaction. Since the knowledge point to be enlightened can be integrated into the multimodal interaction process, the target user can master the knowledge point to be enlightened while immersing in reading the target picture book resource, thereby improving the target user's understanding and acceptance of the enlightenment knowledge point, thereby improving the target user's thinking enlightenment effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] The above and other features, advantages, and aspects of the various embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic and that the originals and elements are not necessarily drawn to scale.

[0038] Figure 1 A schematic diagram of the system architecture provided for this application;

[0039] Figure 2 A flowchart of a picture book interaction method for thought enlightenment provided in this application;

[0040] Figure 3 A schematic diagram of a picture book selection interface provided by this application;

[0041] Figure 4 A schematic diagram of a multimodal interactive interface for imparting ranking knowledge points to target users;

[0042] Figure 5 A schematic diagram of a multimodal interactive interface for continuing to play a target picture book resource;

[0043] Figure 6 A schematic diagram of a voice interaction interface provided by this application;

[0044] Figure 7 (a) is a schematic diagram of a game-like interactive interface for creating a bar chart;

[0045] Figure 7(b) is a schematic diagram of a game-like interactive interface for graphic cutting;

[0046] Figure 7 (c) is a schematic diagram of a game-like interactive interface with overlapping and dragging graphics;

[0047] Figure 8 A schematic diagram of an editing interface of an editor provided in this application;

[0048] Figure 9 A schematic diagram of the structure of a picture book interactive device for thought enlightenment provided in this application;

[0049] Figure 10 This is a schematic diagram of the structure of an electronic device provided in this application. DETAILED DESCRIPTION

[0050] The following describes the embodiments of the present application in conjunction with the accompanying drawings. The terms used in the implementation methods of the present application are only used to explain the specific embodiments of the present application and are not intended to limit the present application.

[0051] The embodiments of the present application are described below in conjunction with the accompanying drawings. Those skilled in the art will appreciate that, with the development of technology and the emergence of new scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0052] The terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequential order. It should be understood that the terms used in this way can be interchangeable under appropriate circumstances, and this is merely a way of distinguishing the objects of the same attributes when describing them in the embodiments of the present application. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, so that the process, method, system, product or equipment comprising a series of units need not be limited to those units, but may include other units that are not clearly listed or inherent to these processes, methods, products or equipment.

[0053] This application provides a picture book interaction method for thinking enlightenment, which can be applied to scenarios where thinking enlightenment is required for target users (such as children, especially young children).

[0054] Optionally, the picture book interaction method for thinking enlightenment provided by this application can be applied to Figure 1 The system architecture shown in FIG. 1 may include a terminal 100 and a server 200. The server 200 may include one or more servers ( Figure 1 (This section includes a server as an example).

[0055] The terminal 100 can be used alone to execute the picture book interaction method for thinking enlightenment provided in the embodiment of the present application. In addition, the terminal 100 and the server 200 can also be used in conjunction to execute the picture book interaction method for thinking enlightenment provided in the embodiment of the present application.

[0056] Next describe Figure 1 The product form of the mid-terminal 100;

[0057] The terminal 100 in the embodiment of the present application can be a mobile phone, a tablet computer, a wearable device, a vehicle-mounted device, an augmented reality (AR) / virtual reality (VR) device, a laptop computer, an ultra-mobile personal computer (UMPC), a netbook, a personal digital assistant (PDA), etc., and the embodiment of the present application does not impose any restrictions on this.

[0058] The terminal 100 may include components such as a radio frequency unit, a memory, an input unit, a display unit, a camera (optional), an audio circuit (optional), a speaker (optional), a microphone (optional), a headphone jack (optional), a processor, an external interface, and a power supply. Those skilled in the art will appreciate that the aforementioned components are merely examples and do not limit the terminal or multi-function device. The terminal or multi-function device may include more or fewer components, or may combine certain components or use different components.

[0059] The input unit can be used to receive input digital or character information and generate key signal input related to user settings and function control of the portable multi-function device. Specifically, the input unit may include a touch screen (optional) and / or other input devices. Specifically, other input devices may include, but are not limited to, one or more of a physical keyboard, function keys (such as volume control keys, power keys, etc.), a trackball, a mouse, a joystick, etc.

[0060] Among them, the input device can receive input data and so on.

[0061] The display unit can be used to display information input by the user or information provided to the user, various menus of the terminal, interactive interfaces, file display, and / or playback of any multimedia file. In an embodiment of the present application, the display unit can be used to display various interactive interfaces, processing results, etc. in the picture book interactive method for mental enlightenment.

[0062] The memory can be used to store software codes related to the picture book interaction method for thinking enlightenment, the processor can execute the steps of the picture book interaction method for thinking enlightenment, and can also schedule other units (such as the above-mentioned input unit and display unit) to realize corresponding functions.

[0063] The radio frequency unit (optional) can be used to send and receive information or receive and send signals during a call.

[0064] In this embodiment of the present application, the radio frequency unit can send data to the server 200 and receive the processing results sent by the server 200.

[0065] It should be understood that the radio frequency unit is optional and can be replaced by other communication interfaces, such as a network port.

[0066] The terminal 100 further includes a power source (such as a battery) for supplying power to various components.

[0067] The terminal 100 also includes an external interface, which may be a standard Micro USB interface or a multi-pin connector, and may be used to connect the terminal 100 to other devices for communication, or to connect a charger to charge the terminal 100.

[0068] The server 200 includes a bus, a processor, a communication interface and a memory. The processor, the memory and the communication interface communicate with each other via the bus.

[0069] Among them, the memory can be used to store software codes related to the picture book interaction method for thinking enlightenment, the processor can execute the steps of the picture book interaction method for thinking enlightenment of the chip, and can also schedule other units to realize corresponding functions.

[0070] It should be noted that the picture book interaction method for thinking enlightenment provided in this application is applied to Figure 1 The system architecture shown is only an example. In addition, it can also be applied to other systems, such as the terminal 100 or the server 200, etc. This application does not make any specific limitations.

[0071] In order to enable those skilled in the art to better understand the present application, the picture book interaction method for thinking enlightenment according to an embodiment of the present application is described in detail below with reference to the accompanying drawings.

[0072] Reference Figure 2 , which is a flow chart of a picture book interaction method for thinking enlightenment provided by an embodiment of the present application, such as Figure 2 As shown, the method may include:

[0073] Step S201: Acquire target picture book resources.

[0074] In this embodiment, the target picture book resource is obtained from the picture book materials of each picture book page, and the picture book materials are obtained based on the knowledge points to be enlightened and the preset picture book theme. Here, the knowledge points to be enlightened refer to the knowledge points to be enlightened in thinking, such as color recognition knowledge points, sorting knowledge points, classification knowledge points, or size knowledge points, etc.

[0075] In one possible implementation, the picture book materials can be materials that are suitable for the user level of the target user. Specifically, this embodiment can divide the users to be enlightened into multiple user levels. For example, they can be divided into three user levels: low, medium, and high according to age. Then, for each user level, the knowledge points to be enlightened and the picture book themes that match the user level are determined. Based on the knowledge points to be enlightened and the picture book themes, the picture book materials suitable for the user level are determined. For example, children aged 0-2 are at the low level. If the knowledge point to be enlightened is color cognition and the picture book theme is "the color of fruit", the picture book materials can be fruits and other materials. Children aged 3-6 are at the intermediate level. If the knowledge point to be enlightened is sorting knowledge points and the picture book theme is "new members of the band", the picture book materials can be various musical instruments and other materials.

[0076] In another possible implementation, the picture book material can be material suitable for the target user. Specifically, this embodiment can collect user portrait data of the target user, and then determine, for the target user, the knowledge points to be enlightened and the picture book themes that match the user portrait data of the target user, and determine the picture book material suitable for the target user based on the knowledge points to be enlightened, the picture book themes and the user portrait data of the target user.

[0077] Optionally, the process of acquiring the target picture book resource in this embodiment may include: responding to a click operation of the target user on the target picture book resource, and acquiring the target picture book resource.

[0078] For example, see Figure 3 As shown in the schematic diagram of the picture book selection interface, this embodiment can display multiple pre-generated picture books on the picture book selection interface. Suppose the target user clicks Figure 3 In the picture book selection interface shown, for example, “new band member” is selected, this embodiment can obtain the target picture book resource corresponding to “new band member”.

[0079] Optionally, in order to prevent the target user from clicking on an inappropriate target picture book resource, the present embodiment can classify the pre-generated multiple picture books according to levels (optionally, the level can be the user level, or the difficulty level set under the user level, etc.), so that before the picture book selection interface is displayed to the target user, the level selection interface is first displayed to the target user, and the level suitable for the target user is unlocked in the level selection interface, and the level unsuitable for the target user is locked. Thus, the target user can first select the appropriate level in the level selection interface, and then enter Figure 3 The picture book selection interface shown.

[0080] It should be noted that the above-mentioned level selection interface and picture book selection interface are only examples. In addition, there may be other display forms, which are not specifically limited in this application.

[0081] Step S202: Play the target picture book resource, and perform at least one multimodal interaction with the target user according to the picture book material during the playback process, so as to impart the target user with the knowledge points to be enlightened and / or continue to play the target picture book resource through the multimodal interaction.

[0082] Multimodal interaction refers to the input and output of data information through multiple sensory modalities (such as voice, vision, touch, gestures, facial expressions, brain waves, etc.). In this embodiment, different multimodal interaction modes can be set for some or all picture book materials in the target picture book resource, so that when the corresponding picture book material is played, the corresponding multimodal interaction can be carried out with the target user according to the preset multimodal interaction mode.

[0083] Here, the purpose of multimodal interaction includes: imparting the knowledge points to be enlightened to the target users, and / or continuing to play the target picture book resources.

[0084] For example, see Figure 4 The schematic diagram of a multimodal interactive interface for imparting sorting knowledge points to target users is shown. The target user can drag Figure 4 The three instruments in the (trombone, trumpet, and French horn) are sorted multiple times to obtain multiple sorting results; see Figure 5 The schematic diagram of a multimodal interactive interface for continuing to play the target picture book resource is shown. The target user can click Figure 5 to play the piano to continue playing the target picture book resource "New Members in the Band". If the target user does not click, it will stay on Figure 5 The picture book page shown does not continue to play.

[0085] In summary, the picture book interaction method for thinking enlightenment provided by this application obtains the target picture book resource. The picture book material in each picture book page contained in the target picture book resource is obtained based on the knowledge points to be enlightened and the preset picture book theme, and is suitable for the target user or the user level of the target user. Since the picture book material is more suitable for the target user, the target user can easily understand the target picture book resource without the assistance of others, thereby improving the target user's reading interest in the target picture book resource, thereby improving the target user's thinking enlightenment effect. Further, in the process of playing the target picture book resource, the present application can perform at least one multimodal interaction with the target user based on the played picture book material, and impart the knowledge points to be enlightened to the target user during the multimodal interaction. Since the knowledge points to be enlightened can be integrated into the multimodal interaction process, the target user can master the knowledge points to be enlightened while immersing in reading the target picture book resource, thereby improving the target user's understanding and acceptance of the enlightenment knowledge points, thereby improving the target user's thinking enlightenment effect.

[0086] In some embodiments of the present application, the process of "teaching the target user the knowledge points to be enlightened through multimodal interaction" in the above step S202 is introduced.

[0087] In this embodiment, when the target user is taught the knowledge points to be enlightened through multimodal interaction, the first interaction data output by the target user during the multimodal interaction process can be obtained. If the first interaction data does not meet the preset interaction data requirements, guidance information related to the knowledge points to be enlightened is output to the target user to guide the target user to output second interaction data that meets the interaction data requirements.

[0088] That is, when conducting multimodal interaction with the target user, this embodiment can determine whether guidance is needed for the target user based on the first interaction data output by the target user during the multimodal interaction. If the first interaction data does not meet the preset interaction data requirements, it means that the target user has not yet mastered the knowledge points to be enlightened. In this case, guidance information related to the knowledge points to be enlightened is output to the target user, so that the target user can gradually master the knowledge points to be enlightened under the gradual guidance of the guidance information, and output the second interaction data that meets the interaction data requirements based on the knowledge points to be enlightened that have been mastered.

[0089] Optionally, the process of "outputting guidance information related to the knowledge point to be enlightened to the target user" can be: displaying the guidance information to the target user in the form of animation effects, and / or playing the guidance information to the target user in the form of voice.

[0090] For example, see Figure 4 Assuming that the target user can only sort out part of the performance order (the first interaction data is the part of the performance order sorted by the target user, and the preset interaction data requirement is all performance orders), then guidance information related to the sorting knowledge points can be output to the target user in the form of voice combined with animation effects. For example, the guidance information is "(guidance voice) We haven't found all the performance orders yet! If you want to find all the arrangements, you must be orderly. We can fix the first instrument to be played in order (the corresponding animation effect is: there is a highlighted box in the first position, flashing twice, and then disappearing), and then arrange the instruments to be played later. For example, let the trombone play first, and then arrange the order of the trumpet and French horn. How many orders are there (the corresponding animation effect is: the trombone is in the first position, highlighted, and then the trumpet and French horn exchange positions)? Then who can play first? I'll leave it to you to try." Through the above guidance voice and animation effects, the target user can be guided to sort out all the performance orders (i.e., the second interaction data) to meet the interaction data requirements.

[0091] It should be noted that the above-mentioned guidance information is displayed in the form of animation effects and voice only as an example. In addition, there may be other implementation methods, for example, displaying guidance information to target users in the form of knowledge cards, etc. This application does not make specific limitations.

[0092] This embodiment can determine whether the target user has mastered the knowledge points to be enlightened through the interaction data output by the target user during the multimodal interaction process, and when the target user has not mastered the knowledge points to be enlightened, the target user can be reminded and guided in a timely manner through a variety of guidance methods such as animation effects and voice. The diversified guidance methods can better attract the interest of the target user, so that the target user can understand and master the knowledge points to be enlightened more quickly, further improving the thinking enlightenment effect.

[0093] In order to enable those skilled in the art to better understand the present application, the process of step S202 of "performing at least one multimodal interaction with the target user according to the picture book material during the playback process" is introduced below.

[0094] As mentioned above, different multimodal interaction modes can be set for some or all of the picture book materials in the target picture book resources, so that when the corresponding picture book materials are played, corresponding multimodal interactions can be performed with the target user according to the preset multimodal interaction mode.

[0095] In one possible implementation, tag types may be preconfigured for some or all picture book materials in the target picture book resource, where different tag types correspond to different multimodal interaction methods. Thus, this embodiment may perform multimodal interaction with the target user based on the tag type of the picture book material being played.

[0096] For example, the first tag type corresponds to trigger-based interaction, the second tag type corresponds to voice interaction, and the third tag type corresponds to game-based interaction. Therefore, if the picture book material played has the first tag type, a trigger-based interaction will be performed with the target user; if the picture book material played has the second tag type, a voice interaction will be performed with the target user; and if the picture book material played has the third tag type, a game-based interaction will be performed with the target user.

[0097] The trigger interaction mentioned above refers to the interaction mode in which the target user interacts with the target picture book resource by touching the terminal device where the target picture book resource is located. For example, see Figure 5 The target user can click on the corresponding picture book material on the touch screen of the terminal device to interact with the target picture book resource. Of course, the target user can also click on the physical button or virtual button connected to the terminal device to control the position of the touch point (such as the mouse cursor) to click on the corresponding picture book material, thereby achieving the purpose of interacting with the target picture book resource.

[0098] The above-mentioned voice interaction refers to the interaction mode in which the target user outputs voice data to interact with the target picture book resource. For example, see Figure 6 , when it appears Figure 6 After the microphone symbol is displayed, the target user can output voice data. In this embodiment, a voice collection device can be called to collect the voice data output by the target user to achieve the purpose of interacting with the target picture book resource according to the voice data output by the target user.

[0099] The above-mentioned game-like interaction refers to the interaction mode in which the target users interact with the target picture book resources by triggering game elements that incorporate the knowledge points to be enlightened. Figure 4 ,The game elements are three musical instruments and three boxes. Target users can interact with the target picture book resources by dragging the three musical instruments to the three boxes.

[0100] It should be noted that in game-like interaction, dragging is only one implementation method, and there may be other implementation methods, such as voice interaction, etc., which are not specifically limited in this application.

[0101] It should also be noted that the above three multimodal interaction methods are only examples and are not intended to limit the present application. In addition, other multimodal interaction methods may be provided, such as somatosensory interaction, etc.

[0102] The embodiment of the present application can more effectively attract the target user's interest in reading the target picture book resource through multimodal interaction, so that the target user can master the knowledge points to be enlightened that the target picture book resource intends to teach to the user during immersive reading, thereby improving the effect of thinking enlightenment.

[0103] The following embodiment provides a detailed introduction to the target picture book resources.

[0104] In one possible implementation, the target picture book resource is a picture book resource based on a picture book story, that is, this embodiment can integrate the knowledge points to be enlightened into the picture book story, and conduct at least one multimodal interaction with the target user in the process of telling the picture book story, such as trigger interaction, voice interaction, game-style interaction, etc., to impart the knowledge points to be enlightened to the target user through multimodal interaction.

[0105] Optionally, the picture book stories in the target picture book resources include one or more of the following stages: scenario introduction stage, problem elicitation stage, independent exploration stage, step-by-step guidance stage, scenario escalation stage, problem closure stage, and method transfer stage.

[0106] Optionally, during the scenario introduction stage, a story synopsis of the picture book story can be introduced. Here, the story synopsis refers to information that includes an overview of the picture book story but does not include the ending of the story. Through the story synopsis, the target user can roughly understand the content of the story told by the current picture book story. Since the story synopsis does not include the ending of the story, suspense can be retained, quickly attracting the target user's interest in continuing to read.

[0107] In order to reduce the unfamiliarity and difficulty of target users in learning new knowledge, the picture book story can optionally use daily life scenes familiar to the target users as the background of the story summary, so that this embodiment can integrate the knowledge points to be enlightened into the daily life scenes familiar to the target users, so that they can naturally come into contact with the knowledge points to be enlightened, thereby improving the target users' acceptance of new knowledge.

[0108] Optionally, during the question-elicitation phase, you can introduce the story context based on the story summary and introduce a first question related to the knowledge point to be introduced, guiding the target user to focus on specific key knowledge points and activating their critical thinking. The interaction data mentioned above is required to be the standard answer to the first question.

[0109] Optionally, the question elicitation stage can be combined with multimodal interaction methods, such as combining AI (Artificial Intelligence) voice Q&A in the question elicitation stage to guide target users to participate more actively and solve the first problem.

[0110] Optionally, during the autonomous exploration stage, this embodiment may incorporate a game-like interaction method, allowing the target user to conduct a preliminary exploration of the first problem through game-like interaction. Even if the target user boldly attempts to solve the problem based on existing experience and preliminary cognition, he or she may conduct repeated trial and error during the period and independently construct a thinking path.

[0111] In order to improve the initial exploration enthusiasm of the target users, this embodiment can provide a variety of (for example, more than 30) interesting game interactions, including but not limited to the schematic diagram of the game-like interactive interface for making a bar chart as shown in Figure 7 (a), the schematic diagram of the game-like interactive interface for graphic cutting as shown in Figure 7 (b), the schematic diagram of the game-like interactive interface for graphic overlapping and dragging as shown in Figure 7 (c), etc., so that the target users can learn and master the knowledge points to be enlightened during the game.

[0112] Optionally, in the step-by-step guidance stage, this embodiment may output guidance information to the target user when no second interaction data is obtained in the preliminary exploration.

[0113] Optionally, the process of "outputting guidance information to the target user" may include: outputting guidance information to the target user at least once, wherein the amount of guidance information contained in the latter guidance information is greater than the amount of guidance information contained in the previous guidance information, that is, this embodiment can help the target user to clarify his ideas and improve his thinking process through multiple guidances with gradually increasing amounts of guidance information.

[0114] Optionally, in the scenario upgrade stage, this embodiment can continue to play the picture book story and raise a second question during the playback process after obtaining the second interactive data based on the guidance information. Here, the second question involves the same knowledge points to be enlightened as the first question, but the difficulty is equal to or higher than the first question.

[0115] Optionally, in the closed-loop problem stage, this embodiment can again raise the first question based on the second question, and review and summarize the solution ideas for the first question.

[0116] By reviewing and summarizing, the target users can further deepen their impression of the enlightenment knowledge points, so that they can better master the enlightenment knowledge points.

[0117] Optionally, in the method migration stage, this embodiment can allow the target user to answer the second question through game-like interaction after reviewing and summarizing, and continue to play the picture book story until the end after the answer is correct.

[0118] In order to help those skilled in the art better understand the above stages, an exemplary explanation is given below using a picture book story with the theme of "New Member in the Band".

[0119] Scenario introduction phase: A new member, the trumpet, joins the band. Each instrument enters in a specific order. Let's see how the band performs. The target user can trigger interactions with the band members (e.g., click on them). By listening to the music played by the band members, they can tell that the trumpet's performance always goes awry, which naturally leads to the mathematical problem of permutation.

[0120] Problem posing stage: The band is recruiting new members, and there are many musical instruments that have signed up. Through AI voice interaction (for example, the target user inputs the name of the instrument that has signed up: trumpet, trombone and French horn), it is learned that three instruments, trumpet, trombone and French horn, have entered the interview. At this time, a problem arises: the interviews have a sequence, how should they be arranged?

[0121] Autonomous exploration stage: Target users complete preliminary exploration based on their existing life experience through game-like interactions (such as drag-and-drop interactions) and try to solve the above arrangement problems.

[0122] Step-by-step guidance stage: Provide guidance information to target users in a timely and accurate manner based on their responses.

[0123] If the target user hasn't found all possible permutations after the initial exploration, guidance message 1 will appear: (Guidance Voice) We haven't found all possible permutations yet! To find all possible permutations, they must be organized. We can fix the first instrument to play (the corresponding animation effect is: a highlighted box in the first position flashes twice, then disappears), and then arrange the subsequent instruments. For example, how many possible permutations are there for the trombone to play first, and then the trumpet and French horn (the corresponding animation effect is: the trombone is first, highlighted, and then the trumpet and French horn swap positions)? Then who can play first? Let you try.

[0124] After closing guidance message 1, the target user can try to solve the problem again. If all possible arrangements haven't been found, guidance message 2 will appear: (Guidance Voice) We can arrange the order from left to right, fixing the first instrument to play. Let's have the trombone play first, then arrange the trumpet and French horn in order, with the trumpet playing first and the French horn last (the corresponding animation effect is: the trumpet and French horn swap positions, and the trombone, French horn, trumpet form a group and fly into the display area in that order). These are two possible arrangements. Then, let the trumpet play first in order (the corresponding animation effect is: the trumpet switches to the first position and becomes highlighted). Then arrange the order of the trombone and French horn, which also has two possible arrangements (the corresponding animation effect is: the trumpet, trombone, French horn form a group in that order, and then the trumpet, French horn, trombone form a group in that order, and then the trumpet, French horn, trombone form a group in that order, and fly into the display area in that order). Similarly, let's arrange the French horn first (animation: the French horn moves to first position and becomes highlighted). There are also two possible arrangements (animation: the French horn, trombone, and trumpet form a group, followed by the French horn, trumpet, and trombone, each group flying into the display area below). This completes the complete playing order, for a total of six possible arrangements.

[0125] And so on, until the target user arranges all the performance sequences.

[0126] It should be noted that each of the above-mentioned guiding information can be reused for viewing, so as to facilitate faster sorting by the target user.

[0127] Scenario escalation stage: The trumpeter's interview was very successful and he joined the band. The band began to arrange and prepare for the next concert. The target user can hear the band's music through triggered interactions (such as click interactions), and then escalate the problem: the trumpeter was the third to enter the song, but was very nervous and would rush the beat. After many practices, he finally found the beat. But how should the other instruments be arranged when the trumpet enters the song third?

[0128] Problem closure stage: Through AI voice interaction (e.g., by asking the question, "Do you remember how you found the complete performance order for the three instruments?", and the target user provides a voice answer), the entire thinking process is comprehensively reviewed, scattered knowledge points to be enlightened and answering experience are systematized and structured, good mathematical thinking habits are cultivated, and the mastery of knowledge points to be enlightened is strengthened in guided question-and-answer sessions.

[0129] Method transfer phase: The trumpet is the third of four instruments. Using game-like interactions (such as drag-and-drop), players apply their knowledge to solve the arrangement of other instruments. At the end of the story, they reveal how the other instruments help the trumpet relax.

[0130] It should be noted that the specific contents of the above-mentioned scenario introduction stage, problem elicitation stage, independent exploration stage, step-by-step guidance stage, scenario escalation stage, problem closure stage, and method transfer stage are only examples. In addition, each stage may also have other contents, which are not specifically limited by this application. For example, the scenario escalation stage may also introduce a second knowledge point to be enlightened during the playback of the picture book story, etc.

[0131] This embodiment provides a targeted picture book resource based on picture book stories. Through the format of picture book stories, young target users can be more proactive in learning enlightenment knowledge points, achieving better enlightenment effects. The themes of the picture book stories are realistic and closely related to the knowledge points to be enlightened, allowing for learning while playing games. This promotes a better transition from figurative thinking to abstract thinking for target users, further enhancing the enlightenment effect.

[0132] In some other embodiments of the present application, the process of the aforementioned step S201 “obtaining target picture book resources” is introduced.

[0133] In this embodiment, the picture book materials of each picture book page can be obtained according to the above process.

[0134] Optionally, picture book materials may include audio materials and visual materials.

[0135] Optionally, visual materials may include: picture materials, video materials and animation effect materials.

[0136] Optionally, the audio materials may include: vocal materials and other sound materials used to tell the story of the picture book (such as the playing sounds of various instruments such as trumpets, French horns, and trombones).

[0137] In an optional embodiment, the vocal material includes real-person recorded sound material and / or sound material reproduced from real-person speech. For example, the sound material reproduced from real-person speech can be obtained by processing text material using speech synthesis technology to produce sound material consistent with real-person speech. Optionally, the speech synthesis technology can be a large speech synthesis model, such as a TTS (Text-to-Speech) model, which is obtained by fine-tuning a pre-trained large model using training data consisting of text material and the corresponding real-person recorded sound material.

[0138] Optionally, the speech speed, sentence structure, and language style of the above-mentioned vocal materials are adapted to the target user. For example, taking language style as an example, if the target user is a young child, the vocal material can be a sound material recorded by an experienced kindergarten teacher, or a sound material reproduced based on the voice of the kindergarten teacher; taking speech speed as an example, the speech speed of the vocal material is controlled at about 100-180 words per minute based on the user level of the target user, and the speech speed gradually increases as the user level upgrades; taking sentence structure as an example, the higher the user level of the target user, the more complex and longer the sentences.

[0139] This embodiment uses experienced preschool teachers to perform large-scale voice recording, which makes the experience more immersive, more emotional, and more able to capture the attention of target users. The speech speed and sentence structure are adapted to the target users, and the problem of target users not wanting to read because the material is too simple or too difficult will not occur, thereby improving the reading experience of the target users.

[0140] Furthermore, in this embodiment, the picture book materials of each picture book page can be configured on the time track through a pre-built editor to obtain the target picture book resources.

[0141] Optionally, the above-mentioned editor can be an editor built using the Tauri framework. This editor allows people without picture book production experience to independently produce picture book content by disassembling the data structure of editable elements (such as the position in the picture book, picture size, transparency, animation and other attributes), and use the time track to drag and drop multiple nodes such as pictures, animations, bubble windows and other related to the picture book content, and trigger node configuration, forming a complete and rich picture book resource, reducing the cost and cycle of picture book production.

[0142] Optionally, the editing backend corresponding to the above-mentioned editor can adopt an asynchronous communication protocol and build a Cocos preview environment to achieve two-way instant synchronization of canvas operations and property panel modifications, so that when people are making picture book content, they can view the overall effect of the picture book in real time, thereby improving production efficiency.

[0143] For example, see Figure 8 , which is a schematic diagram of the editing interface of an editor provided in this application. Figure 8 The leftmost side shows the various picture book pages, and the picture book material display area is used to display a picture book page (such as Figure 8 For example, in the picture book material on page 3, the property setting area of the picture book material is used to set the properties of each picture book material, such as setting the execution time of the picture book material, whether to display it, etc. The editing effect display area is used to visually display the editing effect of each picture book material, that is, the effect set in the property setting area of the picture book material can be displayed in real time in the editing effect display area. Figure 8The editing interface also provides a time track, on which various time nodes are set. The audio track editing area and the event track editing area are used to configure the appearance time of audio materials and visual materials on the picture book page respectively.

[0144] Optionally, after editing is completed, the above editor can support one-click saving or exporting of production resources, greatly reducing the cost of repeated development.

[0145] Optionally, the above editor can also have a built-in resource verification mechanism to automatically intercept picture book materials with oversized sizes or abnormal formats to ensure that the editor does not use picture book materials with oversized sizes or abnormal formats to create picture book resources, thereby improving the quality of picture book resources.

[0146] In summary, the target picture book resources provided in this embodiment can provide personalized teaching to target users, allowing them to receive sufficient guidance at their existing level and thus master the knowledge points to be enlightened more quickly. At the same time, this embodiment builds a Tauri-based front-end editor and a Cocos-based editing backend, which can achieve cross-platform dynamic picture book production and shorten the dynamic picture book development cycle from the traditional 10 days to 2 days per picture book resource, greatly saving picture book production time and reducing picture book production costs.

[0147] The above describes a picture book interaction method for thinking enlightenment provided by an embodiment of the present application. The following describes a device for executing the above picture book interaction method for thinking enlightenment.

[0148] See also Figure 9 , Figure 9 This is a structural diagram of a picture book interactive device for thinking enlightenment provided in an embodiment of the present application. Figure 9 As shown, the picture book interactive device for thinking enlightenment may include:

[0149] The picture book resource acquisition module 901 is used to acquire target picture book resources, wherein the target picture book resources are obtained from picture book materials on each picture book page. The picture book materials are obtained based on the knowledge points to be enlightened and the preset picture book themes, and are suitable for the target user or the user level of the target user;

[0150] The picture book resource playing module 902 is used to play the target picture book resource and perform at least one multimodal interaction with the target user based on the picture book material during the playing process, so as to impart the target user with the knowledge points to be enlightened and / or continue to play the target picture book resource through the multimodal interaction.

[0151] In one possible implementation, the target user is taught the knowledge points to be enlightened through multimodal interaction, including:

[0152] Acquire first interaction data output by the target user during the multimodal interaction;

[0153] If the first interaction data does not meet the preset interaction data requirements, guidance information related to the knowledge point to be enlightened is output to the target user to guide the target user to output second interaction data that meets the interaction data requirements.

[0154] In a possible implementation, when the picture book resource playback module performs at least one multimodal interaction with the target user based on the picture book material during playback, it can be specifically used to:

[0155] If the picture book material being played is of the first tag type, a trigger interaction is performed with the target user, wherein the trigger interaction refers to an interaction method in which the target user interacts with the target picture book resource by touching the terminal device where the target picture book resource is located;

[0156] If the picture book material being played is of the second tag type, voice interaction is performed with the target user, wherein voice interaction refers to an interaction method in which the target user outputs voice data to interact with the target picture book resource;

[0157] If the picture book material played is of the third label type, a game-like interaction is performed with the target user, wherein the game-like interaction refers to an interaction method in which the target user interacts with the target picture book resource by triggering game elements that incorporate the knowledge points to be enlightened.

[0158] In a possible implementation, the target picture book resource is a picture book resource based on a picture book story, and the picture book story includes at least one of a scenario introduction stage, a problem elicitation stage, an independent exploration stage, a step-by-step guidance stage, a scenario escalation stage, a problem closure stage, and a method transfer stage;

[0159] The scenario introduction stage is used to introduce the story synopsis of the picture book story, where the story synopsis refers to information that includes the story overview of the picture book story but does not include the ending of the story;

[0160] And / or, the question elicitation stage is used to introduce the story background based on the story summary and elicit a first question related to the knowledge point to be enlightened, wherein the interactive data is required to be a standard answer to the first question;

[0161] and / or, the autonomous exploration phase is used to enable the target user to conduct preliminary exploration of the first question through game-like interaction;

[0162] and / or, the step-by-step guidance phase is used to output guidance information to the target user when no second interaction data is obtained in the preliminary exploration;

[0163] And / or, the scenario upgrade stage is used to, after obtaining the second interaction data based on the guidance information, continue playing the picture book story and raise a second question during the playing process, wherein the second question involves the same knowledge point to be enlightened as the first question but is equal to or more difficult than the first question;

[0164] And / or, the closed-loop question phase is used to raise the first question again based on the second question, and to review and summarize the solution ideas for the first question;

[0165] And / or, the method transfer stage is used to enable the target user to answer the second question through game-like interaction after reviewing and summarizing, and after answering correctly, continue to play the picture book story until the end of the story.

[0166] In a possible implementation, when the picture book resource playback module outputs guidance information to the target user, it can be specifically used to:

[0167] The guidance information is output to the target user at least once, wherein the amount of guidance information included in the latter guidance information is greater than the amount of guidance information included in the former guidance information.

[0168] In one possible implementation, the picture book materials in the above-mentioned picture book resource acquisition module include audio materials and visual materials. The audio materials include vocal materials used to tell the picture book story. The vocal materials include sound materials recorded by real people and / or sound materials reproduced based on real people's voices. The speaking speed, sentence structure, and language style of the vocal materials are all adapted to the target users.

[0169] In a possible implementation, when acquiring the target picture book resource, the picture book resource acquisition module may be used to:

[0170] Get picture book materials for each picture book page;

[0171] The picture book materials of each picture book page are configured on the time track through a pre-built editor to obtain the target picture book resources. The editor is built using the Tauri framework.

[0172] An embodiment of the present application further provides an electronic device, including at least one processor and a memory connected to the processor, wherein:

[0173] Memory is used to store computer programs;

[0174] The processor is used to execute the computer program so that the electronic device can implement the various steps of the above-mentioned picture book interaction method for thinking enlightenment.

[0175] refer to Figure 10, which shows a schematic diagram of the structure of an electronic device suitable for implementing the embodiments of the present application. The electronic device in the embodiments of the present application may include, but is not limited to, fixed terminals such as mobile phones, laptops, PDAs (personal digital assistants), PADs (tablet computers), desktop computers, etc. Figure 10 The electronic device shown is merely an example and should not limit the functions and scope of use of the embodiments of the present application.

[0176] like Figure 10 As shown, the electronic device may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 11, which can perform various appropriate actions and processes based on programs stored in a read-only memory (ROM) 12 or programs loaded from a storage device 18 into a random access memory (RAM) 13. When the electronic device is powered on, RAM 13 also stores various programs and data required for the operation of the electronic device. Processing device 11, ROM 12, and RAM 13 are interconnected via a bus 14. An input / output (I / O) interface 15 is also connected to bus 14.

[0177] Typically, the following devices may be connected to the I / O interface 15: an input device 16 including, for example, a touch screen, a touchpad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 17 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 18 including, for example, a memory card, a hard disk, etc.; and a communication device 19. The communication device 19 may allow the electronic device to communicate with other devices wirelessly or by wire to exchange data. Figure 10 The electronic device is shown with various devices, but it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed instead.

[0178] An embodiment of the present application also provides a computer program product including computer-readable instructions. When the computer-readable instructions are executed on an electronic device, the electronic device implements any one of the picture book interaction methods for thinking enlightenment provided in the embodiments of the present application.

[0179] A computer-readable storage medium is also provided in an embodiment of the present application. The storage medium carries one or more computer programs. When the one or more computer programs are executed by an electronic device, the electronic device can implement any one of the picture book interaction methods for thinking enlightenment provided in the embodiment of the present application.

[0180] It should also be noted that the device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separate, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed across multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the present embodiment. In addition, in the drawings of the device embodiments provided in this application, the connection relationship between the modules indicates that there is a communication connection between them, which can be specifically implemented as one or more communication buses or signal lines.

[0181] Through the description of the above embodiments, those skilled in the art can clearly understand that the present application can be implemented by means of software plus necessary general hardware, and of course can also be implemented by special hardware including application-specific integrated circuits, special CPUs, special memories, special components, etc. In general, all functions performed by computer programs can be easily implemented with corresponding hardware, and the specific hardware structures used to implement the same function can also be diverse, such as analog circuits, digital circuits or special circuits, etc. However, for the present application, software program implementation is a better implementation method in most cases. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a readable storage medium, such as a computer's floppy disk, USB flash drive, mobile hard disk, ROM, RAM, magnetic disk or optical disk, etc., and includes a number of instructions to enable a computer device (which can be a personal computer, training equipment, or network equipment, etc.) to execute the methods described in each embodiment of the present application.

[0182] In the above embodiments, all or part of the embodiments may be implemented by software, hardware, firmware, or any combination thereof. When implemented by software, all or part of the embodiments may be implemented in the form of a computer program product.

[0183] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website, a computer, a training device or a data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website, a computer, a training device or a data center. The computer-readable storage medium can be any available medium that a computer can store or a data storage device such as a training device, a data center, etc. that includes one or more available media integrations. The available medium can be a magnetic medium, (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive (SSD)).

Claims

1. A picture book interaction method for enlightenment of thinking, characterized in that: include: Obtaining target picture book resources, wherein the target picture book resources are obtained from picture book materials on each picture book page, wherein the picture book materials are obtained based on the knowledge points to be enlightened and the preset picture book themes, and are suitable for the target user or the user level of the target user; Play the target picture book resource, and perform at least one multimodal interaction with the target user based on the picture book material during the playback process, so as to impart the knowledge points to be enlightened to the target user through the multimodal interaction and / or continue to play the target picture book resource.

2. The picture book interaction method according to claim 1, characterized in that: The imparting of the knowledge points to be enlightened to the target user through the multimodal interaction includes: Acquiring first interaction data output by the target user during the multimodal interaction; If the first interaction data does not meet the preset interaction data requirements, guidance information related to the knowledge point to be enlightened is output to the target user to guide the target user to output second interaction data that meets the interaction data requirements.

3. The picture book interaction method for thinking enlightenment according to claim 2 is characterized in that: The performing at least one multimodal interaction with the target user according to the picture book material during the playback process includes: If the picture book material being played is of the first tag type, triggering interaction is performed with the target user, wherein the triggering interaction refers to an interaction method in which the target user interacts with the target picture book resource by touching the terminal device where the target picture book resource is located; If the picture book material being played is of the second tag type, performing voice interaction with the target user, wherein the voice interaction refers to an interaction mode in which the target user outputs voice data to interact with the target picture book resource; If the picture book material being played is of the third tag type, a game-like interaction is performed with the target user, wherein the game-like interaction refers to an interaction method in which the target user interacts with the target picture book resource by triggering game elements that incorporate the knowledge points to be enlightened.

4. The picture book interaction method for thinking enlightenment according to claim 3 is characterized in that: The target picture book resource is a picture book resource based on a picture book story, and the picture book story includes at least one of a scenario introduction stage, a problem elicitation stage, an independent exploration stage, a step-by-step guidance stage, a scenario escalation stage, a problem closure stage, and a method transfer stage; The scenario introduction stage is used to introduce the story synopsis of the picture book story, wherein the story synopsis refers to information including the story overview of the picture book story but not including the ending of the story; And / or, the question elicitation stage is used to introduce the story background based on the story summary and elicit a first question related to the knowledge point to be enlightened, wherein the interaction data is required to be a standard answer to the first question; And / or, the autonomous exploration phase is used to enable the target user to conduct preliminary exploration of the first question through the game-like interaction; And / or, the step-by-step guidance stage is used to output the guidance information to the target user when the second interaction data is not obtained in the preliminary exploration; And / or, the scenario upgrade stage is used to, after obtaining the second interaction data based on the guidance information, continue playing the picture book story and raise a second question during the playing process, wherein the second question involves the same knowledge point to be enlightened as the first question but is equal to or more difficult than the first question; And / or, the closed-loop question stage is used to raise the first question again based on the second question, and to review and summarize the solution ideas for the first question; And / or, the method migration stage is used to enable the target user to answer the second question through the game-like interaction after the review and summary, and after the answer is correct, continue to play the picture book story until the end of the story.

5. The picture book interaction method for thinking enlightenment according to claim 4 is characterized in that: The outputting the guidance information to the target user includes: Outputting guidance information to the target user at least once, wherein the amount of guidance information included in the latter guidance information is greater than the amount of guidance information included in the former guidance information.

6. The picture book interaction method for thinking enlightenment according to claim 4 is characterized in that: The picture book materials include audio materials and visual materials. The audio materials include vocal materials used to tell the picture book story. The vocal materials include sound materials recorded by real people and / or sound materials reproduced based on real people's voices. The speaking speed, sentence structure, and language style of the vocal materials are all adapted to the target users.

7. The picture book interaction method for thinking enlightenment according to claim 1 is characterized in that: The step of obtaining the target picture book resource includes: Obtaining picture book materials for each picture book page; The target picture book resource is obtained by configuring the picture book materials of each picture book page on the time track through a pre-built editor, wherein the editor is built using the Tauri framework.

8. A picture book interactive device for enlightenment of thinking, characterized in that: include: A picture book resource acquisition module is used to acquire target picture book resources, wherein the target picture book resources are obtained from picture book materials on each picture book page, and the picture book materials are obtained based on the knowledge points to be enlightened and the preset picture book themes, and are suitable for the target user or the user level of the target user; The picture book resource playback module is used to play the target picture book resource, and during the playback process, perform at least one multimodal interaction with the target user based on the picture book material, so as to impart the knowledge points to be enlightened to the target user through the multimodal interaction and / or continue to play the target picture book resource.

9. A computer program product, characterized in that The method comprises computer-readable instructions, which, when executed on an electronic device, enable the electronic device to implement the picture book interaction method for thinking enlightenment as described in any one of claims 1 to 7.

10. An electronic device, characterized in that: comprising at least one processor and a memory connected to the processor, wherein: The memory is used to store computer programs; The processor is used to execute the computer program so that the electronic device can implement the picture book interaction method for thinking enlightenment as described in any one of claims 1 to 7.

11. A computer storage medium, characterized in that The storage medium carries one or more computer programs, and when the one or more computer programs are executed by an electronic device, the electronic device can implement the picture book interaction method for thinking enlightenment as described in any one of claims 1 to 7.

Images