Picture book reading recording method and device, storage medium and program product
By collecting audio and video data in picture book reading devices, identifying users' emotions and interactions, and generating reading reports, the problem of insufficient intelligence in picture book reading devices is solved, enabling effective monitoring and guidance of children's reading process.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- HEFEI IFLYTEK TOYCLOUD TECH
- Filing Date
- 2025-12-11
- Publication Date
- 2026-04-21
Smart Images
Figure CN121901436A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of artificial intelligence technology, and in particular to a method, device, storage medium, and program product for recording picture book reading. Background Technology
[0002] Picture books are a type of book that primarily uses illustrations and includes a small amount of text. Picture books not only tell stories and impart knowledge, but they also comprehensively help children build their inner world, making them an important resource for early childhood education.
[0003] Parents nowadays have limited time to spend with their children, and picture book reading is very important for young children's learning. Based on this, reading devices have emerged. Children can simply bring the picture book to the front of the reading device, and the device can read the picture book and read it with the child.
[0004] However, current picture book reading devices can only accompany children in reading picture books, and their intelligence needs to be further improved. Summary of the Invention
[0005] In view of the above problems, this application provides a picture book reading recording method, device, storage medium, and program product to improve the intelligence of picture book reading devices. The specific solution is as follows:
[0006] The first aspect of this application provides a picture book reading recording method for a picture book reading device, the method comprising:
[0007] During the reading of the target picture book, user behavior data is collected, including at least one modality of audio and video data;
[0008] After the target picture book reading is completed, the user's emotional distribution, determined based on the user's behavioral data, and the user's interaction with the picture book reading device are obtained.
[0009] Generate a reading report corresponding to the target picture book; the reading report includes: the user's emotional distribution and the user's interaction with the picture book reading device.
[0010] In one possible implementation, determining the user's emotional distribution based on the user's behavioral data includes:
[0011] Emotion recognition is performed on segments of the user's behavioral data to obtain the emotion corresponding to each data segment;
[0012] By performing statistical analysis on the emotions corresponding to each data segment, the emotional distribution of the user can be obtained.
[0013] In one possible implementation, the behavioral data includes audio corresponding to at least one page; obtaining the user's interaction with the picture book reading device includes:
[0014] Statistical analysis is performed on the interaction data between the user and the picture book reading device corresponding to each of the at least one page to obtain the number of interactions and the duration of interaction between the user and the picture book reading device based on each of the at least one page;
[0015] The user's interaction with the picture book reading device includes at least one level of interaction: page-level interaction and picture book-level interaction.
[0016] The page-level interaction includes at least one of the following: the number of interactions corresponding to the page, and the interaction duration corresponding to the page;
[0017] The picture book-level interaction includes at least one of the following: total number of interactions and total interaction duration; the total number of interactions is obtained by summing the number of interactions corresponding to each page; the total interaction duration is obtained by summing the interaction duration corresponding to each page.
[0018] In one possible implementation, where the user's interaction with the picture book reading device includes page-level interactions, the reading report further includes:
[0019] Interactive audio and / or engaging video content corresponding to the page.
[0020] One possible implementation also includes:
[0021] Based on the reading report corresponding to the target picture book, update the comprehensive reading report of picture books read within the most recent preset time period; the comprehensive reading report includes:
[0022] The list of picture books to be read within the preset time period, and the reading report corresponding to each picture book;
[0023] The changes in the interaction between the user and the picture book reading device within the preset time period.
[0024] One possible implementation also includes:
[0025] In response to the report display instruction, a first-level page is output; the content displayed on the first-level page includes: a list of picture books read within the preset time period, the total number of interactions and total duration between the user and the picture book reading device based on each picture book, and the changes in the interaction between the user and the picture book reading device within the preset time period;
[0026] If a selection instruction for any picture book is received, a secondary page is output; the content displayed on the secondary page includes: the user's emotional distribution during the reading of the picture book, each page of the picture book, and the user's interaction with the picture book reading device based on each page.
[0027] In one possible implementation, the changes in the interaction between the user and the picture book reading device within the preset time period include:
[0028] The change in the number of times the user actively asks questions to the picture book reading device within the preset time period;
[0029] The change in the number of times the user responds to the picture book reading device within the preset time period;
[0030] The duration of interaction between the user and the picture book reading device within the preset time period.
[0031] A second aspect of this application provides a computer program product including computer-readable instructions that, when executed on an electronic device, cause the electronic device to implement the picture book reading recording method described in the first aspect or any implementation thereof.
[0032] A third aspect of this application provides an electronic device, the electronic device comprising: at least one processor and a memory connected to the processor, wherein:
[0033] The memory is used to store computer programs;
[0034] The processor is used to execute the computer program so that the electronic device can implement the picture book reading recording method of the first aspect or any implementation thereof.
[0035] A fourth aspect of this application provides a computer storage medium carrying one or more computer programs, which, when executed by an electronic device, enable the electronic device to implement the picture book reading recording method described in the first aspect or any implementation thereof.
[0036] Using the above technical solutions, the picture book reading recording method, device, storage medium, and program products provided in this application collect user (e.g., child) behavioral data, such as audio and / or video, during the reading of a target picture book. This behavioral data is used to determine the user's emotions and interaction with the picture book reading device during the reading process. After reading the target picture book, a corresponding reading report is generated, including the user's emotional distribution and interaction with the picture book reading device. Based on this, relevant personnel (e.g., the child's guardian or teacher) can understand the user's reactions during the picture book reading process through the reading report, facilitating understanding of the child's picture book reading progress and improving the intelligence of the picture book reading device. Attached Figure Description
[0037] The above and other features, advantages, and aspects of the embodiments of this disclosure will become more apparent from the accompanying drawings and the following detailed description. Throughout the drawings, the same or similar reference numerals denote the same or similar elements. It should be understood that the drawings are schematic, and the originals and elements are not necessarily drawn to scale.
[0038] Figure 1 A flowchart illustrating an implementation of the picture book reading record method provided in this application;
[0039] Figure 2 A flowchart illustrating an implementation of determining user emotion distribution based on user behavior data provided in this application;
[0040] Figure 3 A partial example of a first-level page provided for this application;
[0041] Figure 4 A partial example of a second-level page provided for this application;
[0042] Figure 5 A schematic diagram of the structure of the electronic device provided in this application. Detailed Implementation
[0043] The embodiments of this application are described below with reference to the accompanying drawings. The terminology used in the implementation section of this application is for explaining specific embodiments only and is not intended to limit the scope of this application.
[0044] The embodiments of this application will now be described with reference to the accompanying drawings. Those skilled in the art will recognize that, with technological advancements and the emergence of new scenarios, the technical solutions provided in the embodiments of this application are equally applicable to similar technical problems.
[0045] The terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such terms are interchangeable where appropriate; this is merely a way of distinguishing objects with the same attributes in the embodiments of this application. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion, so that a process, method, system, product, or apparatus that comprises a series of elements is not necessarily limited to those elements, but may include other elements not explicitly listed or inherent to those processes, methods, products, or apparatuses.
[0046] This application is submitted so that the child's guardian or teacher can understand the child's picture book reading progress when using picture book reading devices.
[0047] like Figure 1 The diagram shown is a flowchart of one implementation of the picture book reading recording method provided in this application, which may include:
[0048] Step S101: During the reading of the target picture book, the picture book reading device collects user behavior data. The user behavior data includes at least one modality of data, including audio and video.
[0049] The target picture book can be an electronic picture book provided by a picture book reading device, or a physical picture book (usually a paper picture book) provided by the user.
[0050] With authorization from the device administrator (e.g., the child's guardian or teacher), the picture book reading device collects audio and / or video of the user (usually the child) from the moment the target picture book is read. The collected audio and / or video is used to identify the user's emotions and interactions with the reading device during the reading process. The video is of the user's head area; since the user is typically facing the reading device while reading, the collected video includes an image of the user's face.
[0051] Step S102: After reading the target picture book, the picture book reading device obtains the user's emotional distribution based on the user's behavioral data, as well as the user's interaction with the picture book reading device.
[0052] Optionally, if the target picture book is a physical picture book, page-turning detection technology can be used to detect whether the user has turned a page during the reading process. If a page has been turned, it is determined whether the current page is a page of the target picture book. If it is, it is determined that the reading of the target picture book has not ended; if the current page is not a page of the target picture book, it is determined that the reading of the target picture book has ended. If no page-turning is detected within a first preset time period and the user has not entered any information, the picture book reading device will exit the picture book reading function, at which point it can also be determined that the reading of the target picture book has ended.
[0053] Optionally, if a user inputs a picture book switching command during the reading of the target picture book, the reading of the target picture book is considered to have ended.
[0054] Optionally, the user's emotional distribution and the user's interaction with the picture book reading device can be determined based on the user's behavioral data after the target picture book reading is completed.
[0055] Optionally, the user's emotional distribution and interaction with the picture book reading device can be determined based on the user's behavioral data before the end of reading the target picture book. Based on this, each segment of behavioral data can be collected and analyzed to determine the user's emotional state and interaction with the picture book reading device corresponding to each segment. If no page turning is detected within the second preset time period, and the user does not input any information, the user's emotional state and interaction with the picture book reading device corresponding to each segment of behavioral data are merged to obtain the user's emotional distribution and interaction with the picture book reading device throughout the entire reading process of the target picture book. The second preset time period is shorter than the first preset time period. As an example, the second and first preset time periods can be in minutes. For example, the first preset time period is 5 minutes, and the second preset time period is 3 minutes; or, the first preset time period is 3 minutes, and the second preset time period is 2 minutes, etc.
[0056] Optionally, before the target picture book reading ends, a portion of the user's behavioral data can be analyzed to determine a partial analysis result (the user's emotional distribution and interaction with the picture book reading device during the initial period of reading the target picture book). After the target picture book reading ends, the unanalyzed behavioral data can be analyzed to determine another partial analysis result (the user's emotional distribution and interaction with the picture book reading device during the later period of reading the target picture book). The two partial analysis results can then be merged to obtain the user's emotional distribution and interaction with the picture book reading device throughout the entire process of reading the target picture book.
[0057] Among the three methods mentioned above for determining the distribution of user emotions and the interaction between the user and the picture book reading device, the latter two methods are more efficient than the first method in recording picture book reading.
[0058] The distribution of user emotions may include, but is not limited to: what emotions the user experienced while reading a picture book on a picture book reading device, the frequency of each emotion, or the percentage of each emotion among all the emotions that occurred.
[0059] User interactions with picture book reading devices can include, but are not limited to, the number of interactions and the duration of interactions. Each question and answer between a user and the picture book reading device is counted as one interaction.
[0060] Step S103: The picture book reading device generates a reading report corresponding to the target picture book. The reading report for the target picture book includes: the user's emotional distribution and the user's interaction with the picture book reading device.
[0061] In this application, the picture book reading device generates a reading report (which can be called a picture book-level reading report) for each picture book at the end of its reading session. This reading report includes at least the user's emotional state and the user's interaction with the picture book reading device.
[0062] The reading report is for relevant personnel to view. Relevant personnel can view the reading report corresponding to the target picture book on the picture book reading device, or on other devices that can communicate with the picture book reading device. In this case, the picture book reading device needs to send the reading report corresponding to the target picture book to the other devices.
[0063] Optionally, to help relevant personnel understand users' reading habits more intuitively, the distribution of users' emotions can be illustrated in the reading report using graphics (such as pie charts, donut charts, bar charts, etc.).
[0064] The picture book reading recording method provided in this application involves a picture book reading device collecting user (e.g., child) behavioral data, such as audio and / or video, during the reading of a target picture book. This behavioral data is used to determine the user's emotions and interaction with the reading device during the reading process. After reading the target picture book, a corresponding reading report is generated, including the user's emotional distribution and interaction with the device. Based on this, relevant personnel (e.g., the child's guardian or teacher) can understand the user's reactions during picture book reading through the reading report, facilitating understanding of the child's picture book reading progress and improving the intelligence of the picture book reading device.
[0065] Reading performance can include, but is not limited to, motivation, attention, and interest. For example, if a user experiences significant mood swings and / or interacts frequently with the picture book reading device, it indicates that the user is quite interested in the target picture book, demonstrating high motivation and attention. Conversely, if a user's mood remains relatively stable and they interact little with the picture book reading robot, it suggests that the user is not very interested in the target picture book, indicating low motivation.
[0066] In an optional embodiment, a flowchart illustrating one method for determining a user's emotional distribution based on their behavioral data is shown below. Figure 2 As shown, it may include:
[0067] Step S201: Perform emotion recognition on the user's behavior data segments to obtain the emotion corresponding to each data segment (denoted as the first type of data segment).
[0068] Optionally, emotion recognition can be performed on segmented user behavior data during picture book reading. Specifically, starting from the beginning of reading the target picture book, behavior data of a preset duration (referred to as the third preset duration for ease of description and differentiation) is collected for each segment (each segment of behavior data of the preset duration is referred to as a first-type data segment for ease of description and differentiation). Emotion recognition is then performed on the collected first-type data segments to obtain the emotion corresponding to that first-type data segment. Each first-type data segment corresponds to one emotion. Different first-type data segments may correspond to the same or different emotions.
[0069] Optionally, emotion recognition can be performed on user behavior data segments only after the picture book reading is completed. Specifically, the collected user behavior data can be divided into multiple first-class data segments of preset duration (i.e., a third preset duration), and emotion recognition can be performed on each first-class data segment to obtain the emotion corresponding to each first-class data segment.
[0070] The third preset duration is shorter than the aforementioned second preset duration. As an example, the third preset duration can be in the range of seconds, such as 1 second, 2 seconds, or 3 seconds.
[0071] Step S202: Perform statistical analysis on the emotions corresponding to each data segment (i.e., the first type of data segment) to obtain the occurrence of different emotions of users during the reading of the target picture book, i.e., the distribution of users' emotions.
[0072] In an optional embodiment, when collecting user behavior data, the interaction data (i.e., dialogue content) between the user and the picture book reading device can be cached or stored in segments by page. Based on this, the interaction data when reading each page can be cached or stored as a data fragment (referred to as the second type of data fragment for ease of description and distinction). If a page is read repeatedly, one page may correspond to multiple second type of data fragments. The lengths of the second type of data fragments corresponding to different pages may be the same or different.
[0073] In real-world reading scenarios, not every page requires user interaction. Therefore, only some pages may correspond to the second type of data fragment. Based on this, given that user behavior data includes audio corresponding to at least one page, one way to obtain information about the user's interaction with the picture book reading device could be:
[0074] Statistical analysis was performed on the user interaction data between the user and the picture book reading device for each of the at least one of the aforementioned pages to obtain the number of interactions and the duration of interaction between the user and the picture book reading device based on each of the at least one of the aforementioned pages. In other words, each of the at least one of the aforementioned pages corresponds to a number of interactions and an interaction duration.
[0075] The second type of data segment for each page records the start and end times of the interaction, thus allowing us to determine the interaction duration for each page.
[0076] The total number of interactions between the user and the picture book reading device based on the target picture book is obtained by summing the number of interactions corresponding to each page.
[0077] Sum the interaction times for each page to get the total interaction time between the user and the picture book reading device based on the target picture book.
[0078] User interaction with picture book reading devices includes, but is not limited to, at least one level of interaction: page-level interaction and picture book-level interaction.
[0079] Among them, page-level interaction includes at least one of the following: the number of interactions corresponding to the page, and the interaction duration corresponding to the page;
[0080] The interactive data for picture book level includes at least one of the following: total number of interactions and total interaction duration.
[0081] In an optional embodiment, where the user's interaction with the picture book reading device includes page-level interactions, the reading report may also include: interactive audio and / or highlights of the page.
[0082] Featured videos are video clips extracted from the corresponding videos on the page, provided that the page's behavioral data includes video content.
[0083] Optionally, the system can identify whether a user's response to a question posed by the picture book reading device is engaging based on interaction data. If the user's response is engaging, the video segment containing the audio clip of the engaging response and its corresponding question can be identified as an engaging video. For example, the questions posed by the picture book reading device and the user's responses can be input into a large-scale model, which can then determine whether the user's responses are engaging.
[0084] Alternatively, it can identify whether the user's expression is exaggerated based on a video segment of a third preset duration. If so, the video segment of the third preset duration is identified as a highlight video, or the video segment of the third preset duration and the preceding video segment of the third preset duration are identified as highlight videos.
[0085] By providing audio and / or engaging video clips for the corresponding pages in the report, relevant personnel can gain a more detailed understanding of users' actual reading behavior.
[0086] In an optional embodiment, a comprehensive reading report of picture books read within a recent period can also be updated based on the reading report corresponding to the target picture book. This reading report includes:
[0087] The list of picture books read by the user through the picture book reading device within a preset time period (referred to as the fourth preset time period for ease of description), and the reading report corresponding to each picture book; the fourth preset time period can be a duration on the order of days, such as the most recent day, the most recent week, or the most recent month.
[0088] Within the fourth preset time period, changes in the interaction between the user and the picture book reading device may include, but are not limited to, changes in the following dimensions:
[0089] The change in the number of times users actively ask questions to the picture book reading device within the fourth preset time period; the change in the number of questions may include, but is not limited to: the number of times users actively ask questions each day within the fourth preset time period.
[0090] The number of times the user responds to the picture book reading device within the fourth preset time period; this number of responses may include, but is not limited to, the number of times the user responds to the picture book reading device each day within the fourth preset time period.
[0091] The duration of user interaction with the picture book reading device within the fourth preset time period; the duration of this variation may include, but is not limited to, the daily duration of user interaction with the picture book reading device within the fourth preset time period.
[0092] In an optional embodiment, a comprehensive reading report can also be output if a report display instruction is received. The user can trigger the report display instruction by performing a first preset operation on the picture book reading device.
[0093] Optionally, in response to the report display instruction, a first-level page is output, which displays the following content: a list of picture books read within the fourth preset time period, the total number of interactions and total duration between the user and the picture book reading device based on each picture book, and the changes in the interaction between the user and the picture book reading device within the fourth preset time period.
[0094] like Figure 3 The image shown is a partial example of a first-level page provided in an embodiment of this application. The graph corresponding to "Number of Asked Questions" reflects the number of times users actively asked questions each day in the past week, totaling 30 questions. The graph corresponding to "Number of Answered Questions" reflects the number of times users answered questions each day in the past week, totaling 30 questions. The graph corresponding to "Total Interaction Time" reflects the total time users spent interacting with the picture book reading device each day in the past week, totaling 3000 seconds. Figure 3 The data in the upper right corner shows that after completing the target picture book reading, the number of times users proactively asked questions increased by 6, the number of times they answered questions increased by 3, and the interaction time increased by 36 seconds. The graph allows viewers to gain a more direct and detailed understanding of the user's reading progress.
[0095] When reading the same picture book at different times, different reading reports will be generated for that picture book at different times. For example Figure 3 In the example shown, if a user reads the picture book "Let's Go to the Sea Together" twice using a picture book reading device, then two reading reports will be generated for this picture book.
[0096] Figure 3 In the example shown, the reading time for each picture book is also displayed.
[0097] If a selection instruction for any picture book is received, a second-level page is output. The content displayed on the second-level page includes: the user's emotional distribution during the reading of any of the picture books, each page of any picture book, and the user's interaction with the picture book reading device based on each page.
[0098] Users can perform a second preset operation on any picture book on the first-level page, thereby triggering a selection command for any picture book.
[0099] In other words, in this application, part of the picture book-level reading report (such as the number of interactions and the duration of interactions) is displayed on the first-level page, and the other part is displayed on the second-level page.
[0100] like Figure 4 The image shown is a partial example of a secondary page provided in an embodiment of this application. This example is the page displayed after a user selects the picture book "Grandpa Will Always Find a Way". In this example, while reading the picture book "Grandpa Will Always Find a Way", the user experiences three emotions: positive (also known as excitement), stable (also known as neutral), and sad (also known as depression). The distribution of these three emotions is represented by different colors in the ring, allowing the user to quickly and clearly understand the distribution of their emotions.
[0101] For each page of a picture book, only one page can be displayed at a time, and users can interact with the book through controls (such as...). Figure 4 Use the arrows located on either side of the picture book page to switch the displayed page.
[0102] Furthermore, the page can display tags corresponding to each page at the bottom (e.g., ... Figure 4 The labels for each page (using dots) are used to indicate whether the user is interacting with the picture book reading device based on that page, or to indicate whether the page is currently being displayed. Optionally, the labels for each page can use different colors to indicate whether the user is interacting with the picture book reading device based on that page, and whether the page is filled with color to indicate whether the page is currently being displayed. Figure 4 In the example shown, gray indicates that the user has not interacted with the picture book reading device on the corresponding page, yellow indicates that the user has interacted with the picture book reading device on the corresponding page, and no color fill (e.g., the second dot from the left) indicates that the currently displayed page is the second page. The reading time of the currently displayed page is also displayed at the top of the page.
[0103] Below the currently displayed page, audio and video recordings of the user's interaction with the picture book reading device based on the displayed page are also included. In other words, if the user interacts with the picture book reading device via voice based on the displayed page, the audio of this interaction can be displayed below the picture book page; if a corresponding video recording is recognized, that video will also be displayed. If no corresponding video recording is recognized, only the audio will be displayed.
[0104] If the user does not interact with the picture book reading device via voice based on the displayed page, there will be no associated audio and video content at the bottom of the picture book page.
[0105] The interactive audio and exciting videos at the bottom of the picture book page will update in sync with the page display.
[0106] Using statistical charts (such as donut charts representing the distribution of emotions), audio, and video can help relevant personnel gain a more comprehensive understanding of users' picture book reading habits, allowing for better planning of children's picture book reading.
[0107] This application also provides an electronic device in its embodiments. (See reference...) Figure 5 The diagram illustrates a structural schematic of an electronic device suitable for implementing embodiments of this application. The electronic device in these embodiments can be a terminal device, such as a smartphone, tablet computer, computer, smartwatch, picture book reading robot, etc. Figure 5 The electronic device shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of this application.
[0108] like Figure 5 As shown, the electronic device may include a processing unit (e.g., a central processing unit, a graphics processing unit, etc.) 501, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 502 or a program loaded from a storage device 508 into a random access memory (RAM) 503. When the electronic device is powered on, the RAM 503 also stores various programs and data required for the operation of the electronic device. The processing unit 501, ROM 502, and RAM 503 are interconnected via a bus 504. An input / output (I / O) interface 505 is also connected to the bus 504.
[0109] Typically, the following devices can be connected to I / O interface 505: input devices 506 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 507 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 508 including, for example, memory cards, hard drives, etc.; and communication devices 509. Communication device 509 allows electronic devices to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 5 Electronic devices with various devices are shown, but it should be understood that it is not required to implement or have all of the devices shown. More or fewer devices may be implemented or have alternatively.
[0110] This application also provides a computer program product including computer-readable instructions, which, when executed on an electronic device, cause the electronic device to implement any of the picture book reading recording methods provided in this application.
[0111] This application also provides a computer-readable storage medium that carries one or more computer programs. When the one or more computer programs are executed by an electronic device, the electronic device can implement any of the picture book reading recording methods provided in this application.
[0112] It should be noted that the device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. In addition, in the device embodiment drawings provided in this application, the connection relationship between modules indicates that they have a communication connection, which can be implemented as one or more communication buses or signal lines.
[0113] Through the above description of the embodiments, those skilled in the art can clearly understand that this application can be implemented by means of software plus necessary general-purpose hardware, or it can be implemented by special-purpose hardware including application-specific integrated circuits, special-purpose CPUs, special-purpose memory, special-purpose components, etc. Generally, any function performed by a computer program can be easily implemented by corresponding hardware, and the specific hardware structure used to implement the same function can also be diverse, such as analog circuits, digital circuits, or special-purpose circuits. However, for this application, software program implementation is more often a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a readable storage medium, such as a computer floppy disk, USB flash drive, mobile hard disk, ROM, RAM, magnetic disk, or optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, training equipment, or network device, etc.) to execute the methods described in the various embodiments of this application.
[0114] In the above embodiments, the functionality can be implemented entirely or partially through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented entirely or partially as a computer program product. Those skilled in the art can use different methods to implement the described functions for each specific solution, but such implementation should not be considered beyond the scope of this application.
[0115] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions may be transmitted from one website, computer, training device, or data center to another website, computer, training device, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that a computer can store or a data storage device such as a training device or data center that integrates one or more available media. The available media may be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., DVDs), or semiconductor media (e.g., solid-state drives (SSDs)).
[0116] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.
[0117] The above description of the disclosed embodiments enables those skilled in the art to make or use this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.
Claims
1. A method for recording picture book reading, used in a picture book reading device, characterized in that, The method includes: During the reading of the target picture book, user behavior data is collected, including at least one modality of audio and video data; After the target picture book reading is completed, the user's emotional distribution, determined based on the user's behavioral data, and the user's interaction with the picture book reading device are obtained. Generate a reading report corresponding to the target picture book; the reading report includes: the user's emotional distribution and the user's interaction with the picture book reading device.
2. The method according to claim 1, characterized in that, Determining the user's emotional distribution based on the user's behavioral data includes: Emotion recognition is performed on segments of the user's behavioral data to obtain the emotion corresponding to each data segment; By performing statistical analysis on the emotions corresponding to each data segment, the emotional distribution of the user can be obtained.
3. The method according to claim 1, characterized in that, The behavioral data includes audio corresponding to at least one page; obtaining the user's interaction with the picture book reading device includes: Statistical analysis is performed on the interaction data between the user and the picture book reading device corresponding to each of the at least one page to obtain the number of interactions and the duration of interaction between the user and the picture book reading device based on each of the at least one page; The user's interaction with the picture book reading device includes at least one level of interaction: page-level interaction and picture book-level interaction. The page-level interaction includes at least one of the following: the number of interactions corresponding to the page, and the interaction duration corresponding to the page; The picture book-level interaction includes at least one of the following: total number of interactions and total interaction duration; the total number of interactions is obtained by summing the number of interactions corresponding to each page; the total interaction duration is obtained by summing the interaction duration corresponding to each page.
4. The method according to claim 3, characterized in that, If the user's interaction with the picture book reading device includes page-level interactions, the reading report also includes: Interactive audio and / or engaging video content corresponding to the page.
5. The method according to any one of claims 1-4, characterized in that, Also includes: Based on the reading report corresponding to the target picture book, update the comprehensive reading report of the picture books read within the most recent preset time period; The comprehensive reading report includes: The list of picture books to be read within the preset time period, and the reading report corresponding to each picture book; The changes in the interaction between the user and the picture book reading device within the preset time period.
6. The method according to claim 5, characterized in that, Also includes: In response to the report display instruction, a first-level page is output; the content displayed on the first-level page includes: a list of picture books read within the preset time period, the total number of interactions and total duration between the user and the picture book reading device based on each picture book, and the changes in the interaction between the user and the picture book reading device within the preset time period; If a selection instruction for any picture book is received, a secondary page is output; the content displayed on the secondary page includes: the user's emotional distribution during the reading of the picture book, each page of the picture book, and the user's interaction with the picture book reading device based on each page.
7. The method according to claim 5, characterized in that, The changes in the interaction between the user and the picture book reading device within the preset time period include: The change in the number of times the user actively asks questions to the picture book reading device within the preset time period; The change in the number of times the user responds to the picture book reading device within the preset time period; The duration of interaction between the user and the picture book reading device within the preset time period.
8. A computer program product, characterized in that, It includes computer-readable instructions that, when executed on an electronic device, cause the electronic device to implement the picture book reading recording method as described in any one of claims 1 to 7.
9. An electronic device, characterized in that, The electronic device includes at least one processor and a memory connected to the processor, wherein: The memory is used to store computer programs; The processor is used to execute the computer program to enable the electronic device to implement the picture book reading recording method as described in any one of claims 1 to 7.
10. A computer storage medium, characterized in that, The storage medium carries one or more computer programs that, when executed by an electronic device, enable the electronic device to implement the picture book reading recording method as described in any one of claims 1 to 7.