Method, device, computer equipment and storage medium for assisting picture book reading

Abstract

The present invention discloses a method, device, computer equipment and storage medium for assisting picture book reading, wherein the method for assisting picture book reading includes: obtaining real-time picture book features, matching target picture books and target picture book pages based on picture book databases and real-time picture book features; extracting picture book page texts corresponding to target picture book pages based on picture book texts corresponding to target picture books; analyzing picture book page texts using a language processing module to obtain interactive page content; matching interactive page content based on an interactive content database, and if the matching result is a failure, converting the interactive page content into interactive page audio and playing the interactive page audio through an audio processing module. The method can enhance the participation of readers, especially children, in the process of reading picture books, facilitate readers to deeply understand the knowledge carried by picture book content, and enhance the ability to learn picture book knowledge by themselves.

Description

Technical Field

[0001] The present invention relates to the technical field of natural language understanding, and in particular to a method, device, computer equipment and storage medium for assisting picture book reading. Background Art

[0002] Picture books are the first choice of children's books for families and are the most suitable books for young children to read. They not only tell stories and spread knowledge, but also help children build their spiritual world and cultivate multiple intelligences. As more and more parents are becoming more aware of picture books, it has become a common phenomenon to buy a large number of picture books for their children. However, not all parents have the time or ability to lead their children to read picture books in depth, and children cannot absorb the knowledge provided by picture books efficiently. How to help children achieve a deep understanding of picture book knowledge has become an urgent problem to be solved. Summary of the invention

[0003] The embodiments of the present invention provide a method, an apparatus, a computer device and a storage medium for assisting picture book reading, so as to solve the problem of helping children to achieve a deep understanding of picture book knowledge.

[0004] A method for assisting picture book reading, comprising:

[0005] Acquire real-time picture book features, and match target picture books and target picture book pages based on the picture book database and the real-time picture book features;

[0006] Based on the picture book text corresponding to the target picture book, extract the picture book page text corresponding to the target picture book page;

[0007] The language processing module is used to analyze the text of the picture book page to obtain the interactive page content;

[0008] The interactive page content is matched based on the interactive content database. If the matching result is a failure, the interactive page content is converted into interactive page audio and the interactive page audio is played through the audio processing module.

[0009] Preferably, obtaining the real-time picture book feature, and matching the target picture book and the target picture book page based on the picture book database and the real-time picture book feature, includes:

[0010] Acquire picture book page text recognized in real time, picture book page image or picture book page video taken in real time on the same picture book page to be recognized; perform feature matching in the picture book database based on the picture book page text, picture book page image or picture book page video to obtain the corresponding target picture book and target picture book page.

[0011] Preferably, a language processing module is used to analyze the text of the picture book page to obtain the interactive page content, including:

[0012] Use the language understanding module to process the text of picture book pages and generate structured data;

[0013] Combine the picture book knowledge base and language generation module to extract information from structured data and generate interactive page content.

[0014] Preferably, the structured data includes keywords;

[0015] Combine the picture book knowledge base and language generation module to extract information from structured data and generate interactive page content, including:

[0016] Extract the semantic relationship of keywords;

[0017] Filter keywords based on word meaning relationship and picture book knowledge base, and obtain the filter results as reserved words;

[0018] Sort, aggregate and grammaticalize reserved words to obtain interactive page content.

[0019] Preferably, the interactive content database includes user IDs;

[0020] After matching the interactive page content based on the interactive content database, it also includes:

[0021] If the matching result is successful, the interactive page play rate and the most recent play time corresponding to the interactive page content are obtained based on the user ID; if the interactive page play rate is greater than the play rate threshold or the most recent play time is later than the play time threshold, stop converting the interactive page content into interactive page audio.

[0022] Preferably, the interactive page content includes interactive page questions and interactive page answers;

[0023] Play interactive page audio, including:

[0024] Play the interactive page question and play the interactive page answer after the preset question waiting time.

[0025] Preferably, it also includes:

[0026] Get interactive page content request, which includes interactive page ID and user interactive audio;

[0027] Convert user interaction audio into user interaction text;

[0028] The language understanding module is used to analyze the user interaction text and the picture book page corresponding to the interaction page ID to obtain the response content of the interaction page;

[0029] Convert interactive page reply content into interactive page reply audio and play the interactive page reply audio.

[0030] A device for assisting picture book reading, comprising:

[0031] A picture book feature acquisition module is used to acquire real-time picture book features and match target picture books and target picture book pages based on the picture book database and real-time picture book features;

[0032] The picture book text extraction module is used to extract the picture book page text corresponding to the target picture book page based on the picture book text corresponding to the target picture book; the interactive page content acquisition module is used to analyze the picture book page text using the language processing module to obtain the interactive page content; the interactive page audio playback module is used to match the interactive page content based on the interactive content database. If the matching result is a failure, the interactive page content is converted into interactive page audio through the audio processing module and the interactive page audio is played.

[0033] A computer device comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor implements the above-mentioned method for assisting picture book reading when executing the computer program.

[0034] A computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method for assisting picture book reading is implemented.

[0035] The above-mentioned method, device, computer equipment and storage medium for assisting picture book reading analyze the picture book page text through the language understanding module, generate interactive page content and then convert it into interactive page audio for playback during the reading process, so as to complete the interactive reading of picture books, enhance the participation of readers, especially children, in the process of reading picture books, help readers to deeply understand the knowledge carried by the picture book content, improve the ability to learn picture book knowledge by themselves, and save the time of manual reading. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments of the present invention will be briefly introduced below. Obviously, the accompanying drawings in the following description are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative labor.

[0037] Figure 1 is a schematic diagram of an application environment of a method for assisting picture book reading in one embodiment of the present invention;

[0038] Figure 2 is a flow chart of a method for assisting picture book reading in one embodiment of the present invention;

[0039] Figure 3 is a first flow chart of a method for assisting picture book reading in another embodiment of the present invention;

[0040] Figure 4is a second flow chart of a method for assisting picture book reading in another embodiment of the present invention;

[0041] Figure 5 is a third flow chart of a method for assisting picture book reading in another embodiment of the present invention;

[0042] Figure 6 is a schematic diagram of a flow chart of information extraction from structured data by a language generation module in a method for assisting picture book reading in another embodiment of the present invention;

[0043] Figure 7 is a fourth flow chart of a method for assisting picture book reading in another embodiment of the present invention;

[0044] Figure 8 is a fifth flow chart of a method for assisting picture book reading in another embodiment of the present invention;

[0045] Fig. 9 is a schematic diagram of the entire process of a method for assisting picture book reading in another embodiment of the present invention;

[0046] Fig.10 is a schematic diagram of a device for assisting picture book reading in one embodiment of the present invention;

[0047] Fig.11 is a schematic diagram of a computer device in one embodiment of the present invention. DETAILED DESCRIPTION

[0048] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0049] The method for assisting picture book reading provided by the embodiment of the present invention can be applied in the following aspects: Figure 1 In the application environment, the method for assisting picture book reading is applied in a system for assisting picture book reading, and the system for assisting picture book reading includes a client and a server, wherein the client communicates with the server through a network. The client, also known as the user end, refers to a program corresponding to the server that provides local services to the client. Furthermore, the client is a computer program, an APP program for a smart device, or a third-party applet embedded in other APPs. The client can be installed on, but not limited to, various computer devices such as personal computers, laptops, smart phones, tablet computers, and portable wearable devices. The server can be implemented with an independent server or a server cluster consisting of multiple servers.

[0050] Deep Reading is a new reading mode that is based on knowledge graphs and integrates book-related knowledge sources. It has the capabilities of content understanding, association analysis, and reader behavior analysis. It supports personalized, context-aware knowledge recommendations and proactively provides readers with comprehensive, relevant, and intelligent deep knowledge services.

[0051] In one embodiment, if Figure 2 As shown, a method for assisting picture book reading is provided, and the method is applied in Figure 1 The server in the example is used as an example, and the specific steps include the following:

[0052] S10. Acquire real-time picture book features, and match target picture books and target picture book pages based on the picture book database and the real-time picture book features.

[0053] Among them, picture book features are features that can be used to identify a specific picture book, such as the unique design of the cover or back cover, the picture of a page in the picture book, and the design of the side page of the picture book. Real-time picture book features are picture book features captured by this system in real time, which are used to identify a specific picture book or a page in a specific picture book. The target picture book is a picture book determined by the system after matching the real-time picture book features. The target picture book page is a specific page in the picture book where the real-time picture book feature is located, which can be the cover page, the middle page of the picture book, or the back cover page, etc.

[0054] The picture book database is a database of all picture books that can be recognized by the storage system. The stored content may include the ISBN number, title, author, abstract, picture book text, picture book image, and picture book audio of each picture book.

[0055] Specifically, the method for assisting picture book reading provided in this embodiment can form a system for assisting picture book reading. For ease of description, the system for assisting picture book reading is referred to as the system below. The system can obtain real-time picture book features through a camera in a real-time capture device, perform feature recognition on the captured real-time picture book features, and locate the target picture book page in the target picture book, thereby facilitating the subsequent steps to prepare interactive page content based on the current target picture book page. The interactive page content can be a specific and in-depth explanation of the picture book content on the current page, or it can be a question generated in real time based on the content on the picture book page, which is not specifically limited here.

[0056] S20. Based on the picture book text corresponding to the target picture book, extract the picture book page text corresponding to the target picture book page.

[0057] The picture book text is a text that records the words displayed on each page of the target picture book.

[0058] Specifically, the system can retrieve the picture book text corresponding to the target picture book and the text on the specific picture book page corresponding to the target picture book page through the picture book database as the picture book page text, thereby achieving precise interaction and forming scenario-based and personalized interactive page content.

[0059] S30. Use the language processing module to analyze the text of the picture book page to obtain the interactive page content.

[0060] Among them, the language understanding module is a module that reads the meaning of the text on the picture book page and generates the corresponding interactive page content.

[0061] Specifically, Natural Language Processing (NLP) is the use of computers to process, understand, and use human languages ​​(such as Chinese, English, etc.). It is a branch of artificial intelligence, an interdisciplinary subject between computer science and linguistics, and is often referred to as computational linguistics. Since natural language is the fundamental mark that distinguishes humans from other animals, without language, human thinking would be out of the question, so natural language processing embodies the highest task and realm of artificial intelligence. In other words, only when computers have the ability to process natural languages ​​can machines achieve true intelligence.

[0062] Natural language processing includes grammatical analysis, semantic analysis, and paragraph understanding. From an application perspective, natural language processing has broad application prospects. Especially in the information age, the application of natural language processing is all-encompassing, such as: machine translation, handwritten and printed character recognition, speech recognition and text-to-speech conversion, information retrieval, information extraction and filtering, text classification and clustering, public opinion analysis and opinion mining, etc. It involves data mining related to language processing, machine learning, knowledge acquisition, knowledge engineering, artificial intelligence research, and linguistics research related to language computing.

[0063] Natural language processing (NLP) is divided into natural language understanding (NLU) and natural language generation (NLG). NLU is responsible for understanding text content, while NLG is responsible for generating text content based on information, which can be voice, video, picture, text, etc. Correspondingly, in this embodiment, the language processing module includes a language understanding module and a language generation module.

[0064] Natural Language Understanding (NLU): Make computers understand natural language (human language and text), etc., focusing on understanding. Specifically, it is to understand language, text, etc., and extract useful information for downstream tasks. It can be to structure natural language, such as word segmentation, part-of-speech tagging, syntactic analysis, etc.; it can also be representation learning, vector representation (Embedding) of characters, words, and sentences, and text classification for building text representation; it can also be information extraction, such as information retrieval (including personalized search and semantic search, text matching, etc.), and information extraction (named entity extraction, relationship extraction, event extraction, etc.).

[0065] Natural Language Generation (NLG): Provide structured data, text, charts, audio, video, etc. to generate text in the form of natural language that humans can understand. NLG can be divided into three categories: text-to-text, such as translation, summarization, etc., text-to-other, such as text-to-image generation, and other-to-text, such as video-to-text generation.

[0066] S40. Match the interactive page content based on the interactive content database. If the matching result is a failure, convert the interactive page content into interactive page audio through the audio processing module and play the interactive page audio.

[0067] Among them, the audio processing module is a module that converts text into audio.

[0068] The interactive content database is a database that records the interactive page content of picture books that have been played for a certain user. It is used to avoid playing the same interactive page content multiple times, thereby improving the flexibility and applicability of the system and enhancing the practical significance of deep learning of picture books.

[0069] Specifically, TTS is part of the human-machine dialogue, allowing the machine to speak. With the support of the built-in chip, through the design of the neural network, the text can be intelligently converted into a natural speech stream. TTS technology converts text files in real time, and the conversion time can be calculated in seconds. Under the action of its unique intelligent voice controller, the voice rhythm of the text output is smooth, making the listener feel natural when listening to the information, without the indifference and jerky feeling of machine voice output. TTS speech synthesis technology covers the first and second level Chinese characters of the national standard, has an English interface, automatically recognizes Chinese and English, and supports mixed reading of Chinese and English. All sounds use real mandarin as the standard pronunciation, achieving fast speech synthesis of 120-150 Chinese characters / minute, and the reading speed reaches 3-4 Chinese characters / second, so that users can hear clear and pleasant sound quality and coherent and fluent intonation.

[0070] TTS can not only help people with visual impairments read information on computers, but also increase the readability of text documents. TTS applications include voice-driven emails and voice-sensitive systems, and are often used in conjunction with voice recognition programs.

[0071] This embodiment may use a TTS (Text-To-Speech) module to convert text into playable audio, namely, interactive page audio.

[0072] The method for assisting picture book reading provided in this embodiment analyzes the picture book page text through a language understanding module, generates interactive page content, and then converts it into interactive page audio for playback during the reading process, so as to complete the interactive reading of picture books, enhance the participation of readers, especially children, in the process of reading picture books, help readers to deeply understand the knowledge carried by the picture book content, improve the ability to learn picture book knowledge by themselves, and save time for manual reading.

[0073] In a specific embodiment, if Figure 3 In step S10, the real-time picture book feature is obtained, and the target picture book and the target picture book page are matched based on the picture book database and the real-time picture book feature, which specifically includes the following steps:

[0074] S11. Obtain picture book page text recognized in real time, a picture book page image or a picture book page video captured in real time on the same picture book page to be recognized.

[0075] S12. Based on the picture book page text, picture book page image or picture book page video, feature matching is performed in the picture book database to obtain the corresponding target picture book and target picture book page.

[0076] Specifically, this embodiment may use an image recognition tool or a text recognition tool, such as using an ORC (Optical Character Recognition) electronic device (such as a scanner or a digital camera) to check the characters printed on paper, determine their shapes by detecting dark and light patterns, and then use a character recognition method to translate the shapes into computer text; that is, for printed characters, an optical method is used to convert the text in a paper document into a black and white dot matrix image file, and the text in the image is converted into a text format through recognition software for further editing and processing by a language processing module.

[0077] In a specific embodiment, if Figure 4 In step S30, the language processing module is used to analyze the text of the picture book page to obtain the interactive page content, which specifically includes the following steps:

[0078] S31. Use the language understanding module to process the picture book page text and generate structured data.

[0079] S32. Combine the picture book knowledge base and language generation module to extract information from structured data and generate interactive page content.

[0080] Among them, structured data is stored in the database and can be analyzed or processed in a traditional way using conventional methods to express logical data. These data types can be roughly divided into two categories: numerical data (images, numbers, etc.) and categorical data. Structured data can be represented and stored using a relational database, and is presented as two-dimensional data. The general characteristics are: data is in rows, a row of data represents the information of an entity, and the attributes of each row of data are the same. For example, as shown in Table 1:

[0081] id name age gender <![CDATA[ 1 ]]> Liu Yi 20 male <![CDATA[ 2 ]]> Chen Er 35 female <![CDATA[ 3 ]]> Zhang San <![CDATA[ 28 ]]> male

[0082] Table 1

[0083] The picture book knowledge base is a knowledge graph library used to define the interactive content rules of picture books. The interactive content rules in this embodiment include: the judgment of a single variable or multiple variables, the questioning rules of the application scenario, the way to propose the interactive page content and the points that can be proposed, the number of words in the interactive content, the voice of the dubbing person for subsequent dubbing, etc. For example, if it is determined that there is a red apple in the interactive page content, the following interactive questions can be asked to the reader based on the questioning rules of the application scenario: "What color is the apple, what is it called in English, what stories or proverbs do you know about apples, what other fruits do you like", etc.

[0084] Specifically, the system can further extract the semantics and semantic relationships of the picture book page text through the attributes and entities provided by the structured data, and then match the corresponding interactive page content according to the picture book knowledge base.

[0085] In a specific embodiment, the structured data includes keywords. Figure 5 and Figure 6 In step S332, information is extracted from the structured data by combining the picture book knowledge base and the language generation module to generate interactive page content, which specifically includes the following steps:

[0086] S321. Extract the semantic relationship of keywords.

[0087] S322. Filter keywords based on word meaning relationship and picture book knowledge base, and obtain the filtered results as reserved words.

[0088] S323. Sort, aggregate and grammatically process the reserved words to obtain interactive page content.

[0089] Grammaticalization specifically includes processes such as adding connectives, generating reference expressions, and language realization.

[0090] Specifically, the specific implementation process of this embodiment is as follows:

[0091] (1)1 Content Determination Determine the information that needs to be included in the generated target article.

[0092] (2) Text Structuring: Rationally organize the text order according to the determined content. For example, take an article about a mountain: foot of the mountain -> mountainside -> mountaintop.

[0093] (3) Sentence Aggregation: Merge the content in a sentence. For example, some information can be combined into a sentence.

[0094] (4) Grammaticalization: After the content of each sentence is determined, appropriate conjunctions are added to the content.

[0095] (5) Referencing Expression Generation (REG) For given words and phrases, determine the domain they belong to, perform domain identification through REG, and replace them with vocabulary from that domain.

[0096] (6) Linguistic Realisation: Combining all relevant words and certain phrases into a well-structured complete sentence.

[0097] In one embodiment, the interactive content database includes user IDs. Figure 7 The method further includes the following steps, namely, after step S40, namely, after matching the interactive page content based on the interactive content database:

[0098] S401. If the matching result is successful, the interactive page play rate and the most recent play time corresponding to the interactive page content are obtained based on the user ID.

[0099] S402. If the interactive page play rate is greater than the play rate threshold or the most recent play time is later than the play time threshold, stop converting the interactive page content into interactive page audio.

[0100] The interactive page play rate is the ratio of the number of times the interactive page content is played to the number of times all interactive content on the interactive page is played. The most recent play time is the most recent time when the same interactive page content was played last time.

[0101] The playback threshold is the maximum interactive page playback rate of interactive page content. It is understandable that when a certain interactive page content has been played for a period of time not later than the playback time threshold, but its playback rate threshold is too high, it should be temporarily avoided from being pushed to readers again. It is understandable that the playback time threshold is set to the interval time from the current time, which can be set to one week, two weeks, etc. according to the application scenario, and is not specifically limited here.

[0102] This embodiment can effectively control the number of times the same interactive page content is played by setting the playback rate threshold and the playback time threshold, avoiding repetitive problems caused by multiple playbacks in a short period of time, and can form other new problems, thereby improving the reliability and effectiveness of deep learning of picture book reading knowledge.

[0103] In a specific embodiment, the interactive page content includes interactive page questions and interactive page answers. In step S40, that is, playing the interactive page audio, specifically includes the following steps:

[0104] S41. Play the interactive page question, and play the interactive page answer after the preset question waiting time.

[0105] Specifically, this embodiment can not only ask questions to readers, but also continue to provide answers to the questions to readers, so as to help readers effectively understand the questions and determine whether the answers they give are right or wrong, truly promote the effect of in-depth learning of picture book knowledge, and enhance the intelligence and self-service nature of learning picture book knowledge.

[0106] In a specific embodiment, if Figure 8 The method further specifically comprises the following steps:

[0107] S501. Obtain an interactive page content request, where the interactive page content request includes an interactive page ID and user interactive audio.

[0108] S502. Convert the user interaction audio into user interaction text.

[0109] S503. Use the language understanding module to analyze the user interaction text and the picture book page corresponding to the interaction page ID to obtain the interaction page reply content.

[0110] S504. Convert the interactive page reply content into interactive page reply audio and play the interactive page reply audio.

[0111] Specifically, the interactive page content request is a request that readers actively initiate to ask questions on the picture book page. When there is no one to assist in reading the picture book, if the reader has any questions about the current picture book page, he or she can promptly ask questions to the provider through the instruction buttons on the picture book page, so that the system can truly become a reading companion to assist reading.

[0112] It can be understood that in order to more accurately understand the meaning of the question raised by the reader, the interactive page content request includes the page number of the picture book page where the question is asked, that is, the page number of the picture book page represented by the interactive page ID. The user interactive audio is the audio of the question input by the user into the system.

[0113] The method for assisting picture book reading provided in this embodiment is as follows: Fig. 9 As shown, the language understanding module is used to analyze the text of the picture book page, generate the interactive page content, and then convert it into interactive page audio for playback during the reading process, so as to complete the interactive reading of the picture book, enhance the participation of readers, especially children, in the process of reading picture books, help readers to deeply understand the knowledge carried by the picture book content, improve the ability to learn picture book knowledge by themselves, and save the time of manual reading.

[0114] It should be understood that the order of execution of the steps in the above embodiment does not necessarily mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present invention.

[0115] In one embodiment, a device for assisting picture book reading is provided, and the device for assisting picture book reading corresponds one-to-one with the method for assisting picture book reading in the above embodiment. Fig.10 As shown, the device for assisting picture book reading includes a picture book feature acquisition module 10, a picture book text extraction module 20, an interactive page content acquisition module 30, and an interactive page audio playback module 40. The functional modules are described in detail as follows:

[0116] The picture book feature acquisition module 10 is used to acquire real-time picture book features and match target picture books and target picture book pages based on the picture book database and the real-time picture book features.

[0117] The picture book text extraction module 20 is used to extract the picture book page text corresponding to the target picture book page based on the picture book text corresponding to the target picture book.

[0118] The interactive page content acquisition module 30 is used to analyze the text of the picture book page using the language processing module to acquire the interactive page content.

[0119] The interactive page audio playing module 40 is used to match the interactive page content based on the interactive content database. If the matching result is a failure, the interactive page content is converted into interactive page audio through the audio processing module and the interactive page audio is played.

[0120] Preferably, the picture book feature acquisition module 10 includes a real-time picture book page text acquisition submodule and a target picture book page acquisition submodule. The functional modules are described in detail as follows:

[0121] The submodule for obtaining real-time picture book page text is used to obtain picture book page text recognized in real time, picture book page images or picture book page videos taken in real time on the same picture book page to be recognized.

[0122] The target picture book page acquisition submodule is used to perform feature matching in the picture book database based on the picture book page text, picture book page image or picture book page video to acquire the corresponding target picture book and target picture book page.

[0123] Preferably, the module 30 for obtaining interactive page content includes a submodule for generating structured data and a submodule for generating interactive page content. The detailed description of each functional module is as follows:

[0124] The structured data generation submodule is used to process the picture book page text using the language understanding module to generate structured data.

[0125] The interactive page content generation submodule is used to extract information from structured data and generate interactive page content by combining the picture book knowledge base and the language generation module.

[0126] Preferably, the interactive page content generation submodule includes extracting word meaning relationship units, obtaining reserved word units and obtaining interactive page content units. The detailed description of each functional module is as follows:

[0127] Extract word meaning relationship units, which are used to extract word meaning relationships of keywords.

[0128] Obtain reserved word units for filtering keywords based on word meaning relationships and picture book knowledge bases, and obtain filtering results as reserved words.

[0129] The interactive page content unit is obtained to sort, aggregate and grammatically process the reserved words to obtain the interactive page content.

[0130] Preferably, the device for assisting picture book reading further includes a module for obtaining playback time and a module for stopping conversion of interactive page content. The detailed description of each functional module is as follows:

[0131] The module for obtaining the playback time is used to obtain the interactive page playback rate and the most recent playback time corresponding to the interactive page content based on the user ID if the matching result is successful.

[0132] The module for stopping conversion of interactive page content is used to stop converting interactive page content into interactive page audio if the interactive page playback rate is greater than the playback rate threshold or the most recent playback time is later than the playback time threshold.

[0133] Preferably, the playing interactive page audio module 40 includes a playing interactive page question submodule. The functional modules are described in detail as follows:

[0134] The submodule for playing interactive page questions is used to play interactive page questions and play interactive page answers after a preset question waiting time.

[0135] Preferably, the device for assisting picture book reading also includes a content request acquisition module, a user interactive text conversion module, a reply content acquisition module, and an interactive page reply audio playback module. The functional modules are described in detail as follows:

[0136] The content request acquisition module is used to obtain the interactive page content request, which includes the interactive page ID and user interactive audio.

[0137] The user interaction text conversion module is used to convert the user interaction audio into user interaction text.

[0138] The module for obtaining reply content is used to analyze the user interaction text and the picture book page corresponding to the interaction page ID using the language understanding module to obtain the reply content of the interaction page.

[0139] The interactive page reply audio playing module is used to convert the interactive page reply content into interactive page reply audio and play the interactive page reply audio.

[0140] For the specific definition of the device for assisting picture book reading, please refer to the definition of the method for assisting picture book reading above, which will not be repeated here. The various modules in the above-mentioned device for assisting picture book reading can be implemented in whole or in part by software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0141] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Fig.11 As shown. The computer device includes a processor, a memory, a network interface and a database connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used for data related to the method of assisting picture book reading. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a method of assisting picture book reading is implemented.

[0142] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the method for assisting picture book reading in the above embodiment is implemented, for example Figure 2Alternatively, when the processor executes the computer program, the functions of each module / unit of the device for assisting picture book reading in the above embodiment are realized, for example Fig.10 The functions of the modules 10 to 40 are shown in Figure 1. To avoid repetition, they will not be described in detail here.

[0143] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the method for assisting picture book reading in the above embodiment is implemented, for example Figure 2 Alternatively, when the computer program is executed by the processor, the functions of each module / unit in the device for assisting picture book reading in the above device embodiment are realized, for example Fig.10 The functions of the modules 10 to 40 are shown in Figure 1. To avoid repetition, they will not be described in detail here.

[0144] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments of the present application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0145] Those skilled in the art can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example. In actual applications, the above-mentioned functions can be distributed and completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.

[0146] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention, and should be included in the protection scope of the present invention.

Claims

1. A method for assisting picture book reading, characterized in that: include: Acquire real-time picture book features, and match target picture books and target picture book pages based on the picture book database and the real-time picture book features; Based on the picture book text corresponding to the target picture book, extract the picture book page text corresponding to the target picture book page; The language processing module is used to analyze the text of the picture book page to obtain the interactive page content, including: using the language understanding module to process the text of the picture book page to generate structured data; combining the picture book knowledge base and the language generation module to extract information from the structured data to generate the interactive page content; The picture book knowledge base is a knowledge graph library used to define the interactive content rules of picture books. The interactive content rules include at least one of the following: question rules for application scenarios, ways to propose interactive page content, the number of words in the interactive content, and the voice of the voice actor; Match the interactive page content based on the interactive content database. If the matching result is a failure, the interactive page content is converted into interactive page audio through the audio processing module and the interactive page audio is played; Structured data includes keywords; Combine the picture book knowledge base and language generation module to extract information from structured data and generate interactive page content, including: Extract the semantic relationship of keywords; Filter keywords based on word meaning relationship and picture book knowledge base, and obtain the filter results as reserved words; Sort, aggregate and grammaticalize reserved words to obtain interactive page content; The interactive content database includes user IDs; After matching the interactive page content based on the interactive content database, it also includes: If the matching result is successful, the interactive page play rate and the most recent play time corresponding to the interactive page content are obtained based on the user ID; If the interactive page play rate is greater than the play rate threshold or the most recent play time is later than the play time threshold, stop converting the interactive page content into interactive page audio.

2. The method for assisting picture book reading according to claim 1, characterized in that: The step of acquiring the real-time picture book feature and matching the target picture book and the target picture book page based on the picture book database and the real-time picture book feature includes: Obtaining picture book page text recognized in real time, picture book page image or picture book page video captured in real time on the same picture book page to be recognized; Based on the picture book page text, picture book page image or picture book page video, feature matching is performed in the picture book database to obtain the corresponding target picture book and the target picture book page.

3. The method for assisting picture book reading according to claim 1, characterized in that: The interactive page content includes interactive page questions and interactive page answers; Play the interactive page audio, including: Play the interactive page question, and play the interactive page answer after a preset question waiting time.

4. The method for assisting picture book reading according to claim 1, characterized in that: Also includes: Obtaining an interactive page content request, wherein the interactive page content request includes an interactive page ID and user interactive audio; Converting the user interaction audio into user interaction text; The language understanding module is used to analyze the user interaction text and the picture book page corresponding to the interaction page ID to obtain the interaction page reply content; Convert the interactive page reply content into interactive page reply audio and play the interactive page reply audio.

5. A device for assisting picture book reading, characterized in that: include: A picture book feature acquisition module is used to acquire real-time picture book features and match target picture books and target picture book pages based on the picture book database and real-time picture book features; A picture book text extraction module is used to extract the picture book page text corresponding to the target picture book page based on the picture book text corresponding to the target picture book; The interactive page content acquisition module is used to analyze the picture book page text using the language processing module to acquire the interactive page content, including: using the language understanding module to process the picture book page text to generate structured data; combining the picture book knowledge base and the language generation module to extract information from the structured data to generate interactive page content; The picture book knowledge base is a knowledge graph library used to define the interactive content rules of picture books. The interactive content rules include at least one of the following: question rules for application scenarios, ways to propose interactive page content, the number of words in the interactive content, and the voice of the voice actor; The interactive page audio playing module is used to match the interactive page content based on the interactive content database. If the matching result is a failure, the interactive page content is converted into interactive page audio through the audio processing module and the interactive page audio is played; Structured data includes keywords; Combine the picture book knowledge base and language generation module to extract information from structured data and generate interactive page content, including: Extract the semantic relationship of keywords; Filter keywords based on word meaning relationship and picture book knowledge base, and obtain the filter results as reserved words; Sort, aggregate and grammaticalize reserved words to obtain interactive page content; The interactive content database includes user IDs; After matching the interactive page content based on the interactive content database, it also includes: If the matching result is successful, the interactive page play rate and the most recent play time corresponding to the interactive page content are obtained based on the user ID; If the interactive page play rate is greater than the play rate threshold or the most recent play time is later than the play time threshold, stop converting the interactive page content into interactive page audio.

6. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the method for assisting picture book reading as described in any one of claims 1 to 4 is implemented.

7. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method for assisting picture book reading as claimed in any one of claims 1 to 4 is implemented.

Images