A smart reading system integrating AI voice dialogue function

Abstract

This application relates to the field of intelligent educational equipment technology and discloses an intelligent point-and-read system integrating AI voice dialogue functionality. The system includes a point-and-read recognition and content localization module for detecting user point-and-read operations on paper book pages and identifying the corresponding point-and-read content nodes; a reading semantic graph construction module for constructing a local reading semantic graph with the point-and-read content nodes as the central nodes and determining the association weights between nodes; a topic inference module for calculating candidate topic probabilities using a Bayesian topic inference model based on the current point-and-read node, historical point-and-read trajectory, and user voice question information, and determining the current dialogue topic by combining the association weights in the local reading semantic graph; and an AI dialogue module for generating a voice response based on the current dialogue topic. This invention can improve the accuracy and stability of AI dialogue topic recognition in point-and-read scenarios and enhance the interactive coherence during physical book reading.

Description

Technical Field

[0001] This application relates to the field of intelligent education equipment technology, specifically to an intelligent reading system that integrates AI voice dialogue functionality. Background Technology

[0002] With the development of artificial intelligence technology, some smart reading devices have introduced voice interaction capabilities on top of traditional reading books. Existing reading books typically identify the user's reading position by setting up matrix touch points, conductive circuits, or optical codes within the paper pages, and then play local audio or cloud content based on the identified position. In recent years, some reading systems have begun to synchronize reading position information to Large Language Model (LLM) systems, enabling users to ask questions about the currently read content via voice, thus upgrading from point-and-read playback to a "point-and-read + AI dialogue" interactive mode.

[0003] However, in actual reading, users' point-and-read behavior often exhibits both continuity and jumpiness. For example, they may continuously point to multiple locations within the same thematic area, switch back and forth between text descriptions and illustrations, jump between different paragraphs within the same chapter, or use pronouns such as "it," "this," or "what was just mentioned" when asking questions. If the system only updates the AI ​​dialogue context based on the latest point-and-read location, it can easily lead to frequent changes or incorrect switching of the dialogue topic, causing a discrepancy between the AI's response and the user's current reading content, thus affecting reading coherence and the immersive learning experience.

[0004] Therefore, it is necessary to provide an intelligent reading system that can maintain the stability of AI dialogue topics during point-reading and voice interaction. Summary of the Invention

[0005] The purpose of this application is to provide an intelligent reading system that integrates AI voice dialogue functionality to solve the problems mentioned in the background art.

[0006] According to a first aspect of this application, a smart reading system integrating AI voice dialogue functionality is provided, the system comprising: The point-reading recognition and content location module is used to detect the user's point-reading operation on the page of the paper book. It identifies the triggered point-reading coordinates by scanning rows and columns, and determines the corresponding point-reading content node according to the pre-established mapping relationship between point-reading coordinates and book content. The reading semantic graph construction module is used to extract related content nodes that have paragraph adjacency, image-text association or chapter affiliation with the central node, with the point-reading content node as the central node, and to construct a local reading semantic graph based on the central node and related content nodes, and to determine the association weight between nodes according to the semantic similarity and structural relationship between nodes. The topic inference module is used to calculate the topic probabilities of multiple candidate topics based on the current reading node, historical reading trajectory and user voice question information when a new reading operation or voice question is detected, and to determine the current dialogue topic by combining the association weights in the local reading semantic graph. The AI ​​dialogue module is used to construct an AI dialogue context based on the current dialogue topic and generate a voice response corresponding to the current reading object.

[0007] According to a second aspect of this application, a computer device is provided for use in the aforementioned system, including a memory, a processor, and a computer program stored in the memory and executable on the processor.

[0008] According to a third aspect of this application, a storage medium is provided for use in the aforementioned system, the storage medium storing a computer program.

[0009] According to a fourth aspect of this application, a computer program product is provided for use in the aforementioned system, comprising computer-readable code executable by a computer device.

[0010] Compared to existing technologies, this invention introduces a reading semantic graph construction mechanism and a Bayesian model-based topic inference mechanism into the intelligent point-and-read system. It comprehensively models the user's current point-and-read node, historical point-and-read trajectory, and voice question information, and corrects the probability of candidate topics by combining node association weights in the local reading semantic graph. This enables more accurate identification of the user's current reading topic in continuous point-and-read and voice interaction scenarios. Compared to methods that rely solely on single point-and-read location or voice content for topic judgment, this invention effectively reduces topic drift caused by adjacent page content, mixed text and images, or local accidental triggering. It improves the matching degree between the AI ​​dialogue context and the actual reading content, making the AI-generated voice response more relevant to the current reading object, thereby enhancing the interactive coherence, stability, and immersive learning experience during physical book reading. Attached Figure Description

[0011] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0012] Other features, objects, and advantages of this application will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings: Figure 1A schematic diagram of the structure of an intelligent reading system integrating AI voice dialogue function provided in an embodiment of this application; Figure 2 A schematic diagram of the structure of a point-and-read paper book page provided in an embodiment of this application; Figure 3 A schematic diagram illustrating the process of determining the association weights between nodes in the reading semantic graph construction module provided in this application embodiment; Figure 4 A schematic diagram illustrating the process of the topic inference module for determining the current dialogue topic in an embodiment of this application; Figure 5 This is a schematic diagram illustrating the process by which the topic inference module, provided in an embodiment of this application, obtains the corrected topic probability distribution. Detailed Implementation

[0013] The technical solution of this application will be further described below with reference to the accompanying drawings and specific embodiments. It should be understood that the embodiments described in this application are only used to explain the technical solution of this application, and are not intended to limit the scope of protection of this application.

[0014] This invention relates to an intelligent point-and-read system integrating AI voice dialogue functionality. This system combines physical book reading with AI voice interaction, enabling real-time sensing of the user's point-and-read position and reading behavior during physical book reading. It then combines this with voice input to understand and respond to the user's current focus. Unlike traditional point-and-read devices that can only play preset voice prompts at single trigger points, this application utilizes semantic graph modeling and topic inference mechanisms to comprehensively analyze the user's current reading content, historical reading trajectory, and voice question information. This allows it to identify the user's true focus during continuous reading and generate a voice response matching the current reading content. Through this approach, the system not only achieves the basic function of immediate voice output upon point-and-read but also supports multi-turn voice dialogues around the reading content, thereby improving the system's comprehension and interactive coherence in complex text-and-image layouts.

[0015] like Figure 1 As shown in the illustration, this application discloses an intelligent point-and-read system 100 integrating AI voice dialogue functionality. This system can be applied to scenarios such as children's early reading, intelligent educational terminals, and interactive learning devices, enabling the fusion of point-and-read functionality and AI voice dialogue during the reading of physical paper books. The system includes: The point-reading recognition and content positioning module 10 is used to detect the user's point-reading operation on the page of the paper book, identify the triggered point-reading coordinates through row and column scanning, and determine the corresponding point-reading content node according to the pre-established mapping relationship between the point-reading coordinates and the book content. like Figure 2As shown, the row and column scanning is achieved through a matrix of row and column contact points set in the interlayer of the paper book pages. As an example, the matrix of row and column contact points can be embedded in the interlayer of the pages using flexible conductive lines, conductive ink lines, or thin film contact structures to ensure that the paper pages can still sense the position of the user's pointing actions while maintaining normal page-turning performance.

[0016] The row and column contact matrix is ​​controlled by a microprocessor, which scans the row and column lines row by row and column by column to detect the contact continuity state, thereby determining the triggered reading coordinates. The microprocessor drives each row line sequentially according to a preset scanning cycle and simultaneously detects the feedback level of each column line; when a row line and a column line are detected to form a continuity response within the same scanning cycle, the trigger coordinates corresponding to the user's current reading can be determined accordingly.

[0017] After identifying the point-and-read coordinates, the corresponding point-and-read content node is further determined based on a pre-established mapping table between point-and-read coordinates and book content. This mapping table can be pre-generated during the book production stage, incorporating page layout information, illustration partitioning information, and chapter structure information, or it can be imported from the backend configuration file during system initialization; details will not be elaborated further. Through this mapping relationship, after determining the point-and-read coordinates, the corresponding point-and-read content node can be quickly located.

[0018] The interactive content nodes include at least one of the following: text paragraph nodes, illustration area nodes, caption nodes, and chapter nodes. Specifically, text paragraph nodes can correspond to a paragraph of text, a semantically complete sentence group, or a knowledge fragment; illustration area nodes can correspond to a local area, object outline, or labeled area within an illustration; caption nodes can correspond to explanatory text below the illustration, numbered explanations, or illustration annotations; and chapter nodes can correspond to page titles, section titles, or chapter identifiers for a specific knowledge unit.

[0019] The reading semantic graph construction module 20 is used to extract related content nodes that have paragraph adjacency, image-text association or chapter affiliation with the central node, with the point-reading content node as the central node, and to construct a local reading semantic graph based on the central node and related content nodes, and to determine the association weight between nodes according to the semantic similarity and structural relationship between nodes. The structural relationships in the local reading semantic graph include at least one of the following: paragraph adjacency, text-image association, and chapter affiliation. Specifically, paragraph adjacency indicates whether two content nodes are adjacent in the page text structure, belong to the same paragraph, or form a sequential relationship; text-image association indicates whether there is a corresponding explanation, local annotation, or object description relationship between a text paragraph node and an illustration area node or caption node; and chapter affiliation indicates whether multiple nodes belong to the same chapter, the same thematic unit, or the same page knowledge block. Based on these structural relationships, this invention identifies content units truly relevant to the current reading object by combining the page content organization method, rather than simply expanding content based on spatial location.

[0020] Reference Figure 3 As shown, this embodiment further provides a specific implementation method for constructing a local reading semantic graph and determining the association weights between nodes.

[0021] In some embodiments, a local reading semantic graph is constructed based on the central node and associated content nodes, and the association weights between nodes are determined according to the semantic similarity and structural relationships between nodes, including: S21. Based on the central node, extract multiple candidate content nodes within a preset spatial range, and divide the candidate content nodes into accompanying nodes and competing nodes according to the relative positional relationship, content continuity relationship and text-image pointing relationship between the candidate content nodes and the central node. Specifically, taking the currently clickable content node as the center, multiple candidate content nodes within a preset spatial range are extracted based on page layout information (such as pre-marked text block coordinates, illustration area boundaries, etc.). The preset spatial range can be a set of content nodes centered on the area of ​​the central node, extending outwards by a certain physical distance (e.g., 5 cm) or a certain semantic level (e.g., the same paragraph, the same illustration area). It should be understood that this spatial range is an adaptive range combined with page structure information; for example, in the same double-page layout, content nodes belonging to the same visual block (e.g., text box, illustration box) as the central node are preferentially extracted.

[0022] After extracting candidate content nodes, further fine-grained segmentation is performed based on the following three types of relationships: (1) Relative positional relationship on the page: This includes whether the candidate node is above, below, to the left, to the right, nested inside, or surrounded by the central node. For example, when the central node is an illustration area, the caption node located directly below it has a "below" positional relationship; when the central node is a text paragraph, the sidebar annotation node located to its right has a "right" positional relationship.

[0023] (2) Content continuity relationship: This is used to determine whether the candidate node has semantic continuity with the central node, such as "title - body text", "body text - supplementary explanation", "question - answer", "illustration body - local detail explanation", etc. This type of relationship can be obtained by analyzing the hierarchy and order of the nodes in the book's content structure tree. For example, two paragraph nodes arranged consecutively under the same title have a content continuity relationship.

[0024] (3) Visual Relationship: This refers to whether there is an explicit visual guide between the candidate node and the central node, such as an arrow connecting the nodes, a numbered index, a label line, a caption reference, or multiple appearances of the same object. For example, when the central node is an illustration area, and that area contains the number " Figure 3 Then the page number is " Figure 3 The caption indicates that the node forms a text-image relationship with the central node.

[0025] Based on the above three types of relationships, candidate content nodes are divided into the following two categories: Accompanying nodes: These are nodes that describe the same object, knowledge unit, or reading focus as the central node. The criteria for classification are: there must be at least one strong correlation between the candidate node and the central node, including text-image pointing relationships (such as arrows, numbered indexes, and caption references), content continuity relationships (such as "title—body text," "body text—supplementary explanation"), and relative layout relationships (such as below, internal nesting, and other adjacent layouts). For example, when the central node is an illustration of a whale spouting water, the text paragraph below it titled "Principles of Whale Spouting Water," the "spout hole" area marked in the illustration, and the arrow pointing to the spout hole are all classified as accompanying nodes because they have clear text-image pointing or content continuity relationships.

[0026] Competing nodes: These are nodes that are spatially adjacent to the central node but actually belong to other independent thematic units, sidebar supplementary content, parallel columns, or adjacent knowledge blocks. The criteria for this classification are: candidate nodes lack the aforementioned strong correlation with the central node, or although there is some spatial proximity, they semantically or structurally point to different themes. For example, the "dolphin leaping" illustration and its explanatory text on the right side of the same page are classified as competing nodes because they lack a direct connection to the central node "whale spouting water" and belong to an independent thematic unit. Although competing nodes may be adjacent on the page, if not suppressed, they can easily interfere with the judgment of the current reading theme.

[0027] It should be understood that by dividing adjacent nodes into competing nodes, we can initially filter out neighboring content that is irrelevant to the current reading topic, which can be used to build a stable semantic graph in the future.

[0028] S22, establish a local reading semantic graph based on the central node and the accompanying nodes, and apply suppression weights to the associated edges corresponding to the competing nodes; After completing the node partitioning, the central node is used as the core, and all its accompanying nodes are treated as graph nodes. Initial connection edges are established based on the original structural relationships between nodes (such as paragraph adjacency, text-image association, and chapter affiliation), thus forming the preliminary structure of a local reading semantic graph. This graph is an undirected weighted graph (or a directed weighted graph, depending on directional requirements), where nodes represent content units and edges represent structural relationships between nodes.

[0029] For competing nodes, any possible connections between them and the central or accompanying nodes are retained, but suppression weights are applied to these connections. It should be understood that by introducing suppression weights: on the one hand, retaining the weak connections between competing nodes can provide a structural basis for smooth topic switching when the user's subsequent reading behavior turns to that competing topic; on the other hand, by suppressing the connections, the influence of competing nodes in the current topic inference is reduced, preventing them from interfering with topic stability.

[0030] Preferably, the specific value of the suppression weight can be determined based on the degree of conflict between the competing node and the central node. For example, if the competing node and the central node are highly unrelated semantically (e.g., calculated through semantic similarity), and their page positions are adjacent but belong to different sections, a higher suppression coefficient (e.g., 0.1) can be set; if the competing node and the central node have some semantic connection but belong to different topic blocks, a medium suppression coefficient (e.g., 0.3) can be set. The suppression weight, for example, acts on the original edge weight in the form of a multiplicative factor, that is, the original edge weight is multiplied by the suppression factor to obtain the final edge weight.

[0031] S23, calculate the semantic association strength, structural support strength and reading interference strength between graph nodes respectively, and determine the association weight between nodes based on the semantic association strength, the structural support strength and the reading interference strength.

[0032] After the local reading semantic graph is constructed, a quantized weight is assigned to each edge in the graph. This weight comprehensively reflects the actual degree of association between two nodes during the reading process. Therefore, this embodiment calculates the following three strength indices: Semantic association strength: This measures the degree of similarity between two nodes in terms of content expression. For text-readable nodes (such as text paragraphs and captions), their semantic vectors can be extracted using pre-trained language models (such as BERT and RoBERTa), and the cosine similarity between the vectors can be calculated. For inset region nodes, their associated label text, region description text, and surrounding explanatory text can be concatenated and encoded, or a multimodal model (such as CLIP) can be used to directly calculate the semantic matching degree between the image region and the text.

[0033] Let the semantic vectors of node i and node j be respectively and Then the semantic association strength .

[0034] Structural continuity strength: This measures the degree of continuity between two nodes in the book's content organization structure. This strength can be calculated based on the distance between the nodes in the content tree and the type of their relationship. For example, consecutive paragraphs within the same chapter and under the same heading have high structural continuity strength; nodes across chapters have lower structural continuity strength. Furthermore, specific structural relationships (such as "heading-body text," "body text-caption," "illustration-caption") can assign higher structural continuity weights.

[0035] Specifically, baseline strength values ​​for different types of relationships can be predefined. Then, combined with the shortest path length of the node in the content tree Attenuation: ,in λ is the decay factor (0 < λ < 1). This represents the most direct structural relationship type between nodes i and j.

[0036] Reading interference intensity: This measures the degree of disturbance that a node may cause to the identification of the current reading topic. This intensity mainly targets competing nodes and their associated paths.

[0037] The intensity of reading interference can be comprehensively evaluated based on the following factors: whether the node is a competing node, the semantic difference between the node and the central node, and whether the area where the node is located belongs to the sidebar or parallel content. For accompanying nodes, their reading interference intensity is set to 0 or a small value; for competing nodes, their interference intensity can be predicted based on the above factors using rules or lightweight models. Interference Intensity The value range is generally [0,1]. The larger the value, the stronger the interference of the edge on the stability of the topic.

[0038] The association weights between nodes are determined based on the three strength indicators mentioned above. An exemplary calculation method is a weighted linear combination: ; in, For adjustable hyperparameters, satisfying (Or adjust as needed). In this way, semantic association and structural inheritance contribute positively to the association weight, while reading interference plays a suppressive role, thereby ensuring that the local reading semantic graph can effectively aggregate relevant nodes and actively suppress interference paths that may cause topic drift.

[0039] It should be noted that if a certain edge has already been given a suppression weight in a previous step (such as the processing of competing node edges in step S22), then the final weight still needs to be multiplied by the suppression factor: ,in The inhibition coefficient determined in step S22 (less than 1).

[0040] The topic inference module 30 is used to calculate the topic probability of multiple candidate topics based on the current reading node, historical reading trajectory and user voice question information when a new reading operation or voice question is detected, and to determine the current dialogue topic by combining the association weights in the local reading semantic graph. In real-world reading scenarios, users' point-and-read behavior often exhibits both continuity (such as repeatedly pointing within the same topic area) and jumpiness (such as switching between illustrations and text). Furthermore, voice questions frequently contain pronouns (such as "it," "this," and "what was just mentioned"). If the dialogue topic is updated solely based on the latest point-and-read position, it can easily lead to frequent topic drift. Therefore, this invention designs a topic probability calculation framework based on Bayesian inference and incorporates posterior correction using the reading semantic graph structure, thereby achieving stable recognition of the current dialogue topic.

[0041] In some embodiments, such as Figure 4 As shown, based on the current reading node, historical reading trajectory, and user voice question information, a Bayesian topic inference model is used to calculate the topic probabilities of multiple candidate topics, and the current dialogue topic is determined by combining the association weights in the local reading semantic graph, including: S31. Based on the node association relationship of the current reading node in the local reading semantic graph, the topic distribution status corresponding to the historical reading trajectory, and the topic pointing information in the user's voice question, determine multiple candidate topics; Specifically, a candidate topic set is generated by combining the following three types of information: The node relationships of the current reading node in the local reading semantic graph: Centered on the current reading node, the topics corresponding to the accompanying nodes that are directly connected to the current node or reachable by a short path are extracted through the local reading semantic graph generated by the reading semantic graph construction module 20. For example, if the current reading node is the illustration area of ​​"whale spouting water", its directly associated accompanying nodes in the graph include the text paragraph node of "whale spouting water principle" and the local annotation node of "spout hole", etc. The topics corresponding to these nodes (such as "whale breathing method" and "explanation of spouting water phenomenon") constitute part of the candidate topics.

[0042] The distribution of topics corresponding to historical reading patterns: Maintain a sliding window (e.g., the last 5 reading records), count the frequency or weighted frequency of each topic within the window, and include high-frequency topics as candidate topics. For example, if a user has continuously read the "whale living habits" related node in the last 3 readings, then this topic should be retained as a candidate topic to reflect reading inertia.

[0043] Topic-related information in user voice questions: The speech recognition and natural language understanding modules analyze whether user questions contain explicit object nouns (e.g., "whale"), attribute words (e.g., "spraying water"), pronouns (e.g., "it," "this"), or action descriptions. If an explicit object noun is detected, the corresponding topic is directly added to the candidate set. If a pronoun is detected, the referent is inferred from the historical reading trajectory, and the topic to which that object belongs is added to the candidate set. For example, if a user asks "Why can it spray water?", and through pronoun resolution it is determined that "it" likely refers to the recently read "whale," then topics related to "whale" are added to the candidate set.

[0044] Using the above method, a set containing multiple candidate topics is constructed. These topics may come from direct mapping of the current reading, or from historical continuation or explicit voice reference, ensuring that subsequent probability calculations cover all reasonable topic options.

[0045] S32, For each candidate topic, calculate the topic prior probability, the topic likelihood probability under the current observation conditions, and the topic retention correction amount, and obtain the initial topic probability of each candidate topic based on the topic prior probability, the topic likelihood probability, and the topic retention correction amount; Specifically, a Bayesian framework is used to calculate the initial probability of each candidate topic. Let the current observation data be... Including the current reading node Local reading semantic graph Historical reading trajectory and voice questions For candidate topics Its initial probability can be expressed as: ; The specific calculation methods for each component are as follows: Prior probability of the topic : Reflects the situation when there is no current observation information, the theme The probability of a topic becoming a subject of conversation is calculated based on historical reading patterns, using an exponentially decaying weighted approach, meaning that reading actions more recent than the current moment contribute more: ; in, For historical read counts, The attenuation factor is taken as 0.8~0.9. For indicator functions, Indicates the first The next click reads the corresponding topic; Indicates the first 10 candidate topics It is an index variable used to iterate through all possible candidate topics (i.e., the entire set of topics). The above formula makes recent reading contribute more to the prior probability, which is consistent with the characteristic of thematic continuity during reading.

[0046] Topic Likelihood Probability : indicates that the current real topic is assumed to be Under these conditions, the current data is observed. The likelihood probability can be decomposed into the product (or logarithmic sum) of multiple observation components, for example: ; in: The current reading node With the topic The matching degree can be determined by the node. With the topic The calculation of the association strength of corresponding nodes in the reading semantic graph, for example, if It is itself a topic If the set of nodes is empty, then the value is 1; otherwise, it is empty. With the topic Maximum semantic similarity between representative nodes.

[0047] Questions to be asked via voice With the topic The relevance can be calculated. semantic vectors and topics The semantic similarity of the text descriptions is obtained, or quantified through keyword matching.

[0048] Semantic graph for local reading With the topic The structural matching degree can be based on the topics belonging to the graph structure. The node ratio or subgraph connectivity calculation.

[0049] Maintain correction amount Used to enhance the discussion on topics from the previous round of dialogue. Candidate topics are presented in a continuous manner to avoid accidental topic switching due to brief reading deviations. Definition: ; in The preset gain coefficient (e.g., 0.2). The association weights of the two topics in the reading semantic graph (if) If the value is 1, then take 1; otherwise, take the normalized value of the cumulative weight of the shortest path between the representative nodes of the two topics. This is an indicator function that takes the value 1 if a pronoun is present in the voice question, and 0 otherwise. This design encourages users to stay on the original topic when they ask questions using pronouns.

[0050] Multiplying (or combining appropriately) the above three terms and normalizing them yields the initial probability distribution for each candidate topic. .

[0051] S33, combine the node association weights between the nodes corresponding to each candidate topic and the current reading node, perform a posteriori correction on the initial topic probability, and obtain the corrected topic probability distribution; While correcting the initial topic probability solely based on the association weights between nodes can reflect the relevance between candidate topics and the current reading node to some extent, this method remains a single-point correction mechanism based on local node relationships. It cannot distinguish whether the association between the candidate topic and the current reading node is established through a stable semantic path or through a disruptive path containing competing nodes. In actual paper book reading scenarios, due to the complex layout structures such as mixed text and images, sidebar descriptions, parallel knowledge blocks, or adjacent topic content, some competing nodes, although spatially adjacent to the current reading node, may not correspond to content that belongs to the current reading object. Furthermore, their high local association weights often amplify the initial topic probability, leading to a bias in topic inference towards non-target topics, resulting in topic drift.

[0052] Furthermore, since node association weights only reflect the direct relationship between two nodes and do not consider the multi-hop transmission path structure from the current reading node to the candidate topic node, it is difficult to determine whether the association of a candidate topic is mainly achieved through a stable path composed of a central node and accompanying nodes, or mainly through a path containing competing nodes, when multiple association paths exist. Especially in children's books, illustrated books, or knowledge card-style layouts, different topics are densely distributed in space. If association paths are not differentiated, it is easy to misjudge candidate topics established through interfering paths as the current dialogue topic.

[0053] Based on the above reasons, this step further designs a graph path-based correction mechanism to obtain the corrected topic probability distribution, as follows: In some embodiments, such as Figure 5As shown, by combining the node association weights between the nodes corresponding to each candidate topic and the current reading node, the initial topic probability is posteriorly corrected to obtain the corrected topic probability distribution, including: S331, extract at least one associated path between the node corresponding to each candidate topic and the current reading node, and calculate the cumulative weight of each associated path; and detect whether each associated path contains a suppression edge corresponding to a competing node, and determine the path suppression coefficient based on the distribution of the suppression edge. For each candidate topic Searching for the current reading node in the local reading semantic graph. To the topic Representative nodes All paths (such as the core content nodes of this topic). The set of paths is denoted as... For each path Calculate its path cumulative weight For example, it could be the product of the weights of each edge on the path: ; in For the edge The associated weights are determined by step S23. If a product method is used, the cumulative weight of the path reflects the intensity of the information transmitted along the path; long paths or low-weight edges will decay rapidly.

[0054] Meanwhile, the detection path Does the path contain a suppressing edge corresponding to a competing node? A suppressing edge is an edge for which a suppressing weight has been applied to the edges associated with competing nodes in step S22; its weight is typically much less than 1. Let the path... The number of upper suppression edges is Then the path inhibition coefficient It can be defined as: ; in This is a suppression attenuation factor (e.g., 0.1~0.5). It should be understood that this path suppression coefficient significantly reduces the overall contribution of paths containing suppression edges, thereby reducing the interference of competing nodes on topic judgment.

[0055] S332, calculate the path correction factor corresponding to each candidate topic based on the cumulative path weight and the path suppression coefficient, and perform posterior correction on the initial topic probability of each candidate topic according to the path correction factor, and normalize the corrected topic probability to obtain the corrected topic probability distribution.

[0056] By combining the contributions of all paths, candidate topics are obtained. Path correction factor : ; It should be understood that this correction factor reflects the overall graph structural association strength from the current reading node to the candidate topic representative node, taking into account both the cumulative weight of the path (semantic and structural strength) and the interference of suppressed edges. Then, the correction factor is applied to the initial probability: ; in The adjustment factor (e.g., 0.5~1.0) controls the strength of the graph structure correction. It should be understood that this process increases the probability of candidate topics that have strong graph connections to the current reading node and are free from interference, while relatively reducing the probability of topics that mainly rely on weak paths or contain suppressing edges.

[0057] Finally, for all candidate topics Normalization is performed to obtain the corrected topic probability distribution. .

[0058] In this embodiment, all reachable paths between the current reading node and the corresponding candidate topic nodes are extracted. A cumulative weight (reflecting the overall semantic and structural transmission strength of the path) is calculated for each path, and the presence of suppression edges corresponding to competing nodes in the path is detected to determine the path suppression coefficient (reflecting the degree of interference on the path). By weighted summarization of multiple paths, it is possible to effectively distinguish between "stable semantic association paths built through accompanying nodes" and "interfering association paths introduced by competing nodes." This increases the weight of candidate topics established through stable reading paths during probability correction and suppresses the probability of candidate topics introduced through competing node paths, thereby improving the stability and accuracy of topic inference.

[0059] S34, determine the target topic based on the corrected topic probability distribution, and set the target topic as the current dialogue topic.

[0060] Appropriate decision-making strategies can be selected based on actual application needs. For example, the topic with the highest probability can be selected as the current dialogue topic. ; If the maximum probability is lower than a preset threshold (e.g., 0.6), the topic is considered ambiguous. In this case, the previous topic can be retained or the user can be proactively asked for clarification. Furthermore, if multiple topics have similar probabilities and are all above the threshold, multiple topics can be simultaneously used as context input to the AI ​​dialogue module to handle potentially complex questions from the user.

[0061] AI dialogue module 40 is used to construct an AI dialogue context based on the current dialogue topic and generate a voice response corresponding to the current reading object.

[0062] After the topic inference module 30 determines the current dialogue topic, this module organizes the current dialogue topic together with the corresponding point-reading content node, the high-weight accompanying node information in the local reading semantic graph, the historical dialogue round information, and the current user's voice question into a structured AI dialogue context, which is used as the input of the Large Language Model (LLM) to generate response content that fits the current reading scenario.

[0063] Specifically, the construction of an AI dialogue context includes the following information elements: Current dialogue topic: The target topic output by the topic inference module 30, such as "how whales breathe". This topic serves as the core semantic anchor of the dialogue, ensuring that the AI's response focuses on the knowledge area that the user is currently interested in.

[0064] Point-and-read content node: The specific content node corresponding to the current point-and-read operation, such as the "whale spouting water" illustration node or its local area node. This node provides the precise object to which the user's current physical point-and-read action is pointing, helping AI understand the concrete context of the user's question.

[0065] High-weighted companion nodes in the local reading semantic graph: From the local reading semantic graph generated by the reading semantic graph construction module 20, companion nodes with high correlation weights to the current reading node (such as semantic correlation strength and structural coherence strength exceeding a preset threshold after comprehensive weighting) are selected, such as the text paragraph node "whale spouting principle" and the "spout hole" annotation node. The text content corresponding to these nodes can be incorporated into the context as supplementary knowledge, enabling AI to refer to relevant information around the current page and generate richer explanations.

[0066] Historical Dialogue Rounds: The system maintains a dialogue history cache, recording user questions and AI responses from the most recent rounds (e.g., 3-5 rounds). The introduction of historical dialogues allows the AI ​​to understand the evolution of the conversation, which is crucial for resolving puns and probing follow-up questions. For example, if the previous round discussed "the respiratory organs of a whale," and the user asks "How high can it spray at once?" in this round, the AI ​​can infer from the historical context that "it" still refers to a whale.

[0067] Current user voice question: The user's question text after being converted by speech recognition (ASR), such as "Why can it spray water?".

[0068] When organizing the context, the above information is formatted according to a preset prompt template to form input suitable for a large language model. For example, the prompt template structure is as follows: [Current Reading Topic] {Topic Description} [Current Page Content] {Text / Description Corresponding to the Click Node} [Related Knowledge] {Text content of high-weight accompanying nodes (presented in list format)} [Dialogue History] {Historical Q&A pairs (arranged in the format "User: ... Assistant: ...")} [User Question] {Current user's voice issue} Please generate a reply based on the above information: Through the structured contextual organization described above, the AI ​​model receives a complete scene description that includes the reading focus, related knowledge, historical context, and current problem, thus enabling it to understand the user's intent more accurately.

[0069] Preferably, the AI ​​dialogue module 40 can invoke cloud-based large language model services (such as the GPT series, Wenxin Yiyan, Tongyi Qianwen, etc.) or locally deployed lightweight language models (such as edge models like Llama and ChatGLM) for inference. It sends the constructed context text to the model service via an API interface and receives the generated response text. Regarding model selection, a balance can be struck based on device computing power, network conditions, and application scenarios: cloud-based models possess stronger semantic understanding and knowledge reserves, suitable for in-depth solutions to complex questions; local models offer faster response times, require no network connection, and are suitable for basic question answering and offline scenarios.

[0070] To improve the accuracy and security of responses, role settings and constraint instructions can be added before model generation. For example, a system prompt could be set: "You are a children's reading assistant. Please answer questions about the current book's content in simple and easy-to-understand language. If the question is outside the scope of the book, please politely guide the user back to the book's content." Such prompts help control the response style and content boundaries.

[0071] After receiving the reply text, the AI ​​dialogue module 40 further calls the text-to-speech (TTS) module to convert the text into a natural and fluent speech signal, which is then output through a speaker. The speech synthesis can employ deep learning-based emotional speech synthesis technology, adjusting the tone according to the contextual emotions (such as encouragement, explanation, and questioning) to enhance the naturalness and friendliness of the interaction.

[0072] To more clearly illustrate the workflow of the AI ​​dialogue module, the following explanation uses specific examples: A user is reading a point-and-click book about marine animals. The current page features an illustration of a whale spouting water on the left and a dolphin leaping on the right. The user first clicks on the whale spouting water illustration. The point-and-click recognition and content location module 10 determines the current node as the "whale spouting water illustration area." The semantic graph construction module 20 extracts accompanying nodes, including the text paragraph on "the principle of whale spouting water" (located below the illustration) and a partial annotation of the "spout." The topic inference module 30, combining previous point-and-click readings (previously reading content related to "whale breathing") and the current point-and-click reading, determines the current topic of the conversation to be "how whales breathe." The user then asks, "Why can it spout water?" AI dialogue module 40 constructs the following context: Current topic of conversation: How whales breathe Interactive content node: Whale spouting water illustration area Related information: Text paragraph on the principle of whale spouting water ("Whales breathe through the spout on their heads; the jet of water they spout is actually seawater and mist exhaled during their exhalation..."), spout labeling. Dialogue History: [Previous User: "What do whales use to breathe?" Previous Assistant: "Whales breathe with lungs; they have blowholes on the top of their heads."] User question: Why can it spray water? Inputting this context into a large language model, which combines previously discussed information about "fumaroles" with the current illustration and text description, the model correctly understands that "it" refers to a whale and that "spraying water" is related to the breathing process, generating the following response text: "Whales spout water to breathe! They have a special blowhole on their heads. When a whale surfaces, it exhales forcefully with its lungs, and the exhaled air carries seawater out, forming the water column we see. The blowhole in the illustration you just clicked on is where whales breathe."

[0073] Finally, the speech synthesis module converts the above text into speech output, enabling natural dialogue interaction with the user.

[0074] Through the aforementioned context construction and model invocation process, the AI ​​dialogue module 40 can fully utilize the topic recognition results and semantic graph information provided by the preceding modules to generate a highly relevant and context-coherent voice response. Since the current dialogue topic has been stably controlled by the reading semantic graph model and the Bayesian topic inference model, the generated voice response can accurately parse pronouns and continue the historical topic, effectively avoiding irrelevant answers or abrupt topic jumps caused by jumping between reading positions. This significantly improves the interactive coherence and immersive learning experience during physical book reading.

[0075] In some embodiments, the AI ​​dialogue module 40 can also dynamically adjust the dialogue strategy based on user feedback (such as repeated questions or explicit corrections), or guide users to explore the current topic in depth by proactively asking questions, such as adding at the end of the reply: "Do you want to know how high a whale can spray water at once?" to stimulate further interaction.

[0076] This application also provides a computer device applied to the aforementioned system, including a memory, a processor, and a computer program stored in the memory and executable on the processor.

[0077] Computer equipment can be servers, personal computers, mobile terminals, or embedded devices, and its internal structure may include, but is not limited to, processors, memory, communication interfaces, and buses. A processor is an integrated circuit chip with signal processing capabilities; it can be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices. Memory can be volatile or non-volatile, including but not limited to random access memory (RAM), read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory.

[0078] This application also provides a storage medium for use in the aforementioned system, wherein a computer program is stored in the storage medium.

[0079] The storage medium can be any form of computer-readable storage medium, including but not limited to: magnetic storage media (such as floppy disks, hard disks, magnetic tapes), optical storage media (such as optical discs, CD-ROMs, DVDs), semiconductor storage media (such as solid-state drives, USB flash drives, read-only memory, random access memory), and any other tangible medium capable of storing computer programs. The storage medium can be part of the computer device or a separate external storage device.

[0080] This application also provides a computer program product applied to the aforementioned system, including computer-readable code that can be run by a computer device.

[0081] This computer program product can be distributed in software form, such as stored on a storage medium, or transmitted as a signal via a communication network (such as the Internet). The code in the product can be in the form of source code, object code, or intermediate code, as long as it can be recognized and executed by a computer device. By running this computer program product, the computer device can implement the technical solution proposed in this application and achieve the corresponding technical effects.

[0082] The above description represents the preferred embodiments of the present invention. It should be noted that, for those skilled in the art, various improvements and modifications can be made without departing from the principles of the present invention, and these improvements and modifications are also considered to be within the scope of protection of the present invention.

Claims

1. A smart reading system integrating AI voice dialogue function, characterized in that, The system includes: The point-reading recognition and content location module is used to detect the user's point-reading operation on the page of the paper book. It identifies the triggered point-reading coordinates by scanning rows and columns, and determines the corresponding point-reading content node according to the pre-established mapping relationship between point-reading coordinates and book content. The reading semantic graph construction module is used to extract related content nodes that have paragraph adjacency, image-text association or chapter affiliation with the central node, with the point-reading content node as the central node, and to construct a local reading semantic graph based on the central node and related content nodes, and to determine the association weight between nodes according to the semantic similarity and structural relationship between nodes. The topic inference module is used to calculate the topic probabilities of multiple candidate topics based on the current reading node, historical reading trajectory and user voice question information when a new reading operation or voice question is detected, and to determine the current dialogue topic by combining the association weights in the local reading semantic graph. The AI ​​dialogue module is used to construct an AI dialogue context based on the current dialogue topic and generate a voice response corresponding to the current reading object; A local reading semantic graph is constructed based on the central node and related content nodes. The association weights between nodes are determined according to the semantic similarity and structural relationships between nodes, including: S21. Based on the central node, extract multiple candidate content nodes within a preset spatial range, and divide the candidate content nodes into accompanying nodes and competing nodes according to the relative positional relationship, content continuity relationship and text-image pointing relationship between the candidate content nodes and the central node. S22, establish a local reading semantic graph based on the central node and the accompanying nodes, and apply suppression weights to the associated edges corresponding to the competing nodes; S23, calculate the semantic association strength, structural support strength and reading interference strength between graph nodes respectively, and determine the association weight between nodes based on the semantic association strength, the structural support strength and the reading interference strength.

2. The intelligent reading system integrating AI voice dialogue function according to claim 1, characterized in that, The row and column scanning is achieved through a row and column contact matrix set in the interlayer of the paper book pages; the row and column contact matrix is ​​controlled by a microprocessor to scan the row and column lines row by row and column by column to detect the contact conduction status, thereby determining the triggered reading coordinates.

3. The intelligent reading system integrating AI voice dialogue function according to claim 1, characterized in that, The interactive content nodes include at least one of the following: text paragraph nodes, illustration area nodes, caption nodes, and chapter nodes.

4. The intelligent reading system integrating AI voice dialogue function according to claim 1, characterized in that, The structural relationships in the local reading semantic graph include at least one of the following: paragraph adjacency relationship, image-text association relationship, and chapter affiliation relationship.

5. The intelligent reading system integrating AI voice dialogue function according to claim 1, characterized in that, Based on the current reading node, historical reading trajectory, and user voice question information, a Bayesian topic inference model is used to calculate the topic probabilities of multiple candidate topics. These probabilities are then combined with the association weights in the local reading semantic graph to determine the current dialogue topic, including: S31. Based on the node association relationship of the current reading node in the local reading semantic graph, the topic distribution status corresponding to the historical reading trajectory, and the topic pointing information in the user's voice question, determine multiple candidate topics; S32, For each candidate topic, calculate the topic prior probability, the topic likelihood probability under the current observation conditions, and the topic retention correction amount, and obtain the initial topic probability of each candidate topic based on the topic prior probability, the topic likelihood probability, and the topic retention correction amount; S33, combine the node association weights between the nodes corresponding to each candidate topic and the current reading node, perform a posteriori correction on the initial topic probability, and obtain the corrected topic probability distribution; S34, determine the target topic based on the corrected topic probability distribution, and set the target topic as the current dialogue topic.

6. The intelligent reading system integrating AI voice dialogue function according to claim 5, characterized in that, By combining the node association weights between the nodes corresponding to each candidate topic and the current reading node, the initial topic probability is posteriorly corrected to obtain the corrected topic probability distribution, including: S331, extract at least one associated path between the node corresponding to each candidate topic and the current reading node, and calculate the cumulative weight of each associated path; and detect whether each associated path contains a suppression edge corresponding to a competing node, and determine the path suppression coefficient based on the distribution of the suppression edge. S332, calculate the path correction factor corresponding to each candidate topic based on the cumulative path weight and the path suppression coefficient, and perform posterior correction on the initial topic probability of each candidate topic according to the path correction factor, and normalize the corrected topic probability to obtain the corrected topic probability distribution.

Images