Digital textbook intelligent proofreading and content error correction system based on deep learning

Abstract

The invention discloses a digital textbook intelligent proofreading and content error correction system based on deep learning, and relates to the technical field of digital textbook intelligent correction. Comprising a multi-modal analysis module which is used for analyzing a digital textbook source file, separating and extracting text and non-text media contents, and generating a unified document object model containing semantic association; the knowledge management module is used for acquiring original data from a multi-source heterogeneous knowledge source and constructing a cross-domain knowledge graph; and driving automatic synchronization and evolution of the cross-domain knowledge graph based on an incremental learning mechanism. According to the method, the unified document object model is constructed and aligned with cross-modal semantics, so that correlation analysis and consistency verification of text and non-text media contents are realized, and the problem of multi-modal content splitting is solved; through a dynamic knowledge graph construction mechanism based on incremental learning and conflict judgment, automatic acquisition, fusion and evolution of external knowledge are realized, and the accuracy and timeliness of fact verification are ensured.

Description

Technical Field

[0001] This invention relates to the field of intelligent correction technology for digital textbooks, specifically to an intelligent proofreading and content correction system for digital textbooks based on deep learning. Background Technology

[0002] With the deep integration of information technology and education, digital textbooks have gradually replaced traditional paper textbooks, becoming an important medium for education and teaching. Digital textbooks typically contain multiple modalities such as text, images, audio, and video, offering rich formats and strong interactivity. However, their diverse sources and complex editing processes also lead to frequent problems such as factual errors, logical contradictions, and inconsistencies across modalities. Currently, conventional automated proofreading methods are mostly rule-based, capable of handling only simple spelling and grammar errors, struggling to cope with complex logical errors, and unable to effectively verify the consistency of multimodal content.

[0003] For example, the invention patent with publication number CN120087339A discloses an intelligent proofreading system for educational supplementary books, including: a knowledge graph construction module, used to collect authoritative textbooks, academic literature, teaching syllabi, and other materials, and using natural language processing technology and knowledge extraction algorithms to construct a knowledge graph covering knowledge points of various disciplines and establish the relationships between knowledge points; and a text parsing module, which can split the input educational supplementary book text into sentences and paragraphs, and use technologies such as part-of-speech tagging and syntactic analysis to extract keywords, key phrases, and other information, and analyze the grammatical structure and semantic information of the text. This invention effectively improves the proofreading efficiency and accuracy of pure text content.

[0004] However, the above and similar technical solutions still have the following shortcomings: they only perform error detection for a single modality and lack a unified understanding and correlation analysis of multimodal content; they lack continuously evolving external knowledge support, making it difficult to guarantee the accuracy and timeliness of fact-checking; and their logical verification is limited to the text structure and lacks the ability to deeply understand and reason about the logical contradictions in the underlying facts. Summary of the Invention

[0005] The purpose of this invention is to provide a deep learning-based intelligent proofreading and content correction system for digital textbooks, in order to solve the problems mentioned in the background art.

[0006] To achieve the above objectives, the present invention provides the following technical solution: a deep learning-based intelligent proofreading and content correction system for digital textbooks, comprising: Multimodal parsing module: used to parse digital textbook source files, separate and extract text and non-text media content, and generate a unified document object model containing semantic relationships; The knowledge management module is used to acquire raw data from multiple heterogeneous knowledge sources and construct cross-domain knowledge graphs; and to drive the automated synchronization and evolution of cross-domain knowledge graphs based on incremental learning mechanisms. Semantic understanding and reasoning module: used to perform deep semantic encoding on the unified document object model through a multimodal pre-trained model, and to realize knowledge association, factual verification and logical detection of named entities based on cross-domain knowledge graph; Cross-modal verification module: Used to compare the semantic representations of textual and non-textual media content in a unified semantic space using a cross-modal pre-trained model to detect their consistency; Decision generation and feedback module: This module integrates, classifies, and prioritizes the detection results from each module, and generates corrective suggestions through a suggestion generator based on a deep learning model, which are then presented to the administrator via an interactive interface. Continuous learning module: Used to collect interactive feedback data from administrators on correction suggestions, and to use this data to optimize the models in the system.

[0007] Furthermore, the non-textual media content includes non-textual static content and temporal media content; the multimodal parsing module includes: The system receives digital teaching material source files in different formats, parses their encapsulation formats, and decomposes them into independent basic components; these basic components include text streams, image files, audio files, and video files. Based on the data characteristics of each basic constituent unit, all the decomposed basic constituent units are classified, their element types are identified, and the physical layout information and sequence position of each element in the source file are recorded; the element types include text elements, non-text static elements, and time-series media elements. For text-based elements, plain text content is extracted, and the logical hierarchy of the unified document object model is inferred and constructed by combining its text format information, physical layout information, and sequence position. For non-text static elements, visual semantic features are extracted using a convolutional neural network. For temporal media elements, textual content summarization is extracted using a recurrent neural network. Based on the plain text content, logical hierarchical structure, visual semantic features, and textual content summary, the contextual relationships between elements are analyzed, and semantic associations are identified and established. The results of all the above steps are integrated and encapsulated to output a structured unified document object model that contains complete content, hierarchy, and inherent semantic relationships.

[0008] Furthermore, the knowledge management module includes: It automatically acquires raw data from pre-set multi-source heterogeneous knowledge sources and converts raw data of different structures and formats into a unified representation. Using a Bayesian deep learning model, confidence scores are generated for conflicting descriptions of the same entity from different knowledge sources, and automatic detection and conflict adjudication are performed based on the scores to form a unified knowledge fragment. The unified knowledge fragments obtained through conflict resolution are associated and spliced ​​together to construct and store a cross-domain knowledge graph with entity-relationship-entity as the core structure. Continuously monitor the update dynamics of multi-source heterogeneous knowledge sources. When a valid change is detected, automatically acquire the original data of the knowledge source that has changed, perform the above-mentioned fusion process on it, and update the existing cross-domain knowledge graph with the fused knowledge increment. After each incremental update is completed, a new round of logical consistency checks is automatically performed across the entire graph using a graph neural network to ensure that the newly added knowledge does not logically contradict existing knowledge.

[0009] Furthermore, the semantic understanding and reasoning module includes: Receive the unified document object model, call the multimodal pre-trained model, and map the text and non-text media content therein to a high-dimensional semantic space to generate the corresponding deep semantic encoding vector; Based on the deep semantic encoding vector, named entities are identified and extracted from the text content, and linked with entities in a cross-domain knowledge graph to generate an enhanced semantic representation that integrates external knowledge. Based on the enhanced semantic representation, statements representing facts are located from the text content; the semantics of the statement are compared with the corresponding facts recorded in the cross-domain knowledge graph; if there is a semantic conflict, it is determined to be a factual error, and the error information and evidence in the cross-domain knowledge graph are recorded to generate factual error data. Traverse the unified document object model and scan the semantic representations of multiple statements for the same entity; based on the predefined logical rules in the cross-domain knowledge graph, analyze whether there are any incompatible logical conflicts between the semantic representations of the multiple statements; if a conflict is detected, it is determined to be a logical error, and the conflicting statement pair and its contradictions are recorded to generate logical error data. For detected factual and logical errors, an evidence chain is automatically generated; the evidence chain includes error location, error type, and error analysis conclusion.

[0010] Furthermore, the cross-modal verification module includes: Receive all semantically related text-media element pairs from the unified document object model generated by the multimodal parsing module; The cross-modal pre-trained model is invoked to map the text content and non-text media content in each received element pair to the same shared semantic space, and generate text alignment vectors and media alignment vectors that can be directly measured for similarity. In the semantic space, the similarity between the text alignment vector and the media alignment vector of each pair of related elements is calculated, and a quantified semantic consistency score is generated for the element pair. The semantic consistency score is compared with a consistency threshold that is dynamically adjusted based on historical verification data. If the score is lower than the threshold, the text-media element pair is determined to have a multimodal inconsistency error, and a preliminary inconsistency record is generated. All detected inconsistencies are integrated and associated with their original positions in the document object model to obtain multimodal inconsistency error data.

[0011] Furthermore, the decision generation and feedback module includes: It receives and aggregates factual error data and logical error data from the semantic understanding and reasoning module, as well as multimodal inconsistency error data from the cross-modal verification module; The aggregated error data is deduplicated, merged, and categorized to form a structured error list; Based on the error type, the severity and scope of impact are assessed according to preset rules, and the error list is prioritized accordingly. The suggestion generator based on the deep learning model generates correction suggestions for each error according to error type, priority ranking, and cross-domain knowledge graph. The sorted error list and suggested fixes are presented to the administrator through an interactive interface, and all administrator actions are recorded.

[0012] Furthermore, the continuous learning module includes: Collect interactive feedback data from the decision generation and feedback module regarding the administrator's adoption, rejection, and modification of suggested corrections; Based on the interactive feedback data, valuable positive and negative examples are selected to construct a training sample set for model optimization; Using the constructed training sample set, the multimodal pre-trained model, the cross-modal pre-trained model, and the proposal generator are periodically optimized through an incremental learning mechanism; Manage the optimized model version and deploy the performance-enhanced model to the corresponding production environment through a secure process.

[0013] Compared with the prior art, the beneficial effects of the present invention are: A deep learning-based intelligent proofreading and content correction system for digital textbooks improves the efficiency of proofreading and correction and reduces operating costs by focusing human resources on core decision-making processes through fully automated detection and intelligent correction suggestion generation. By prioritizing errors based on severity and scope of impact, it achieves intelligent error sorting, optimizes the allocation of proofreading resources, and guides administrators to prioritize critical issues.

[0014] Meanwhile, by constructing a unified document object model and cross-modal semantic alignment, the system achieves correlation analysis and consistency verification of textual and non-textual media content, solving the problem of fragmented multimodal content. Through a dynamic knowledge graph construction mechanism based on incremental learning and conflict adjudication, the system realizes the automatic acquisition, fusion, and evolution of external knowledge, ensuring the accuracy and timeliness of fact-checking. By combining the enhanced semantic representation of the knowledge graph with predefined logical rules, the system achieves deep semantic understanding and reasoning of deep-seated factual logical contradictions. Attached Figure Description

[0015] Figure 1 This is a schematic diagram of the structure of the digital textbook intelligent proofreading and content correction system of the present invention; Figure 2 This is a schematic diagram of the unified document object model construction method of the present invention; Figure 3 This is a schematic diagram of the knowledge management method of the present invention; Figure 4 This is a schematic diagram of the semantic understanding and reasoning method of the present invention; Figure 5 This is a schematic diagram of the cross-modal consistency verification method of the present invention; Figure 6 This is a schematic diagram of the method for generating modification suggestions according to the present invention. Detailed Implementation

[0016] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0017] like Figure 1As shown, this invention provides a technical solution: a deep learning-based intelligent proofreading and content correction system for digital textbooks, comprising a multimodal parsing module, a knowledge management module, a semantic understanding and reasoning module, a cross-modal verification module, a decision generation and feedback module, and a continuous learning module that communicate with each other via a system bus. This system is deployed on one or more servers and can accept digital textbook source files from users or publishing systems through an interface.

[0018] The multimodal parsing module is used to parse digital textbook source files, separate and extract text and non-text media content, and generate a unified document object model containing semantic associations; the non-text media content includes non-text static content and time-series media content.

[0019] like Figure 2 As shown, this invention provides a method for constructing a unified document object model; Specifically: The system receives digital teaching material source files in different formats, parses their encapsulation formats, and decomposes them into independent basic components; these basic components include text streams, image files, audio files, and video files. Based on the data characteristics of each basic constituent unit, all the decomposed basic constituent units are classified, their element types are identified, and the physical layout information and sequence position of each element in the source file are recorded; the element types include text elements, non-text static elements, and time-series media elements. For text-based elements, plain text content is extracted, and the logical hierarchy of the unified document object model is inferred and constructed by combining its text format information, physical layout information, and sequence position. For non-text static elements, visual semantic features are extracted using a convolutional neural network. For temporal media elements, textual content summarization is extracted using a recurrent neural network. Based on the plain text content, logical hierarchical structure, visual semantic features, and textual content summary, the contextual relationships between elements are analyzed, and semantic associations are identified and established. The results of all the above steps are integrated and encapsulated to output a structured unified document object model that contains complete content, hierarchy, and inherent semantic relationships.

[0020] It should be noted that the formats of the digital teaching material source files include, but are not limited to, EPUB, PDF, and DOCX. By calling the corresponding document format parsing library, the file's encapsulation structure is read and decomposed into independent basic constituent units (this can be achieved through data stream extraction technology).

[0021] The system performs pattern recognition based on data features for each basic component, specifically including: identifying text elements by analyzing file header information and data patterns; identifying non-text static elements by recognizing file formats (such as JPEG and PNG) and distinguishing them from the text stream; and identifying temporal media elements by recognizing file formats. The text elements include, but are not limited to: body text, titles, list items, figure captions, and multiple-choice options. The non-text static elements include: photographs, visualizations, chemical molecular structures, and formulas embedded as images; the temporal media elements include: audio explanations, demonstration videos, and animation content. The system uses document layout analysis algorithms (such as computer vision-based page segmentation technology) to record the physical coordinates, size, and page / region to which each element belongs in the original page. Simultaneously, it uses a depth-first traversal algorithm to record the sequential position of each element in the document content stream. This information is bound as metadata to the corresponding elements.

[0022] For text elements, the system first extracts the plain text content through character encoding conversion and text cleanup techniques (such as using Python libraries). Then, the system performs logical structure reasoning: using rule-based hierarchical inference algorithms (such as structure parsing based on XPath or CSS selectors), it analyzes the text's formatting information (such as font, font size, and bold) and the physical layout information and sequence position recorded in the previous steps. For example, the text with the largest font size and centered is identified as a "heading," and the text blocks below it with bullet points are identified as "list items," thus automatically constructing the document's logical hierarchical structure, such as chapters, paragraphs, and lists. For non-text static elements, the system uses a deep neural network based on convolutional operations (such as a RestNet-50 pre-trained model) to process them, forming a high-dimensional feature vector that can represent the core semantic content of the image, i.e., a visual semantic feature vector. For temporal media elements, the system first uses automatic speech recognition technology (such as the open-source tool Kaldi) to convert the audio track into text, and uses optical character recognition technology (such as Tesseract OCR) to extract the text in the video frames; then, it uses a neural network model based on recurrent connections (such as LSTM) to process the converted text sequence and generate a textual content summary that condenses the core semantics of the original media.

[0023] The system integrates and analyzes the outputs of the preceding steps using cross-modal semantic alignment techniques (such as the contrastive learning mechanism using the CLIP model) to establish semantic relationships between elements. Specifically, within a shared semantic space, it compares text content, visual semantic feature vectors, and text content summaries. For example, by calculating the semantic similarity between feature vectors, the system can automatically associate text paragraphs describing "photosynthesis" with images showing the cell structure of plant leaves and video summaries of the degradation process, identifying and establishing semantic relationships between them, and constructing text-media element pairs. Finally, the system structurally integrates and encapsulates the processing results of all the aforementioned steps, outputting a machine-readable unified document object model.

[0024] Knowledge Management Module: Connected to the multimodal parsing module, it is used to acquire raw data from multiple heterogeneous knowledge sources and construct cross-domain knowledge graphs; and to drive the automated synchronization and evolution of cross-domain knowledge graphs based on an incremental learning mechanism.

[0025] like Figure 3 As shown, the present invention provides a knowledge management method; Specifically: It automatically acquires raw data from pre-set multi-source heterogeneous knowledge sources and converts raw data of different structures and formats into a unified representation. Using a Bayesian deep learning model, confidence scores are generated for conflicting descriptions of the same entity from different knowledge sources, and automatic detection and conflict adjudication are performed based on the scores to form a unified knowledge fragment. The unified knowledge fragments obtained through conflict resolution are associated and spliced ​​together to construct and store a cross-domain knowledge graph with entity-relationship-entity as the core structure. Continuously monitor the update dynamics of multi-source heterogeneous knowledge sources. When a valid change is detected, automatically acquire the original data of the knowledge source that has changed, perform the above-mentioned fusion process on it, and update the existing cross-domain knowledge graph with the fused knowledge increment. After each incremental update is completed, a new round of logical consistency checks is automatically performed across the entire graph using a graph neural network to ensure that the newly added knowledge does not logically contradict existing knowledge.

[0026] It is important to note that the system automatically acquires raw data from multiple heterogeneous knowledge sources (such as professional databases, authoritative encyclopedia websites, and academic journal websites) through pre-defined data interface calls and web scraping technologies (such as using Python libraries). This data may exist in structured, semi-structured, or unstructured forms. Using ontology-based semantic mapping techniques (such as using the OWL ontology language to define a unified probabilistic model), data patterns from different sources are aligned to a unified core concept model. For example, the "inventor" field from source A and the "creator" field from source B are both mapped to a unified "inventor" relation. Subsequently, a data cleaning process combining rules and dictionaries is used to handle issues such as spelling errors, inconsistent units, and inconsistent date formats in the raw data. The cleaned and aligned data is then transformed into a unified and standardized data representation form within the system, typically represented as standardized triples of (subject entity, relation, object entity).

[0027] The system identifies and aggregates different descriptive statements about the same entity from all knowledge sources through entity resolution and coreference resolution algorithms, forming a conflict description set. A Bayesian deep learning model (such as a Bayesian neural network based on Pyro) is used to calculate a quantified confidence score for each candidate description in the conflict description set. The input to this model is a feature vector that integrates multi-dimensional evidence. The features extracted for each candidate description include, but are not limited to: source authority score (the historical accuracy and generally accepted authority quantification value of the knowledge source from which the candidate description originates); description timeliness index (the publication time or version number of the candidate description in the source data); and cross-source synergy (the number of independent knowledge sources supporting the candidate description and its own average authority). The Bayesian deep learning model processes the input feature vector through its forward propagation and utilizes its inherent probabilistic inference mechanism, ultimately outputting a confidence score for each candidate description. The system automatically selects the candidate description with the highest confidence score as reliable knowledge based on a preset confidence threshold rule (e.g., setting a threshold of 0.7, and accepting those above this threshold) or an optimal selection algorithm. The system encapsulates the credible knowledge it determines, its corresponding confidence score, and all supporting source information into a structured data unit, namely a unified knowledge fragment.

[0028] The system uses relational path discovery and graph structure construction algorithms to associate the unified knowledge fragments generated in the above steps. It identifies identical entities in different triples and connects them to form a knowledge network. The completed cross-domain knowledge graph, centered on entity-relationship-entity relationships, is stored in a dedicated database (such as Neo4j) that supports efficient graph traversal and relation queries. The system also builds inverted indexes for entities and relations to enable millisecond-level knowledge retrieval and reasoning in subsequent modules.

[0029] The system continuously monitors the update dynamics of each knowledge source through a publish-subscribe messaging mechanism or periodic content differential comparison technology. When a valid change is detected in a knowledge source, the system automatically triggers an incremental data processing flow to acquire the changed original data and independently execute the above steps to generate a knowledge increment package to be merged. The system then securely merges this knowledge increment package into the existing cross-domain knowledge graph through transactional graph update operations, achieving smooth evolution of knowledge rather than a complete reconstruction.

[0030] The effective changes are determined using a change intelligent filtering algorithm based on multi-strategy fusion. The specific judgment logic and process of this algorithm include: For content-based substantive change detection based on information entropy, the system calculates the information entropy change value of the updated content from the knowledge source. For text content, it analyzes the text differences before and after the update using a semantic change measurement method based on statistical language. When the information entropy change value exceeds a preset substantive change threshold (e.g., 0.5), the updated content is initially determined to be a substantive change. For knowledge-based relevance filtering based on entity-relation mapping, the system maintains a list of core entities and relation types relevant to the local domain. When a change is detected, the system identifies the core entities and relation types involved in the updated content through real-time entity linking and relation extraction technology. Only when the changed entity or relation exists in the list is the change considered relevant to the current cross-domain knowledge graph, and the updated content is initially determined to be a relevance change. Based on the quality improvement judgment of the confidence assessment, the system initiates a lightweight pre-fusion confidence assessment process for changes that are initially determined to be substantial and relevant. This process quickly evaluates the quality of the change itself (such as the authority of the source and the completeness of the statement) and its consistency with existing knowledge in the cross-domain knowledge graph. If the change can resolve existing knowledge conflicts, correct known errors, or provide more authoritative evidence, it is assigned a higher quality improvement score. When the quality improvement score of a change exceeds the quality acceptance threshold (e.g., 0.8), it is ultimately determined to be a valid change.

[0031] The system predefines a series of logical rules to characterize the logical constraints that knowledge should satisfy. For example, it defines rules such as "birth date before death date" and "a city cannot be the capital of two provinces simultaneously." After each incremental update, the system automatically initiates a consistency check across the entire knowledge graph. This check is implemented using a graph neural network (such as the GAT model) based on graph structure information propagation. For detected logical contradictions, the system marks them as high-priority conflicts and initiates the aforementioned conflict resolution process for re-resolution, or directly issues an alert to the system administrator for manual intervention, ensuring that the cross-domain knowledge graph always maintains logical consistency.

[0032] Semantic understanding and reasoning module: Connected to the multimodal parsing module and knowledge management module, it is used to perform deep semantic encoding on the unified document object model through a multimodal pre-trained model, and realize knowledge association, factual verification and logical detection of named entities based on cross-domain knowledge graph.

[0033] like Figure 4 This invention provides a semantic understanding and reasoning method; Specifically: Receive the unified document object model, call the multimodal pre-trained model, and map the text and non-text media content therein to a high-dimensional semantic space to generate the corresponding deep semantic encoding vector; Based on the deep semantic encoding vector, named entities are identified and extracted from the text content, and linked with entities in a cross-domain knowledge graph to generate an enhanced semantic representation that integrates external knowledge. Based on the enhanced semantic representation, statements representing facts are located from the text content; the semantics of the statement are compared with the corresponding facts recorded in the cross-domain knowledge graph; if there is a semantic conflict, it is determined to be a factual error, and the error information and evidence in the cross-domain knowledge graph are recorded to generate factual error data. Traverse the unified document object model and scan the semantic representations of multiple statements for the same entity; based on the predefined logical rules in the cross-domain knowledge graph, analyze whether there are any incompatible logical conflicts between the semantic representations of the multiple statements; if a conflict is detected, it is determined to be a logical error, and the conflicting statement pair and its contradictions are recorded to generate logical error data. For detected factual and logical errors, an evidence chain is automatically generated; the evidence chain includes error location, error type, and error analysis conclusion.

[0034] It is important to note that the system receives the Unified Document Object Model from the multimodal parsing module and calls the multimodal pre-trained model. This model contains parallel and independent encoding pathways, specifically: a text encoding pathway, which inputs the plain text content from the document into a deep text encoding network (such as BERT) based on a self-attention mechanism to generate a deep text semantic vector that can represent the deep meaning of the entire text context; and a media encoding pathway, which inputs non-textual media content (such as visual semantic feature vectors and textual content summaries) into a deep feature encoding network (such as a multilayer perceptron MLP) based on forward propagation to generate a deep media semantic vector that can represent its core semantics. The generated deep text semantic vectors and deep media semantic vectors are stored back into the Unified Document Object Model and associated with the original content elements, serving as the basic semantic representation for all subsequent analyses in this module.

[0035] The training data for the multimodal pre-trained model includes, but is not limited to: image data with textual descriptions (such as publicly available web page images and their annotations), plain text corpora (such as encyclopedias, news articles, and books), videos and their corresponding texts (such as instructional videos and narration), and alignment data between knowledge graphs and text. All data used for model pre-training must undergo rigorous cleaning, filtering, and standardization to ensure that the model can learn broad, neutral, and deep cross-modal semantic understanding capabilities.

[0036] The system targets deep text semantic vectors and employs context-aware sequence labeling techniques (such as the BERT-CRF model). By analyzing the vector distribution patterns of words and their contexts in the semantic space, it accurately locates and extracts all named entities (such as names, locations, and scientific terms) mentioned in the text. For each identified text entity, the system performs semantic similarity matching between the entity's referent and the target entity in a cross-domain knowledge graph. The entity referent refers to a word or phrase that actually appears in the text and points to a specific entity; the target entity refers to a unique and clearly defined entity node in the cross-domain knowledge graph. By calculating the spatial distance (such as cosine similarity) between the text entity vector and the candidate entity node vectors in the cross-domain knowledge graph, the system selects the most matching entity for linking. Upon successful linking, the system retrieves the entity's attributes and relationships from the cross-domain knowledge graph and combines this external knowledge with the deep semantic vectors of the original text through vector concatenation and attention-weighted fusion mechanisms, forming a richer, enhanced semantic representation.

[0037] The system utilizes enhanced semantic representation based on text, employing a joint model (such as using the spaCy library) based on dependency parsing and semantic role labeling to automatically locate and parse statements representing objective facts from the text. This model extracts the core semantic structure of the statements, typically represented as (subject, relation, object) triples. The system performs semantic-based triple consistency matching (e.g., using cosine similarity with a threshold of 0.9) between the parsed textual fact triples and the corresponding facts recorded in the cross-domain knowledge graph. When a semantic conflict is detected between the textual statement and the records in the cross-domain knowledge graph, it is determined to be a factual error. The system automatically records the location of the error, the erroneous original statement, and cites the correct facts from the cross-domain knowledge graph as evidence, generating structured factual error data.

[0038] For text content, the system traverses the unified document object model and uses entity coreference resolution algorithms (such as a coreference resolution model based on neural networks) to locate and aggregate all text statements scattered across different paragraphs that refer to the same entity. For non-textual media content, the system uses deep media semantic vectors and textual content summaries, employing semantic clustering techniques (such as the K-means clustering algorithm) to aggregate information describing the same topic or entity. The system calls a predefined logical rule base (including transitive laws, mutual exclusion laws, attribute uniqueness rules, etc.) and uses a hybrid inference engine combining symbolic reasoning based on logical constraints and vector space reasoning to analyze whether there are incompatible logical conflicts among multiple statements aggregated within the same modality. Once a logical conflict is detected, the system determines it as a logical error, records all statements constituting the logical conflict and their positions in the document, explicitly marks the logical rules violated, and generates structured logical error data.

[0039] The system integrates and formats the generated factual and logical error data using structured error report templates and natural language generation technologies (such as the Jinja2 template engine or GPT series models). For each error, a complete chain of evidence is automatically generated, which includes at least: error location (the chapter, page, and specific location of the error); error type (clearly categorized as "factual error" or "logical error"); and error analysis conclusions, combining context and cross-domain knowledge graph evidence to generate a concise, human-readable analysis explanation.

[0040] Cross-modal verification module: connected to the multimodal parsing module, used to compare the semantic representations of text and non-text media content in a unified semantic space through a cross-modal pre-trained model to detect their consistency.

[0041] like Figure 5 This invention provides a cross-modal consistency verification method; Specifically: Receive all semantically related text-media element pairs from the unified document object model generated by the multimodal parsing module; The cross-modal pre-trained model is invoked to map the text content and non-text media content in each received element pair to the same shared semantic space, and generate text alignment vectors and media alignment vectors that can be directly measured for similarity. In the semantic space, the similarity between the text alignment vector and the media alignment vector of each pair of related elements is calculated, and a quantified semantic consistency score is generated for the element pair. The semantic consistency score is compared with a consistency threshold that is dynamically adjusted based on historical verification data. If the score is lower than the threshold, the text-media element pair is determined to have a multimodal inconsistency error, and a preliminary inconsistency record is generated. All detected inconsistencies are integrated and associated with their original positions in the document object model to obtain multimodal inconsistency error data.

[0042] It is important to note that the system reads all text-media element pairs identified and established by the multimodal parsing module from the Unified Document Object Model. These element pairs refer to specific text segments and specific media that are semantically defined to describe or be associated with each other within the document structure. For each extracted element pair, the system performs standardization preprocessing to ensure consistency in subsequent encoding processes. For text content, Unicode standardization and special character filtering are performed; for non-text media content, resolution standardization and color space unification are performed.

[0043] The system invokes a cross-modal pre-trained model (such as the CLIP model) pre-trained using a contrastive learning mechanism. This model is jointly trained on a massive number of (text, image) pairs, bringing matching pairs closer together and pushing away mismatched pairs. The parameters of the text encoder and media encoder are jointly optimized. The text content of the element pair is input into the text encoding sub-network of this model, which generates a text alignment vector through its internal multi-layer transformer structure; the non-text media content of the element pair is input into the media encoding sub-network of this model, which generates a media alignment vector through its internal deep convolutional network structure. The outputs of the two encoders are constrained to the same high-dimensional semantic space, so that semantically consistent content pairs have similar vector representations.

[0044] The similarity of the aligned vectors is measured. Specifically, the system calculates the similarity between each pair of text-aligned vectors and media-aligned vectors using a vector space similarity measurement algorithm. Preferably, a cosine similarity calculation function is used, which evaluates the consistency of their directions by calculating the cosine value of the angle between the two vectors; the function output threshold is [-1, 1], which is mapped to the [0, 1] interval through a linear transformation to generate a standardized semantic consistency score, where 1 represents complete consistency and 0 represents complete inconsistency. The similarity measurement is not limited to cosine similarity; any mathematical method that can quantify the degree of similarity between two vectors is acceptable.

[0045] The system dynamically calculates the consistency threshold based on historical verification data using a sliding window statistical method. Specifically, it calculates the mean and standard deviation of the consistency scores in the most recent N verification batches (e.g., N=100), and determines the dynamic threshold for the current batch (e.g., threshold = mean - 2 × standard deviation) according to Chebyshev's inequality. The semantic consistency score of each element pair is compared with the dynamic threshold. When the score is lower than the threshold, the text-media element pair is determined to have a multimodal inconsistency error, and an inconsistency record containing the element identifier, consistency score, dynamic threshold, and difference description is generated.

[0046] By employing the directional indexing mechanism of the Unified Document Object Model, inconsistency records are associated with spatial layout information in the original document, including metadata such as page coordinates, layer information, and display area. All inconsistency records are then integrated to generate multimodal inconsistency error data, which includes: error location, error type, error content, consistency score, and dynamic threshold data.

[0047] Decision generation and feedback module: Connected to the semantic understanding and reasoning module and the cross-modal verification module, it is used to integrate, classify and prioritize the detection results of each module, and generate correction suggestions through a suggestion generator based on a deep learning model, which are then presented to the administrator through an interactive interface.

[0048] like Figure 6 This invention provides a method for generating correction suggestions; Specifically: It receives and aggregates factual error data and logical error data from the semantic understanding and reasoning module, as well as multimodal inconsistency error data from the cross-modal verification module; The aggregated error data is deduplicated, merged, and categorized to form a structured error list; Based on the error type, the severity and scope of impact are assessed according to preset rules, and the error list is prioritized accordingly. The suggestion generator based on the deep learning model generates correction suggestions for each error according to error type, priority ranking, and cross-domain knowledge graph. The sorted error list and suggested fixes are presented to the administrator through an interactive interface, and all administrator actions are recorded.

[0049] It is important to note that the system receives factual and logical error data from the semantic understanding and reasoning module, and multimodal inconsistency error data from the cross-modal verification module, using a pre-defined standardized data exchange protocol (such as JSON over HTTP). All received error data is temporarily stored in a unified temporary data pool. The system assigns a globally unique tracking identifier (such as a UUID) to each error data unit and records its reception timestamp and source module, completing the initial data aggregation.

[0050] The system employs a semantic vector-based clustering algorithm (e.g., using Sentence-BERT to generate vectors and DBSCAN for clustering) to clean the aggregated error data. This algorithm maps the textual description of each error (e.g., error location, error content) to a semantic vector and calculates the similarity between vectors (e.g., cosine similarity). If the semantic vector similarity of multiple errors exceeds a preset threshold (e.g., 0.85) and the error location information highly overlaps, these errors are determined to be duplicate reports of the same problem and are merged into a single master error record. The system automatically classifies the deduplicated errors based on a predefined multi-dimensional error classification system. This system includes at least the following dimensions: error type, such as factual errors, logical errors, and multimodal inconsistency errors; content domain, such as literature, history, and physics; and the object involved, such as text, images, and audio. All deduplicated and classified error data is organized according to a fixed field format (e.g., error ID, type, location, description, evidence, and status), outputting a structured error list.

[0051] The system quantifies the severity of each error in a multi-error list according to preset rules. These rules consider at least the following factors: core knowledge deviation (the degree to which the error deviates from the core concept; a greater deviation results in a higher score, e.g., using a 0-1 scale, a deviation > 0.7 results in a higher score); misleading intensity (how easily the error can lead to serious misunderstanding, e.g., using a binary rating system, yes = 1, no = 0); and scope of influence (the range of influence of the error's location; a wider scope results in a higher score, e.g., a chapter title error scores 1, a footnote error scores 0.2). The system then sorts the error list in descending order based on the calculated severity scores. For errors with the same score, a second sorting is performed using the rule engine; for example, "factual errors" are processed first, followed by "logical errors." The final result is a sorted error list with clearly defined priorities.

[0052] The system invokes a sequence-to-sequence deep learning model (such as the BART model) as the suggestion generator. This model takes the complete context of the error (including the error type, error content, related context paragraphs, priority, and correct evidence retrieved from a cross-domain knowledge graph) as input. Through its internal encoder-decoder mechanism, the deep learning model first deeply understands the semantics of the input information, and then generates word-by-word natural language correction suggestions that conform to human language habits at the decoding end. For example, for the factual error "All mammals are viviparous," if the correct facts recorded in the cross-domain knowledge graph are (platypus, is, mammal) and (platypus, reproductive method, oviparous), the generated correction suggestion is: "It is suggested to change it to 'The vast majority of mammals are viviparous, but there are exceptions, such as the platypus, which is a twin mammal.' Basis: The platypus' reproductive method is oviparous." The system uses a graphical user interface (such as a web-based React frontend) to clearly present the sorted list of errors and corresponding correction suggestions to the administrator. The interface design employs a linked list and highlighting technique; when an administrator clicks on an error in the list, the system automatically highlights the specific location of the error in the document preview view on the right. The system provides clear operation interfaces (such as "Accept," "Reject," and "Manual Modify" buttons) and fully records the administrator's final processing record for each correction suggestion, as well as any custom modifications. The system encapsulates the administrator's final processing record to form interactive feedback data, which is synchronized in real time to the continuous learning module for subsequent optimization of the system model. This interactive feedback data includes: error ID, error type, error content, correction suggestion, administrator's final processing record, timestamp, and related contextual information and model version number.

[0053] Continuous learning module: Used to collect interactive feedback data from administrators on correction suggestions, and to use this data to optimize the models in the system.

[0054] The continuous learning module includes: Collect interactive feedback data from the decision generation and feedback module regarding the administrator's adoption, rejection, and modification of suggested corrections; Based on the interactive feedback data, valuable positive and negative examples are selected to construct a training sample set for model optimization; Using the constructed training sample set, the multimodal pre-trained model, the cross-modal pre-trained model, and the proposal generator are periodically optimized through an incremental learning mechanism; Manage the optimized model version and deploy the performance-enhanced model to the corresponding production environment through a secure process.

[0055] It is important to note that during the initialization phase, this system builds and maintains a benchmark test set isolated from the training data. This test set contains representative samples of each error type, with correct annotations, and is used for performance testing before model version updates. The system receives interactive feedback data from the administrator's interface in real time. The system performs a data quality verification process on each batch of received interactive feedback data, specifically checking whether required fields are complete, whether the format conforms to predefined rules, and whether the data logic is reasonable (for example, a "reject" action should not contain text modified by the administrator). For data records that fail verification, the system marks them as invalid and isolates them, preventing them from proceeding to the next step, while simultaneously generating an alarm log.

[0056] From the validated interactive feedback data, the system selects the most valuable samples for model optimization and constructs corresponding training sets for different models. Specifically, the system uses a sample value evaluation algorithm based on operation type and content changes to evaluate the validated interactive feedback data. High-value positive examples mainly come from correction suggestions "adopted" by the administrator, forming (incorrect input, correct output) sample pairs to reinforce the correct behavior of the model. High-value negative examples mainly come from correction suggestions "rejected" or "manually modified" by the administrator, forming (incorrect input, incorrect output) sample pairs or (incorrect input, better output) sample pairs to correct the model's error tendencies and make the model learn more accurately. The better output refers to the content manually modified by the administrator. According to a dynamic category balancing strategy, the system extracts samples proportionally from the selected high-value positive and negative examples to construct a balanced training sample set, maintaining the balance of positive and negative samples and preventing the model from developing new biases due to an excess of one type of feedback.

[0057] The system employs an incremental learning algorithm based on elastic weight consolidation to optimize the model. Utilizing the constructed training sample set, the system performs parallel and targeted incremental optimization of the core models through a routing mechanism based on sample error types. For multimodal and cross-modal pre-trained models, the system routes all samples from the training sample set originating from factual, logical, and multimodal inconsistency judgments to these two models for optimization, correcting their internal parameters to more accurately understand the deep semantics and relationships of textual and non-textual media content, thus reducing the false judgment rate. For the suggestion generator, the system routes all samples from the entire training sample set to this model for optimization, improving the factual accuracy, logical rationality, linguistic standardization, and consistency with administrator modification preferences in its output. The system's model optimization is controlled by a hybrid trigger mechanism based on time and data volume. When either a preset periodic event (e.g., every 15 days) or the number of newly collected compliant training samples reaches a preset threshold (e.g., 1000) is met, the system automatically initiates an optimization process.

[0058] Each optimized new model needs to be evaluated on a base test set, and its overall performance should be no less than that of the current production model version (e.g., accuracy ≥ 95%). New models that pass the evaluation will be securely deployed to the production environment.

[0059] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended embodiments and their equivalents.

Claims

1. A deep learning-based intelligent proofreading and content correction system for digital textbooks, characterized in that, include: Multimodal parsing module: used to parse digital textbook source files, separate and extract text and non-text media content, and generate a unified document object model containing semantic relationships; Knowledge Management Module: Used to acquire raw data from multiple heterogeneous knowledge sources and build cross-domain knowledge graphs; And for use in driving the automated synchronization and evolution of cross-domain knowledge graphs based on incremental learning mechanisms; Semantic understanding and reasoning module: used to perform deep semantic encoding on the unified document object model through a multimodal pre-trained model, and to realize knowledge association, factual verification and logical detection of named entities based on cross-domain knowledge graph; Cross-modal verification module: Used to compare the semantic representations of textual and non-textual media content in a unified semantic space using a cross-modal pre-trained model to detect their consistency; Decision generation and feedback module: This module integrates, classifies, and prioritizes the detection results from each module, and generates corrective suggestions through a suggestion generator based on a deep learning model, which are then presented to the administrator via an interactive interface. Continuous learning module: Used to collect interactive feedback data from administrators on correction suggestions, and to use this data to optimize the models in the system.

2. The intelligent proofreading and content correction system for digital textbooks based on deep learning according to claim 1, characterized in that: The non-text media content includes non-text static content and time-series media content; The multimodal parsing module includes: S1. Receive digital textbook source files in different formats, parse their encapsulation format, and decompose them into independent basic constituent units; the basic constituent units include text streams, image files, audio files, and video files; S2. Based on the data characteristics of each basic constituent unit, classify all the decomposed basic constituent units, identify their element types, and record the physical layout information and sequence position of each element in the source file; the element types include text elements, non-text static elements, and time-series media elements. S3. For text-type elements, extract the plain text content, and combine its text format information with the physical layout information and sequence position to deduce and construct the logical hierarchy structure of the unified document object model; for non-text static elements, use a convolutional neural network to extract visual semantic features; for temporal media elements, use a recurrent neural network to extract textual content summaries. S4. Based on the plain text content, logical hierarchical structure, visual semantic features, and textual content summary, analyze the contextual relationships between elements, identify and establish semantic associations; S5. Integrate and encapsulate the processing results of all the above steps to output a structured Unified Document Object Model that contains complete content, hierarchy, and inherent semantic relationships.

3. The intelligent proofreading and content correction system for digital textbooks based on deep learning according to claim 1, characterized in that: The knowledge management module includes: M1 automatically acquires raw data from preset multi-source heterogeneous knowledge sources and converts raw data of different structures and formats into a unified representation. M2. Using a Bayesian deep learning model, confidence scores are generated for conflicting descriptions of the same entity from different knowledge sources, and automatic detection and conflict adjudication are performed based on the scores to form a unified knowledge fragment. M3. Associate and splice the unified knowledge fragments obtained through conflict resolution to construct and store a cross-domain knowledge graph with entity-relationship-entity as the core structure. M4. Continuously monitor the update dynamics of multi-source heterogeneous knowledge sources. When a valid change is detected, automatically acquire the original data of the knowledge source that has changed, perform the fusion process of steps M1-M3 on it, and update the fused knowledge increment to the existing cross-domain knowledge graph. M5. After each incremental update is completed, a new round of logical consistency checks is automatically performed across the entire graph using a graph neural network to ensure that the newly added knowledge does not logically contradict existing knowledge.

4. The intelligent proofreading and content correction system for digital textbooks based on deep learning according to claim 1, characterized in that: The semantic understanding and reasoning module includes: N1. Receive the unified document object model, call the multimodal pre-trained model, and map the text and non-text media content in it to a high-dimensional semantic space to generate the corresponding deep semantic encoding vector. N2. Based on the deep semantic encoding vector, identify and extract named entities from the text content, and link them with entities in the cross-domain knowledge graph to generate an enhanced semantic representation that integrates external knowledge. N3. Based on the enhanced semantic representation, locate the statement representing the fact from the text content; compare the semantics of the statement with the corresponding facts recorded in the cross-domain knowledge graph; if there is a semantic conflict, it is determined to be a factual error, and the error information and evidence in the cross-domain knowledge graph are recorded to generate factual error data. N4. Traverse the unified document object model and scan the semantic representations of multiple statements for the same entity; based on the predefined logical rules in the cross-domain knowledge graph, analyze whether there are any incompatible logical conflicts between the semantic representations of the multiple statements; if a conflict is detected, it is determined to be a logical error, and the conflicting statement pair and its contradictions are recorded to generate logical error data. N5. For detected factual and logical errors, an evidence chain is automatically generated; the evidence chain includes error location, error type, and error analysis conclusion.

5. The intelligent proofreading and content correction system for digital textbooks based on deep learning according to claim 1, characterized in that: The cross-modal verification module includes: P1. Receive all semantically related text-media element pairs from the unified document object model generated by the multimodal parsing module; P2. Call the cross-modal pre-trained model to map the text content and non-text media content in each received element pair to the same shared semantic space, and generate text alignment vectors and media alignment vectors that can be directly measured for similarity. P3. In the semantic space, calculate the similarity between the text alignment vector and the media alignment vector of each pair of related elements, and generate a quantified semantic consistency score for the element pair. P4. Compare the semantic consistency score with the consistency threshold that is dynamically adjusted based on historical verification data. If the score is lower than the threshold, it is determined that there is a multimodal inconsistency error in the text-media element pair, and a preliminary inconsistency record is generated. P5. Integrate all detected inconsistencies and associate them with their original positions in the document object model to obtain multimodal inconsistency error data.

6. The intelligent proofreading and content correction system for digital textbooks based on deep learning according to claim 1, characterized in that: The decision generation and feedback module includes: Q1. Receive and aggregate factual error data and logical error data from the semantic understanding and reasoning module, as well as multimodal inconsistency error data from the cross-modal verification module; Q2. Deduplicatize, merge, and classify the aggregated error data to form a structured error list; Q3. Based on the error type, assess the severity and scope of impact according to preset rules, and prioritize the error list accordingly. Q4. Using the suggestion generator based on the deep learning model, generate correction suggestions for each error according to the error type, priority ranking, and cross-domain knowledge graph; Q5. Present the sorted error list and correction suggestions to the administrator through the interactive interface, and record all the administrator's processing records.

7. The intelligent proofreading and content correction system for digital textbooks based on deep learning according to claim 1, characterized in that: The continuous learning module includes: H1. Collect interactive feedback data from the decision generation and feedback module regarding the administrator's adoption, rejection, and modification of the correction suggestions; H2. Based on the interactive feedback data, valuable positive and negative examples are selected to construct a training sample set for model optimization. H3. Using the constructed training sample set, the multimodal pre-trained model, cross-modal pre-trained model, and suggestion generator are periodically optimized through an incremental learning mechanism. H4. Manage the optimized model version and deploy the performance-enhanced model to the corresponding production environment through a secure process.

Images