Real-time interactive semantic correction and mapping knowledge domain association system for inscription

Abstract

The invention provides an inscription-oriented real-time interactive semantic correction and knowledge graph association system, and belongs to the technical field of inscription knowledge graphs, the inscription-oriented real-time interactive semantic correction and knowledge graph association system comprises a semantic error correction engine unit, an AR visual interaction unit and a dynamic knowledge graph unit, the semantic error correction engine unit is connected with the AR visual interaction unit, and the dynamic knowledge graph unit is connected with the AR visual interaction unit. The AR visual interaction unit is connected with the dynamic knowledge graph unit, and the semantic error correction engine unit is used for carrying out ancient Chinese context correction on text distortion of inscription caused by long years, inscription errors, common and false characters and transliterated characters and outputting a corrected version conforming to a historical context. According to the method, a unidirectional mode of automatic correction and manual verification of a traditional machine is broken through, a bidirectional enhanced closed loop of machine pre-correction, user refinement and model iteration is constructed, and real-time association and reasoning of knowledge are realized through entity alignment and a graph neural network.

Description

Technical Field

[0001] This invention relates to the field of knowledge graph technology for stone inscriptions, and more particularly to a real-time interactive semantic correction and knowledge graph association system for stone inscriptions. Background Technology

[0002] Due to natural weathering (acid rain erosion, biological adhesion) and preservation environment (temperature and humidity fluctuations), the inscriptions on the stone tablets generally suffer from problems such as blurred characters, damaged strokes, and dirty surfaces, making it difficult to extract and utilize the information they carry efficiently and accurately.

[0003] Traditional epigraphic research relies on manual rubbings, handwritten records, and expert identification, which suffers from low efficiency, strong subjectivity, and poor reusability. On the one hand, the rubbing process may cause secondary damage to the epigraph; on the other hand, manual identification is limited by the scholar's experience, making it prone to errors in interpreting rare characters, variant characters, and variant forms, and hindering the systematic organization of large-scale epigraphs. With the development of digital technology, the digital acquisition of epigraphs (such as high-resolution photography and 3D scanning) has become mainstream. However, the storage and management of digitized images (such as rapid retrieval of massive images), the extraction of textual information (such as the recognition of fuzzy fonts), and the association of semantic knowledge (such as cross-episode analysis of historical events) still face technical bottlenecks and urgently require intelligent solutions. Therefore, it is necessary to design a real-time interactive semantic correction and knowledge graph association system for epigraphs. Summary of the Invention

[0004] The purpose of this invention is to provide a real-time interactive semantic correction and knowledge graph association system for stone inscriptions, thereby solving the technical problems mentioned in the background art.

[0005] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0006] The real-time interactive semantic correction and knowledge graph association system for stone inscriptions includes a semantic error correction engine unit, an AR visualization interaction unit, and a dynamic knowledge graph unit. The semantic error correction engine unit is connected to the AR visualization interaction unit, which in turn is connected to the dynamic knowledge graph unit. The semantic error correction engine unit is used to correct the text distortion caused by the age of the inscription, engraving errors, phonetic loan characters, and transliterated characters in the ancient Chinese context, and outputs a corrected version that conforms to the historical context. The AR visualization interaction unit uses mobile AR technology to realize the real-time overlay of the original appearance of the inscription and the corrected information, allowing users to view, correct, and learn immediately. The dynamic knowledge graph unit associates the corrected inscription information with knowledge of historical figures, events, systems, and geography, and constructs a traceable and inferable multi-dimensional cultural knowledge network graph.

[0007] Furthermore, the semantic error correction engine unit includes a corpus construction module, a domain adaptation model module, and a hybrid correction strategy module. The corpus construction module integrates authoritative digitized resources of inscriptions, annotates error types such as phonetic loan characters, transliterated characters, and colloquial characters with correction tags, associates with philological tools, constructs a knowledge base of ancient Chinese characters, phonetics, and interpretations, and receives user correction feedback in real time, forming an incremental update mechanism for the historical error database and the user contribution database. The domain adaptation model module is used for pre-training on the corpus of the Siku Quanshu (Complete Library of the Four Treasuries) of ancient Chinese and the proper nouns of inscriptions, strengthening the understanding of ancient Chinese grammar and proper nouns of inscriptions. Using the annotated corpus of inscription errors as the training set, it adopts a sequence-to-sequence framework, inputs sentences of ancient and medieval Chinese containing errors, and outputs corrected modern standard Chinese. It captures contextual dependencies through a conditional random field layer, distinguishes error patterns of phonetic loan characters, transliterated characters, and similar-looking characters, and improves the targeting of correction. The hybrid correction strategy module directly outputs the correction results for errors with a confidence level higher than the set level, and for errors with a confidence level lower than the set level, it calls the historical document database for cross-text verification, combines the language model probability to output the top 3 candidate corrections for users to choose from.

[0008] Furthermore, the AR visualization interaction unit includes an inscription recognition and localization module, an interactive interface module, and a feedback loop module. The inscription recognition and localization module uses the YOLOv11 model and CRNN model on the mobile client for image enhancement preprocessing to improve the recognition accuracy of blurred or worn inscriptions. Through synchronous localization and map construction methods, it performs 3D modeling of the inscription, records its spatial position and posture, and ensures accurate alignment between AR annotations and physical inscriptions. The interactive interface module is used for layered display and collaborative annotation. In the layered display, the bottom layer is the original image of the inscription, the middle layer is a semi-transparent correction layer, and the top layer supports user gesture operations. Specifically, long-pressing to select erroneous text will pop up a candidate correction box, and sliding to switch historical documents for corroboration. In collaborative annotation, users can collaboratively correct in real time. Annotation data is synchronized via WebSocket, and the annotation results are automatically stored in the user contribution library. Contributors can check the adoption status of their modifications. The feedback loop module is used to upload manually corrected text and added annotations by users to the backend in real time, triggering incremental training of the semantic error correction engine, forming a positive cycle of user error correction, model optimization, and more accurate correction.

[0009] Furthermore, the dynamic knowledge graph unit includes an entity and relation extraction module, a dynamic graph construction and update module, and a retrieval visualization module. The entity and relation extraction module is used to extract core entities and implicit relations from the corrected inscriptions through named entity recognition and relation extraction models, and to perform entity alignment by combining with a historical knowledge base to achieve homonymous and alias association. The dynamic graph construction and update module uses a hybrid graph database for storage, with nodes representing historical entities and edges representing relations. A vector database stores the semantic vectors of the inscription text. When a new inscription is corrected or a user adds a new association, it triggers an incremental update of the graph, automatically expanding the relations of related nodes, and optimizing the entity embedding representation through a graph neural network to improve the accuracy of association. The retrieval visualization module is used for timeline retrieval, spatial map retrieval, and relation network retrieval. Users can zoom or drag the knowledge graph with gestures, and click on nodes to jump to the corresponding inscription, historical document, or 3D scene.

[0010] Furthermore, in the hybrid correction strategy module, errors with a confidence level higher than the set level that can be clearly judged through training or domain knowledge base have the characteristics of high frequency, regularity, and strong semantic association. These include common homophones, fixed misprints of similar-looking characters, punctuation errors, and segmentation errors. Through the built-in error correction mapping table, the error type is matched, and standard characters or corrected text are directly output.

[0011] Furthermore, in the hybrid correction strategy module, fuzzy errors with a reliability level below the set threshold (meaning no unique correct answer can be determined) are characterized by low frequency, obscurity, and context dependence. These include obscure transliterations, missing characters, ambiguity in sentence segmentation, and misspellings of proper nouns. The specific processing procedure is as follows: cross-text verification, calling the historical document database to perform contextual similarity retrieval, finding historical original texts that highly match the inscription text fragments, language model probability output, using a pre-trained ancient Chinese language model to mask and fill in the error positions, generating several candidate corrections, and outputting the Top 3 in probability order, user selection confirmation, merging the high-matching results of cross-text verification with the Top 3 candidates of the language model, and displaying them to the user through an interactive interface, allowing the user to finally confirm the correction result based on professional knowledge or context.

[0012] Furthermore, when constructing the dataset, the YOLOv11 model collects images of inscriptions with different materials, wear levels, and shooting angles, and labels the inscription areas and corresponding text labels. The backbone network of the YOLOv11 model uses the C2f module instead of the C3 module to reduce the amount of computation and improve the ability to extract features of small targets. SPPF is used instead of PANet in the neck to reduce the information loss caused by downsampling. The head adopts a decoupled head to separate the detection and classification tasks and improve the robustness of localization of blurred inscriptions.

[0013] Furthermore, in the CRNN model, the height of the inscription area is adjusted to 32 pixels by bilinear interpolation, the width is scaled proportionally, feature extraction is enhanced, an attention mechanism is introduced to enhance the response to key stroke features in the worn area, sequence modeling is optimized, a bidirectional LSTM model is used to capture the contextual dependencies between characters, the output layer adopts the CTC loss function to solve the alignment problem between characters and labels, and a special dictionary containing ancient characters, variant characters, and variant characters is constructed.

[0014] The present invention, by adopting the above-described technical solution, has the following beneficial effects:

[0015] This invention breaks through the traditional one-way model of automatic machine correction and manual verification, and constructs a two-way enhanced closed loop of machine pre-correction, user refinement and model iteration. It realizes real-time association and reasoning of knowledge through entity alignment and graph neural networks, and opens up horizontal associations between inscriptions and biographies of people in multiple sources of documents such as official history, local chronicles and literati notes. It constructs the time evolution of historical events, provides efficient inscription correction tools and knowledge association platforms for history and archaeology, shortens the document processing cycle, and assists cultural relic protection experts in assessing the degree of damage to inscriptions by superimposing the original text and correction information using AR. Attached Figure Description

[0016] Figure 1 This is a block diagram illustrating the system principle of this invention;

[0017] Figure 2 This is a block diagram of the semantic error correction engine unit module of the present invention;

[0018] Figure 3 This is a block diagram of the AR visualization interaction unit module of the present invention;

[0019] Figure 4 This is a block diagram of the dynamic knowledge graph unit module of the present invention. Detailed Implementation

[0020] To make the objectives, technical solutions, and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings and preferred embodiments. However, it should be noted that many details listed in the specification are merely to provide the reader with a thorough understanding of one or more aspects of the present invention, and these aspects of the invention can be implemented even without these specific details.

[0021] like Figure 1As shown, the real-time interactive semantic correction and knowledge graph association system for stone inscriptions includes a semantic error correction engine unit, an AR visualization interaction unit, and a dynamic knowledge graph unit. The semantic error correction engine unit is connected to the AR visualization interaction unit, which in turn is connected to the dynamic knowledge graph unit. The semantic error correction engine unit is used to correct the text distortion caused by the age of the inscription, engraving errors, phonetic loan characters, and transliterated characters in the ancient Chinese context, and outputs a corrected version that conforms to the historical context. The AR visualization interaction unit uses mobile AR technology to realize the real-time overlay of the original appearance of the inscription and the corrected information, allowing users to view, correct, and learn immediately. The dynamic knowledge graph unit associates the corrected inscription information with knowledge of historical figures, events, systems, and geography, and constructs a traceable and inferable multi-dimensional cultural knowledge network graph.

[0022] In embodiments of the present invention, such as Figure 2 As shown, the semantic error correction engine unit includes a corpus construction module, a domain adaptation model module, and a hybrid correction strategy module. The corpus construction module integrates authoritative digitized resources of inscriptions, annotates error types such as phonetic loan characters, transliterated characters, and colloquial characters with correction tags, associates with philological tools, constructs a knowledge base of ancient Chinese characters, phonetics, and interpretations, and receives user correction feedback in real time, forming an incremental update mechanism for the historical error database and the user contribution database. The domain adaptation model module is used for pre-training on the corpus of the Siku Quanshu (Complete Library of the Four Treasuries) of ancient Chinese and the proper nouns of inscriptions, strengthening the understanding of ancient Chinese grammar and proper nouns of inscriptions. Using the annotated corpus of erroneous inscriptions as the training set, it adopts a sequence-to-sequence framework, inputs sentences of ancient and medieval Chinese containing errors, and outputs corrected modern standard Chinese. It captures contextual dependencies through a conditional random field layer, distinguishes error patterns of phonetic loan characters, transliterated characters, and similar-looking characters, and improves the targeting of correction. The hybrid correction strategy module directly outputs the correction results for errors with a confidence level higher than the set level, and for errors with a confidence level lower than the set level, it calls the historical document database for cross-text verification, and combines the language model probability to output the Top 3 candidate corrections for users to choose from.

[0023] In embodiments of the present invention, such as Figure 3As shown, the AR visualization interaction unit includes an inscription recognition and localization module, an interactive interface module, and a feedback loop module. The inscription recognition and localization module uses the YOLOv11 model and CRNN model on the mobile client for image enhancement preprocessing to improve the recognition accuracy of blurred or worn inscriptions. Through synchronous localization and map construction methods, it performs 3D modeling of the inscription, records its spatial position and posture, and ensures accurate alignment between AR annotations and physical inscriptions. The interactive interface module is used for layered display and collaborative annotation. In the layered display, the bottom layer is the original image of the inscription, the middle layer is a semi-transparent correction layer, and the top layer supports user gesture operations. Specifically, long-pressing to select erroneous text will pop up a candidate correction box, and sliding to switch historical documents for corroboration. In the collaborative annotation, users can collaboratively correct in real time. The annotation data is synchronized via WebSocket, and the annotation results are automatically stored in the user contribution library. Contributors can check the adoption status of their modifications. The feedback loop module is used to upload manually corrected text and added annotations by users to the backend in real time, triggering incremental training of the semantic error correction engine, forming a positive cycle of user error correction, model optimization, and more accurate correction.

[0024] In embodiments of the present invention, such as Figure 4 As shown, the dynamic knowledge graph unit includes an entity and relation extraction module, a dynamic graph construction and update module, and a retrieval visualization module. The entity and relation extraction module uses named entity recognition and relation extraction models to extract core entities and implicit relations from the corrected inscriptions, and combines this with a historical knowledge base for entity alignment, enabling association of homonyms and aliases. The dynamic graph construction and update module uses a hybrid graph database for storage, where nodes represent historical entities, edges represent relations, and a vector database stores the semantic vectors of the inscription text. When a new inscription is corrected or a user adds a new association, the graph is incrementally updated, automatically expanding the relationships between related nodes. Entity embedding representations are optimized through a graph neural network to improve association accuracy. The retrieval visualization module is used for timeline retrieval, spatial map retrieval, and relation network retrieval. Users can zoom or drag the knowledge graph using gestures, and click on nodes to jump to the corresponding inscription, historical document, or 3D scene. Timeline retrieval, for example, allows viewing all inscriptions related to Su Shi from the Yuanfeng period of the Northern Song Dynasty; spatial map retrieval, for example, displays the distribution of inscriptions recording the Maritime Silk Road throughout the country; and relation network retrieval, for example, displays a graph of Su Shi's interactions with contemporary literati.

[0025] In this embodiment of the invention, the hybrid correction strategy module includes errors with a confidence level higher than the set level that can be clearly judged through training or domain knowledge base. These errors have high frequency, regularity, and strong semantic association characteristics, including common homophones, fixed misprints of similar-looking characters, punctuation errors, and segmentation errors. Through the built-in error correction mapping table, the error type is matched, and standard characters or corrected text are directly output.

[0026] In this embodiment of the invention, the hybrid correction strategy module identifies fuzzy errors with a confidence level below the set threshold (meaning a unique correct answer cannot be determined). These errors are characterized by low frequency, obscurity, and context dependence, including obscure transliterations, missing characters, ambiguity in sentence segmentation, and misspellings of proper nouns. The specific processing steps are as follows: cross-text verification is performed by calling a historical document database to conduct contextual similarity retrieval, searching for historical original texts that highly match the inscription fragments, and outputting probabilistic outputs from the language model. A pre-trained ancient Chinese language model is used to mask and fill in the error locations, generating several candidate corrections. The Top 3 are then output in probability order. The user selects and confirms the result, merging the high-matching results from the cross-text verification with the Top 3 candidates from the language model and displaying them to the user through an interactive interface. The user then confirms the correction result based on their professional knowledge or the context.

[0027] In this embodiment of the invention, when constructing the dataset, the YOLOv11 model collects images of inscriptions with different materials, different degrees of wear, and different shooting angles, and labels the inscription areas and corresponding text labels. The backbone network of the YOLOv11 model uses the C2f module instead of the C3 module to reduce the amount of computation and improve the ability to extract features of small targets. SPPF is used instead of PANet in the neck to reduce the information loss caused by downsampling. The head adopts a decoupled head to separate the detection and classification tasks and improve the robustness of localization of blurred inscriptions.

[0028] In this embodiment of the invention, the CRNN model adjusts the height of the inscription area to 32 pixels through bilinear interpolation, scales the width proportionally, enhances feature extraction, introduces an attention mechanism to enhance the response to key stroke features in the worn area, optimizes sequence modeling, uses a bidirectional LSTM model to capture the contextual dependencies between characters, adopts the CTC loss function in the output layer to solve the alignment problem between characters and labels, and constructs a dedicated dictionary containing archaic characters, variant characters, and variant characters.

[0029] This system provides a standardized semantic correction and knowledge association knowledge spectrum system for digital museums and cultural heritage digitization projects. It constructs a closed-loop mechanism of perception-correction-association-interaction to achieve automated and precise semantic correction of inscriptions, dynamic association of historical knowledge, and multi-dimensional retrieval, enabling immersive and participatory research for scholars and the public.

[0030] Matters not covered in this invention are common knowledge.

[0031] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should also be regarded as within the scope of protection of the present invention.

Claims

1. A real-time interactive semantic correction and knowledge graph association system for stone inscriptions, characterized by: It includes a semantic error correction engine unit, an AR visualization interaction unit, and a dynamic knowledge graph unit. The semantic error correction engine unit is connected to the AR visualization interaction unit, which in turn is connected to the dynamic knowledge graph unit. The semantic error correction engine unit is used to correct the text distortion caused by the age of the inscription, engraving errors, phonetic loan characters, and transliterated characters in the ancient Chinese context, and outputs a corrected version that conforms to the historical context. The AR visualization interaction unit uses mobile AR technology to realize the real-time overlay of the original appearance of the inscription and the corrected information, allowing users to view, correct, and learn immediately. The dynamic knowledge graph unit associates the corrected inscription information with knowledge of historical figures, events, systems, and geography to construct a traceable and inferable multi-dimensional cultural knowledge network graph.

2. The real-time interactive semantic correction and knowledge graph association system for stone inscriptions according to claim 1, characterized in that: The semantic error correction engine unit includes a corpus construction module, a domain adaptation model module, and a hybrid correction strategy module. The corpus construction module integrates authoritative digitized resources of inscriptions, annotates error types such as phonetic loan characters, transliterated characters, and colloquial characters with correction tags, associates with philological tools, constructs a knowledge base of ancient Chinese characters, phonetics, and definitions, and receives user correction feedback in real time, forming an incremental update mechanism for the historical error database and the user contribution database. The domain adaptation model module is used for pre-training on the corpus of the Siku Quanshu (Complete Library of the Four Treasuries) of ancient Chinese and proper nouns from inscriptions to enhance the understanding of ancient Chinese grammar and proper nouns from inscriptions. Using the annotated corpus of inscription errors as the training set, it adopts a sequence-to-sequence framework, inputs sentences of ancient and medieval Chinese containing errors, and outputs corrected modern standard Chinese. It captures contextual dependencies through a conditional random field layer, distinguishes error patterns of phonetic loan characters, transliterated characters, and similar-looking characters, and improves the targeting of correction. The hybrid correction strategy module directly outputs correction results for errors with a confidence level higher than the set level, and for errors with a confidence level lower than the set level, it calls the historical document database for cross-text verification and combines the language model probability to output the top 3 candidate corrections for users to choose from.

3. The real-time interactive semantic correction and knowledge graph association system for stone inscriptions according to claim 1, characterized in that: The AR visualization interaction unit includes an inscription recognition and localization module, an interactive interface module, and a feedback loop module. The inscription recognition and localization module uses the YOLOv11 model and CRNN model on the mobile client for image enhancement preprocessing to improve the recognition accuracy of blurred or worn inscriptions. Through synchronous localization and map construction methods, it performs 3D modeling of the inscription, records its spatial position and posture, and ensures accurate alignment between AR annotations and physical inscriptions. The interactive interface module is used for layered display and collaborative annotation. In the layered display, the bottom layer is the original image of the inscription, the middle layer is a semi-transparent correction layer, and the top layer supports user gesture operations. Specifically, long-pressing to select erroneous text will pop up a candidate correction box, and sliding to switch historical documents for corroboration. In collaborative annotation, users can collaboratively correct in real time. Annotation data is synchronized via WebSocket, and the annotation results are automatically stored in the user contribution library. Contributors can check the adoption status of their modifications. The feedback loop module is used to upload manually corrected text and added annotations by users to the backend in real time, triggering incremental training of the semantic error correction engine, forming a positive cycle of user error correction, model optimization, and more accurate correction.

4. The real-time interactive semantic correction and knowledge graph association system for stone inscriptions according to claim 1, characterized in that: The dynamic knowledge graph unit includes an entity and relation extraction module, a dynamic graph construction and update module, and a retrieval visualization module. The entity and relation extraction module extracts core entities and implicit relations from the corrected inscriptions using named entity recognition and relation extraction models. It then aligns entities with a historical knowledge base to achieve associations based on homonyms and aliases. The dynamic graph construction and update module uses a hybrid graph database for storage, where nodes represent historical entities, edges represent relations, and a vector database stores the semantic vectors of the inscription text. When a new inscription is corrected or a user adds a new association, the graph is incrementally updated, automatically expanding the relationships between related nodes. It also optimizes entity embedding representations through a graph neural network to improve association accuracy. The retrieval visualization module is used for timeline retrieval, spatial map retrieval, and relation network retrieval. Users can zoom or drag the knowledge graph using gestures and click on nodes to jump to the corresponding inscription, historical document, or 3D scene.

5. The real-time interactive semantic correction and knowledge graph association system for stone inscriptions according to claim 2, characterized in that: In the hybrid correction strategy module, errors with a confidence level higher than the set level are those that can be clearly identified through training or domain knowledge base. These errors are characterized by high frequency, regularity, and strong semantic association, including common homophones, fixed misprints of similar-looking characters, punctuation errors, and segmentation errors. Through the built-in error correction mapping table, the error type is matched, and standard characters or corrected text are directly output.

6. The real-time interactive semantic correction and knowledge graph association system for stone inscriptions according to claim 2, characterized in that: In the hybrid correction strategy module, fuzzy errors with a reliability level below the set threshold (meaning no unique correct answer can be determined) are characterized by low frequency, obscurity, and context dependence. These include obscure transliterations, missing characters, ambiguity in sentence segmentation, and misspellings of proper nouns. The specific processing steps are as follows: cross-text verification, calling a historical document database to perform contextual similarity retrieval, searching for historical original texts that highly match the inscription fragments, language model probability output, using a pre-trained ancient Chinese language model to mask and fill in the error locations, generating several candidate corrections, and outputting the Top 3 in probability order, user confirmation, merging the high-matching results from cross-text verification with the Top 3 candidates from the language model, and displaying them to the user through an interactive interface, allowing the user to finally confirm the correction result based on professional knowledge or context.

7. The real-time interactive semantic correction and knowledge graph association system for stone inscriptions according to claim 3, characterized in that: When constructing the dataset for the YOLOv11 model, images of stone inscriptions with different materials, wear levels, and shooting angles are collected, and the inscription areas and corresponding text labels are labeled. The backbone network of the YOLOv11 model uses the C2f module instead of the C3 module to reduce the amount of computation and improve the ability to extract features of small targets. SPPF is used instead of PANet in the neck to reduce the information loss caused by downsampling. A decoupled head is used in the head to separate the detection and classification tasks and improve the robustness of localization of blurred stone inscriptions.

8. The real-time interactive semantic correction and knowledge graph association system for stone inscriptions according to claim 3, characterized in that: In the CRNN model, the height of the inscription area is adjusted to 32 pixels by bilinear interpolation, and the width is scaled proportionally. Feature extraction is enhanced, an attention mechanism is introduced to enhance the response to key stroke features in the worn area, sequence modeling is optimized, a bidirectional LSTM model is used to capture the contextual dependencies between characters, the output layer adopts the CTC loss function to solve the alignment problem between characters and labels, and a special dictionary containing ancient characters, variant characters, and variant characters is constructed.

Images