Deep learning based ancient chinese automatic punctuation and semantic analysis system

Abstract

The application discloses an ancient Chinese automatic sentence breaking and semantic analysis system based on deep learning, and relates to the technical field of ancient Chinese analysis.The system combines a gating recurrent unit (GRU) and a convolutional neural network (CNN) through a sentence breaking module, combines image character features, and can effectively process semantic ambiguity, long-distance dependence, and problems of homophones and variant characters.A knowledge graph containing historical and cultural common knowledge and dynamic relationships between words is constructed through a semantic analysis module, prior knowledge is injected through a graph neural network, and the depth of semantic understanding is improved.A sentence breaking result containing sentence breaking basis and a visual semantic analysis result are provided through an output module, so that users can clearly understand the processing logic and the trustworthiness and convenience are improved.The system can efficiently process a large number of unlabeled ancient Chinese texts, fully excavate the value of various cultural heritage such as classics and inscriptions, promote the research progress of grammar and vocabulary, and help the inheritance and spread of ancient Chinese culture.

Description

Technical Field

[0001] This invention relates to the field of classical Chinese analysis technology, specifically to a system for automatic sentence segmentation and semantic analysis of classical Chinese based on deep learning. Background Technology

[0002] Existing deep learning-based automatic sentence segmentation and semantic analysis systems for Classical Chinese still have the following drawbacks in practical use:

[0003] Existing publicly available literature—CN109829159B—discloses an integrated automatic lexical analysis method for classical Chinese texts, including the following steps: pre-training a Word2Vec model to obtain semantically characteristic character vectors of classical Chinese; adding information data appearing in documents from various dynasties to a database of proper nouns in ancient books to form several proper noun entries; adjusting the parameters of the Bi-LSTM-CRF neural network model; preprocessing the final training corpus into a model-readable form; loading it into the neural network model; continuously iterating and learning; and automatically evaluating the annotation results of the test corpus. This invention employs an integrated annotation method encompassing sentence segmentation, word segmentation, and part-of-speech tagging, eliminating the repetitive annotation process of multiple subtasks in lexical analysis and avoiding the multi-level propagation of repeated annotation errors. It utilizes a deep learning model that automatically learns rich linguistic features, eliminating the need for manually customized feature templates in traditional machine learning. The annotation model described in this invention is accelerated by GPU hardware, significantly shortening model training time and achieving much higher efficiency than traditional machine learning models. The input data consists primarily of classical Chinese training corpus in text form, excluding image data. The processing focuses on the lexical analysis of proper nouns (personal names, place names, etc.), and the text type does not explicitly include complex carriers such as inscriptions or bamboo slips, nor does it address the specific handling of variant characters or phonetic loan characters. Only a database of proper nouns in ancient texts is constructed, containing static proper nouns such as personal names and place names. Entries are selected and added to the training corpus through confidence calculation; essentially, this is corpus expansion, without forming a knowledge graph or a deep fusion mechanism between knowledge and the model. Furthermore, it focuses solely on "sentence segmentation, word segmentation, and part-of-speech tagging." The integrated lexical analysis does not involve discourse-level semantic analysis, and the task is limited to basic lexical processing at the sentence level; it lacks a domain adaptation mechanism and only processes general proper nouns; it lacks a self-supervised learning mechanism, and data augmentation only supplements entries through a proper noun database, relying on manually labeled corpora and making insufficient use of unlabeled data.

[0004] Classical Chinese, as an important carrier of Chinese civilization, contains rich historical, cultural, and academic value. However, the absence of punctuation, the complexity of its grammatical structure, and the phenomena of variant characters and phonetic loan characters formed during historical evolution have led to problems such as low efficiency, strong subjectivity, and susceptibility to academic disputes in traditional manual punctuation and semantic analysis. With the development of digital technology, the use of artificial intelligence to automate the processing of classical Chinese has become a key requirement for academic research and cultural inheritance.

[0005] In Classical Chinese, polysemy and ellipsis of sentences are common, making it difficult for models to accurately determine sentence boundaries through local context. In addition, the logical relationships in long sentences may span dozens of characters, and traditional BiLSTM models have limited ability to model long-distance dependencies. In particular, the existence of phonetic loan characters and variant characters makes it difficult for models to directly map semantics through character shapes. Although existing models learn some phonetic loan rules through pre-training, they still make misjudgments in unlabeled corpora. Inscriptions, bamboo slips, and other texts are often accompanied by image data, and existing models are mostly trained based on pure text, lacking fusion analysis of image features.

[0006] Semantic analysis of Classical Chinese relies on labeled corpora, but publicly available high-quality labeled data is extremely scarce. For example, semantic annotation of ancient Chinese medicine texts needs to be combined with professional terminology, and existing models trained on general corpora experience a drop in F1 score of more than 30% when transferred to the field of Chinese medicine. In addition, although public datasets such as the "Twenty-Four Histories" contain nearly 20,000 entity annotations, their coverage is concentrated on historical figures and events, and they do not provide sufficient support for semantic types such as literary allusions and philosophical concepts. Semantic understanding of Classical Chinese requires historical background knowledge (such as the system of rites and official positions) and cultural allusions (such as the analysis of allusions). Existing models mostly learn common sense indirectly through pre-trained corpora, but lack explicit knowledge integration. In particular, semantic analysis focuses on part-of-speech tagging and entity recognition at the single-sentence level, and lacks the construction of discourse-level semantic networks. Summary of the Invention

[0007] The purpose of this invention is to provide a deep learning-based automatic sentence segmentation and semantic analysis system for Classical Chinese to solve the aforementioned problems.

[0008] To achieve the above objectives, this invention provides the following technical solution: a deep learning-based automatic sentence segmentation and semantic analysis system for Classical Chinese, comprising:

[0009] The data input module is used to input unlabeled ancient Chinese text and corresponding image data. The ancient Chinese text includes classical texts and inscriptions, and the image data includes image information of inscriptions and bamboo slips.

[0010] The preprocessing module is connected to the data input module. The preprocessing module performs cleaning, word segmentation, and preliminary identification of variant characters and phonetic loan characters on the ancient Chinese text, and performs noise reduction, enhancement, and character shape feature extraction on the image data.

[0011] The sentence segmentation module, connected to the preprocessing module, employs a deep learning model that integrates gated recurrent units (GRUs) and convolutional neural networks (CNNs). It combines the glyph features of the preprocessed image to automatically segment the preprocessed Classical Chinese text, addressing semantic ambiguity, long-distance dependencies, and sentence segmentation issues arising from variant characters and phonetic loan characters.

[0012] The formula for calculating the update gate of GRU is as follows: ,

[0013] The formula for resetting the door is as follows: ,

[0014] The formula for calculating the candidate hidden state is: ,

[0015] The formula for calculating hidden state is: ;

[0016] The semantic analysis module is connected to the sentence segmentation module. The semantic analysis module constructs a knowledge graph of classical Chinese containing historical and cultural common sense. Through graph neural network, the prior knowledge of lexical relationships and allusions in the knowledge graph is injected into the deep learning model to perform part-of-speech tagging, entity recognition and text-level semantic network construction on the classical Chinese text after sentence segmentation.

[0017] The output module connects the sentence segmentation module and the semantic analysis module, and outputs the sentence segmentation result and the semantic analysis result.

[0018] Furthermore, the data input module can also input bamboo and wooden slips texts, unearthed document texts, and different types of unannotated ancient Chinese texts, as well as corresponding image data of different carriers. The data input module transmits the input unannotated ancient Chinese texts and corresponding image data to the preprocessing module.

[0019] Furthermore, in the preprocessing module, the identification of variant characters adopts a method based on the ancient phonetic conversion rules, and the identification of variant characters is combined with character structure analysis. The preprocessing module transmits the processed ancient Chinese text and image character features to the sentence segmentation module.

[0020] Furthermore, the deep learning model that integrates GRU and CNN in the sentence segmentation module is pre-trained using large-scale unlabeled classical Chinese text and corresponding image data to enhance the robustness of the sentence segmentation module to complex text. The sentence segmentation module then transmits the segmented classical Chinese text to the semantic analysis module.

[0021] Furthermore, when processing classical Chinese texts with long-distance dependencies, the sentence segmentation module employs an attention mechanism to enhance its ability to capture long-distance logical relationships. The calculation formula for the attention mechanism is as follows:

[0022] , , ,

[0023] The sentence segmentation module performs sentence segmentation based on this and transmits the results to the semantic analysis module.

[0024] Furthermore, the ancient Chinese text knowledge graph in the semantic analysis module also includes the historical evolution of vocabulary, semantic field associations and other dynamic relationships. The semantic analysis module processes the ancient Chinese text after segmentation based on this knowledge graph.

[0025] Furthermore, when performing part-of-speech tagging and entity recognition, the semantic analysis module incorporates domain knowledge. When processing ancient Chinese texts from different fields such as traditional Chinese medicine classics and historical texts, it calls upon the professional knowledge sub-graphs of the corresponding fields. The semantic analysis module then transmits the semantic analysis results to the output module.

[0026] Furthermore, the semantic analysis module employs self-supervised learning and data augmentation methods, utilizing unlabeled corpora for masked language modeling and contrastive learning to alleviate the problem of insufficient labeled data. The semantic analysis module optimizes the processing in this way and transmits the results to the output module.

[0027] Furthermore, the output module outputs sentence segmentation results including sentence segmentation position and sentence segmentation basis, and outputs semantic analysis results including part-of-speech tagging results, entity recognition results, and document-level semantic network visualization. The output module receives sentence segmentation results from the sentence segmentation module and semantic analysis results from the semantic analysis module and outputs them.

[0028] Furthermore, it also includes a feedback module connected to the output module. The feedback module is used to receive user feedback information on the sentence segmentation results and semantic analysis results, and to use the feedback information to optimize the deep learning models in the sentence segmentation module and the semantic analysis module. The feedback module forms a data interaction with the sentence segmentation module and the semantic analysis module.

[0029] Compared with existing technologies, the deep learning-based automatic sentence segmentation and semantic analysis system for Classical Chinese provided by this invention has the following advantages:

[0030] This deep learning-based automatic sentence segmentation and semantic analysis system for Classical Chinese integrates a gated recurrent unit (GRU) and a convolutional neural network (CNN) in its sentence segmentation module. By combining image glyph features, it can effectively handle semantic ambiguity, long-distance dependencies, and issues related to variant characters and phonetic loan characters. The GRU dynamically captures contextual relationships, and the CNN extracts local features. With the assistance of image information, the system significantly reduces the sentence segmentation error rate, especially in the processing of complex texts such as inscriptions and bamboo slips. Through large-scale data pre-training, the system's adaptability to different types and domains of text is enhanced, and its robustness is greatly improved.

[0031] By constructing a knowledge graph containing historical and cultural common sense and dynamic word relationships through a semantic analysis module, and injecting prior knowledge through a graph neural network, the depth of semantic understanding is improved. By combining domain knowledge to call professional subgraphs, the application breadth of fields such as traditional Chinese medicine books and historical classics is expanded, and part-of-speech tagging and entity recognition are more accurate. Self-supervised learning and data augmentation alleviate the problem of insufficient labeled data and help to build a more complete chapter-level semantic network.

[0032] The output module provides punctuation results with supporting evidence and visualized semantic analysis results, enabling users to clearly understand the processing logic and enhancing trust and convenience. The feedback module forms an iterative optimization mechanism, continuously improving performance with increased usage. This saves researchers significant time and greatly improves academic research efficiency. The system efficiently processes large amounts of unannotated classical Chinese texts, fully exploring the cultural heritage value of classics and inscriptions. The constructed lexical relationship network provides new perspectives and tools for classical Chinese research, promoting progress in grammar and vocabulary research and contributing to the inheritance and dissemination of classical Chinese culture. Attached Figure Description

[0033] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present invention. For those skilled in the art, other drawings can be obtained based on these drawings.

[0034] Figure 1 This is a schematic diagram of the overall system flow of the present invention;

[0035] Figure 2 This is a partial system flow diagram of the present invention. Detailed Implementation

[0036] To enable those skilled in the art to better understand the technical solution of the present invention, the present invention will be further described in detail below with reference to the accompanying drawings.

[0037] Please see Figure 1 , 2A deep learning-based automatic sentence segmentation and semantic analysis system for Classical Chinese includes:

[0038] The data input module is used to input unlabeled classical Chinese texts and corresponding image data. The classical Chinese texts include classical texts and inscriptions, and the image data includes image information of inscriptions and bamboo slips.

[0039] The preprocessing module is connected to the data input module. The preprocessing module cleans, segments, and performs preliminary identification of variant characters and homophones in the ancient Chinese text. It also performs noise reduction, enhancement, and character feature extraction on the image data.

[0040] The sentence segmentation module, connected to the preprocessing module, employs a deep learning model that integrates gated recurrent units (GRUs) and convolutional neural networks (CNNs). It combines the glyph features of the preprocessed images to automatically segment the preprocessed Classical Chinese text, addressing semantic ambiguity, long-distance dependencies, and sentence segmentation issues arising from variant characters and phonetic loan characters.

[0041] The formula for calculating the update gate of GRU is as follows: ,

[0042] The formula for resetting the door is as follows: ,

[0043] The formula for calculating the candidate hidden state is: ,

[0044] The formula for calculating hidden state is: ;

[0045] The semantic analysis module connects to the sentence segmentation module. The semantic analysis module constructs a knowledge graph of classical Chinese containing historical and cultural common sense. Through graph neural networks, the prior knowledge of lexical relationships and allusions in the knowledge graph is injected into the deep learning model to perform part-of-speech tagging, entity recognition and text-level semantic network construction on the segmented classical Chinese text.

[0046] The output module connects the sentence segmentation module and the semantic analysis module, and outputs the sentence segmentation results and semantic analysis results.

[0047] The data input module can also input bamboo and wooden slips texts, unearthed document texts, and different types of unannotated ancient Chinese texts, as well as corresponding image data of different carriers. The data input module transmits the input unannotated ancient Chinese texts and corresponding image data to the preprocessing module.

[0048] The data input module adopts a multi-source data interface, supporting input of unannotated classical Chinese texts in formats such as TXT and PDF, such as classic texts like the Analects and Records of the Grand Historian, as well as images of stone inscriptions and bamboo slips in JPG and PNG formats. The module performs format verification on the input text and image data to ensure that the data meets the requirements of subsequent processing, and then transmits it synchronously to the preprocessing module.

[0049] In the preprocessing module, the identification of phonetic loan characters adopts a method based on the ancient phonetic conversion rules, and the identification of variant characters combines character shape structure analysis. The preprocessing module transmits the processed ancient Chinese text and image character shape features to the sentence segmentation module. For the ancient Chinese text, it is first cleaned to remove redundant symbols, garbled text and other interference information. Then, it is segmented using a dictionary-based and statistical word segmentation method. For phonetic loan characters, a recognition rule base is constructed based on the ancient phonetic conversion rules in "Shuowen Jiezi" and "Gu Hanyu Tongjia Zi Da Zidian" for preliminary identification. For variant characters, it is identified by comparing with a standard character shape library, combined with character shape structure features. For image data, Gaussian filtering is used for noise reduction, histogram equalization is used for enhancement, and then edge detection algorithm is used to extract character shape features. After processing, the preprocessed ancient Chinese text and image character shape features are transmitted to the sentence segmentation module.

[0050] The deep learning model that integrates GRU and CNN in the sentence segmentation module is pre-trained on a large scale of unlabeled ancient Chinese text and corresponding image data to enhance the robustness of the sentence segmentation module to complex text. The sentence segmentation module then transmits the segmented ancient Chinese text to the semantic analysis module.

[0051] When processing Classical Chinese texts with long-distance dependencies, the sentence segmentation module employs an attention mechanism to enhance its ability to capture long-distance logical relationships. The formula for calculating the attention mechanism is as follows:

[0052] , , ,

[0053] The sentence segmentation module performs sentence segmentation based on this and transmits the results to the semantic analysis module.

[0054] The ancient Chinese text knowledge graph in the semantic analysis module also includes the historical evolution of vocabulary, semantic field associations and other dynamic relationships. The semantic analysis module processes the ancient Chinese text after segmentation based on this knowledge graph. It constructs an ancient Chinese knowledge graph containing historical and cultural common sense, such as ancient rituals and official systems. The construction of the knowledge graph is completed by crawling authoritative ancient book databases, academic literature and other resources, and is manually reviewed and proofread.

[0055] When performing part-of-speech tagging and entity recognition, the semantic analysis module incorporates domain knowledge. When processing ancient Chinese texts from different fields such as traditional Chinese medicine classics and historical texts, it calls upon the relevant domain-specific professional knowledge sub-graphies, and the semantic analysis module transmits the semantic analysis results to the output module.

[0056] A graph neural network is used to inject prior knowledge from the knowledge graph into a deep learning model to perform part-of-speech tagging and entity recognition on the punctuated ancient Chinese text. When processing texts from different fields, such as ancient Chinese medicine books, a professional knowledge sub-graph of the Chinese medicine field is invoked, which contains professional information such as Chinese medicine terminology and acupoint names.

[0057] The semantic analysis module employs self-supervised learning and data augmentation to perform masked language modeling and contrastive learning using unlabeled corpora to alleviate the problem of insufficient labeled data. The semantic analysis module optimizes the processing in this way and transmits the results to the output module.

[0058] The output module outputs sentence segmentation results including the segmentation location and basis, and semantic analysis results including part-of-speech tagging results, entity recognition results, and a document-level semantic network visualization. The output module receives sentence segmentation results from the sentence segmentation module and semantic analysis results from the semantic analysis module and outputs them; it organizes the sentence segmentation results, clearly marking the segmentation location and basis, such as based on contextual semantics, grammatical rules, etc.; it processes the semantic analysis results to generate part-of-speech tagging results and entity recognition results, and uses visualization technology to display the document-level semantic network in a graphical way; finally, it outputs these results to the user in a clear and easy-to-understand form.

[0059] It also includes a feedback module and a connection output module. The feedback module is used to receive user feedback on the sentence segmentation results and semantic analysis results, and uses the feedback information to optimize the deep learning models in the sentence segmentation module and the semantic analysis module. The feedback module forms a data interaction with the sentence segmentation module and the semantic analysis module.

[0060] I. Training and Usage of the Gated Recurrent Unit (GRU) and Convolutional Neural Network (CNN) Fusion Model

[0061] Model training process

[0062] The training of the sentence segmentation model is divided into three stages:

[0063] Data preparation phase: Collect 100,000 annotated classical Chinese texts (covering classics, history, philosophy, and literature) and corresponding 30,000 images of stone inscriptions and bamboo slips to construct a training set (80%), a validation set (10%), and a test set (10%). The text sections are annotated with punctuation marks (e.g., , ".", "?"), and the images are annotated with the corresponding standard characters for variant characters and ambiguous character forms.

[0064] Pre-training stage: An unsupervised pre-training method is adopted to convert ancient Chinese texts into word vectors (using One-Hot encoding combined with the embedding of ancient Chinese glyph features, with a dimension of 256), and input them into the GRU-CNN fusion model. The GRU layer is set with 2 layers and a hidden layer dimension of 512 to capture sequence dependencies; the CNN layer uses 3 convolutional kernels of different sizes (3×256, 5×256, 7×256) to extract local glyph features. The pre-training objective is "masked sentence segmentation prediction", that is, randomly mask 15% of the sentence segmentation marks, and predict the sentence segmentation type at the masked position through the model. The loss function uses cross-entropy.

[0065] Fine-tuning stage: Based on the labeled data for supervised fine-tuning, the preprocessed text and image glyph features are input (the image features are extracted by ResNet50 and concatenated with the text vector), and the output layer uses the softmax function to predict the sentence segmentation probability. Adjust the learning rate (initially 0.001, decaying by 10% every 3 epochs) through the F1 value (sentence segmentation accuracy) of the validation set, and stop the iterative training when the performance of the validation set does not improve for 5 consecutive epochs. The final F1 value of sentence segmentation of the model on the test set reaches 92.3%.

[0066] Model usage

[0067] For the input unlabeled ancient Chinese text and image data, after the preprocessing module outputs the text word vector and the image feature vector, input them into the trained GRU-CNN model:

[0068] The GRU layer calculates the update gate according to the formula to determine the proportion of retaining the historical hidden state ; calculates the reset gate through to control the use of historical information; finally generates the current hidden state through to capture the long-distance dependencies of the text sequence.

[0069] The CNN layer performs a convolution operation on the text vector, extracts the local features of function words such as "zhi", "hu", "zhe", "ye", etc., and after concatenating with the sequence features output by the GRU, predicts the sentence segmentation position through the fully connected layer.

[0070] II. The association between the formula and the technical solution of this application is described as follows:

[0071] Adaptation of the GRU formula

[0072] In the processing of ancient Chinese, the sentence segmentation weights of function words such as "zhi" and "er" need to be dynamically adjusted. Therefore, the weight matrix of the update gate will tilt towards the positions of function words during training. For example, when the input is "Xue er shi xi zhi", the feature of "er" as a conjunction will make The value increases (approaching 1), strengthening the capture of the subsequent dependence on "learning constantly", which conforms to the rule of "function words determining sentence punctuation" in ancient Chinese. Customization of the attention mechanism formula For the antithetical structure of long ancient Chinese sentences (such as parallel prose), the energy function of the attention mechanism in the function is designed to integrate the calculation of word - property similarity: where is the position - distance penalty term, is an adjustable parameter (default 0.3). For example, when processing "The setting sun and the solitary wild duck fly together; the autumn waters merge with the vast sky in one color", the model will enhance the association between "the setting sun" and "the autumn waters", "the solitary wild duck" and "the vast sky" through the attention weight to assist in sentence punctuation.

[0073] III. Specific implementation methods of injecting the knowledge graph into the graph neural network (GNN):

[0074] Knowledge graph construction

[0075] The knowledge graph contains an entity layer (vocabulary, allusions, historical events, etc.) and a relationship layer (semantic evolution, allusion associations, etc.), and is stored using Neo4j. Example nodes and relationships are as follows: Entities: "Asking for the tripod" (allusion), "King Zhuang of Chu" (person), "Zuo Zhuan" (document) Relationships: "Asking for the tripod → Originates from → Zuo Zhuan" "Asking for the tripod → Involves → King Zhuang of Chu" "Asking for the tripod → Semantic evolution → Coveting political power (after the Han Dynasty)" Fusion of GNN and deep - learning models Knowledge embedding: The TransE algorithm is used to convert entities and relationships in the knowledge graph into low - dimensional vectors (dimension 128). For example, the vector of "Asking for the tripod" contains semantic features such as "allusion" and "decline of the Zhou Dynasty". Graph convolution operation: In the semantic analysis module, the text sequence vectors (from GRU output) after sentence punctuation and the knowledge - graph vectors are input into GNN, and the node features are updated through the graph - convolution formula:

[0076] where is the neighbor of node (for example, the neighbors of "Asking for the tripod" include "King Zhuang of Chu" and "Zuo Zhuan"). Output fusion: The knowledge - enhanced vectors output by GNN are concatenated with the text vectors and input into Bi - GRU for word - property tagging and entity recognition. For example, when identifying the metaphorical meaning of "Asking for the tripod" in a specific context, the knowledge vectors will strengthen the feature weight of "coveting political power".

[0077] IV. Specific operation methods of self - supervised learning and data augmentation:

[0078] Masked language modeling (MLM)

[0079] Randomly mask the unannotated corpus (such as the unpunctuated text of Siku Quanshu): The masking ratio is 15%, of which 80% is replaced with "[MASK]", 10% is replaced with random ancient Chinese words, and 10% remains unchanged. The training objective is to predict the words at the masked positions, and the loss function uses cross-entropy to enable the model to learn the context associations of words (such as in "Learning and reviewing it from time to time, isn't it a pleasure? When friends come from afar, isn't it a joy? When people don't know [MASK], isn't it a gentleman?", the model needs to predict "also"). Contrastive learning data augmentation:

[0080] Generate two positive samples for the same sentence (such as replacing borrowed characters: "exhausted and worn out" → "exhausted and fatigued"; adjusting word order: "Whom shall I go with?" → "With whom shall I go?") and one negative sample (randomly replacing the core word: "Whom shall I go with?" → "Whom shall I fight with?"). Training objective:

[0081] Through the contrastive loss:

[0082] ;

[0083] is the cosine similarity, is the temperature parameter 0.07, making the similarity of positive sample pairs higher than that of negative sample pairs, and enhancing the robustness of the model to borrowed characters and syntactic transformations.

[0084] Example 1: Automatic sentence segmentation and semantic analysis based on ordinary classical texts

[0085] Take "Learning and reviewing it from time to time, isn't it a pleasure? When friends come from afar, isn't it a joy? When people don't know, isn't it a gentleman?" in The Analects of Confucius as an example to show the specific application process of the system.

[0086] The data input module receives this unannotated ancient Chinese text (TXT format), and after passing the format check without errors, it is transmitted to the preprocessing module; the preprocessing module cleans the text to remove possible redundant spaces; uses a dictionary-based word segmentation method to initially segment the text into "Learn / and / review / it / from / time / to / time, / isn't / it / a / pleasure? / When / friends / come / from / afar, / isn't / it / a / joy? / When / people / don't / know, / isn't / it / a / gentleman?"; since there are no borrowed characters and variant characters in the text, the preprocessed text is directly transmitted to the sentence segmentation module.

[0087] The sentence segmentation module enables a deep learning model that combines GRU and CNN. GRU captures context associations according to its update gate, reset gate, etc. calculation formulas, and CNN extracts local features; the model combines text features to judge the sentence segmentation positions. At the same time, due to the absence of long-distance dependence complex situations in the text, the attention mechanism is not activated; the final sentence segmentation result is "Learning and reviewing it from time to time, isn't it a pleasure? When friends come from afar, isn't it a joy? When people don't know, isn't it a gentleman?", and the result is transmitted to the semantic analysis module.

[0088] The semantic analysis module calls the ancient Chinese knowledge graph to perform word - part tagging on the text after sentence segmentation, such as "xue (verb), er (conjunction), shi (noun)", etc.; identifies entities such as "peng (noun, referring to friends)"; constructs a discourse - level semantic network to reflect the parallel relationship between sentences; transmits the semantic analysis results to the output module, and the output module displays the sentence segmentation results, word - part tagging, entity recognition, and semantic network in a clear format; if there is no feedback from the user, the system maintains the current state; if there is feedback, the feedback module uses the information for model optimization.

[0089] Example 2: Automatic Sentence Segmentation and Semantic Analysis Based on Inscriptions Image Texts

[0090] Taking the text in a certain Tang Dynasty inscription image (JPG format) as an example, this text has some variant characters and blurred handwriting, and the content is "Wei Da Tang Kai Yuan mou nian mou yue mou ri, jun hui mou, zi mou, He Dong ren ye".

[0091] The data input module receives the inscription image data, and after format verification, transmits it to the pre - processing module; the pre - processing module performs Gaussian filtering for denoising and histogram equalization for enhancement on the image, extracts glyph features through an edge detection algorithm; at the same time, it identifies the text content in the image. For the variant of the variant character "hui", it is identified as "hui" by comparing with the standard glyph library; transmits the pre - processed text and glyph features to the sentence segmentation module. The sentence segmentation module combines the GRU and CNN models. GRU captures the text sequence association, CNN extracts local features, and at the same time introduces image glyph features to assist in judgment; for the long - time expression "Wei Da Tang Kai Yuan mou nian mou yue mou ri", the attention mechanism is enabled to enhance the capture of long - distance logical relationships, and finally the sentence is segmented into "Wei Da Tang Kai Yuan mou nian mou yue mou ri, jun hui mou, zi mou, He Dong ren ye.", and the result is transmitted to the semantic analysis module.

[0092] The semantic analysis module calls the knowledge graph containing common knowledge such as ancient official positions and geography to perform word - part tagging and entity recognition on the text after sentence segmentation, such as "Da Tang (noun, referring to the Tang Dynasty), He Dong (noun, referring to a place name)"; constructs a semantic network to reflect the subordinate relationship of each part of the information; the output module displays the relevant results, and the feedback module receives user feedback and uses it to optimize the model.

[0093] Example 3: Automatic Sentence Segmentation and Semantic Analysis Based on Traditional Chinese Medicine Ancient Book Texts

[0094] Taking the text in a certain traditional Chinese medicine ancient book (PDF format) "Shang Han wu liu ri, zhong feng, wang lai han re, xiong xie ku man, hei hei bu yu yin shi, xin fan xi ou, huo xiong zhong fan er bu ou, huo ke, huo fu zhong tong, huo xie xia pi ying" as an example.

[0095] The data input module receives this text, verifies it and transmits it to the pre - processing module; the pre - processing module cleans the text, performs word segmentation, and identifies the traditional Chinese medicine professional terms in it. There are no borrowed characters and variant characters, and transmits the processed text to the sentence segmentation module.

[0096] The GRU and CNN models in the sentence segmentation module combine text features to segment sentences. For long sentences containing multiple symptom descriptions, an attention mechanism is enabled to capture the logical relationships between the symptoms. The sentence segmentation result is "Typhoid fever for five or six days, stroke, alternating chills and fever, fullness and discomfort in the chest and hypochondrium, loss of appetite, irritability and nausea, or irritability in the chest without nausea, or thirst, or abdominal pain, or fullness and hardness under the ribs.", which is then transmitted to the semantic analysis module.

[0097] The semantic analysis module calls upon the professional knowledge sub-graph of the TCM domain to perform part-of-speech tagging on the text, such as "typhoid fever (noun, TCM disease name)" and "stroke (noun, TCM disease name)". It identifies entities such as "chest and rib (noun, body part)" and "epiglottic hardness (noun, TCM symptom)". It constructs a semantic network to reflect the parallel and associative relationships between symptoms. The output module displays the results, and the feedback module collects user feedback on the processing of professional terms to optimize the TCM domain sub-graph and model.

[0098] The foregoing has only described certain exemplary embodiments of the present invention by way of illustration. Undoubtedly, those skilled in the art can modify the described embodiments in various ways without departing from the spirit and scope of the present invention. Therefore, the foregoing drawings and descriptions are illustrative in nature and should not be construed as limiting the scope of protection of the claims of the present invention.

Claims

1. A deep learning-based automatic sentence segmentation and semantic analysis system for Classical Chinese, characterized in that, include: The data input module is used to input unlabeled ancient Chinese text and corresponding image data. The ancient Chinese text includes classical texts and inscriptions, and the image data includes image information of inscriptions and bamboo slips. The preprocessing module is connected to the data input module. The preprocessing module performs cleaning, word segmentation, and preliminary identification of variant characters and phonetic loan characters on the ancient Chinese text, and performs noise reduction, enhancement, and character shape feature extraction on the image data. The sentence segmentation module, connected to the preprocessing module, employs a deep learning model that integrates gated recurrent units (GRUs) and convolutional neural networks (CNNs). It combines the glyph features of the preprocessed image to automatically segment the preprocessed Classical Chinese text, addressing semantic ambiguity, long-distance dependencies, and sentence segmentation issues arising from variant characters and phonetic loan characters. The formula for calculating the update gate of GRU is as follows: , The formula for resetting the door is as follows: , The formula for calculating the candidate hidden state is: , The formula for calculating hidden state is: ; The semantic analysis module is connected to the sentence segmentation module. The semantic analysis module constructs a knowledge graph of classical Chinese containing historical and cultural common sense. Through graph neural network, the prior knowledge of lexical relationships and allusions in the knowledge graph is injected into the deep learning model to perform part-of-speech tagging, entity recognition and text-level semantic network construction on the classical Chinese text after sentence segmentation. The output module connects the sentence segmentation module and the semantic analysis module, and outputs the sentence segmentation results and semantic analysis results.

2. The system for automatic sentence segmentation and semantic analysis of Classical Chinese based on deep learning according to claim 1, characterized in that, The data input module can also input bamboo and wooden slips texts, unearthed document texts, and different types of unlabeled ancient Chinese texts, as well as corresponding image data of different carriers. The data input module transmits the input unlabeled ancient Chinese texts and corresponding image data to the preprocessing module.

3. The system for automatic sentence segmentation and semantic analysis of Classical Chinese based on deep learning according to claim 1, characterized in that, In the preprocessing module, the identification of variant characters adopts a method based on the ancient phonetic conversion rules, and the identification of variant characters is combined with the analysis of character structure. The preprocessing module transmits the processed ancient Chinese text and image character features to the sentence segmentation module.

4. The system for automatic sentence segmentation and semantic analysis of Classical Chinese based on deep learning according to claim 1, characterized in that, The deep learning model that integrates GRU and CNN in the sentence segmentation module is pre-trained using large-scale unlabeled classical Chinese text and corresponding image data to enhance the robustness of the sentence segmentation module to complex text. The sentence segmentation module then transmits the segmented classical Chinese text to the semantic analysis module.

5. The system for automatic sentence segmentation and semantic analysis of Classical Chinese based on deep learning according to claim 1, characterized in that, When processing classical Chinese texts with long-distance dependencies, the sentence segmentation module employs an attention mechanism to enhance its ability to capture long-distance logical relationships. The calculation formula for the attention mechanism is as follows: ， ， ， The sentence segmentation module performs sentence segmentation based on this and transmits the results to the semantic analysis module.

6. The system for automatic sentence segmentation and semantic analysis of Classical Chinese based on deep learning according to claim 1, characterized in that, The ancient Chinese text knowledge graph in the semantic analysis module also includes the historical evolution of vocabulary, semantic field associations and other dynamic relationships. The semantic analysis module processes the ancient Chinese text after segmentation based on this knowledge graph.

7. The system for automatic sentence segmentation and semantic analysis of Classical Chinese based on deep learning according to claim 1, characterized in that, When performing part-of-speech tagging and entity recognition, the semantic analysis module incorporates domain knowledge. When processing ancient Chinese texts from various fields such as traditional Chinese medicine classics and historical texts, it calls upon the professional knowledge sub-graphies of the corresponding fields. The semantic analysis module then transmits the semantic analysis results to the output module.

8. The system for automatic sentence segmentation and semantic analysis of Classical Chinese based on deep learning according to claim 1, characterized in that, The semantic analysis module employs self-supervised learning and data augmentation methods, utilizing unlabeled corpora for masked language modeling and contrastive learning to alleviate the problem of insufficient labeled data. The semantic analysis module optimizes the processing through this method and transmits the results to the output module.

9. The system for automatic sentence segmentation and semantic analysis of Classical Chinese based on deep learning according to claim 1, characterized in that, The output module outputs the sentence segmentation results, including the sentence segmentation position and the basis for the sentence segmentation. The output semantic analysis results include part-of-speech tagging results, entity recognition results, and a document-level semantic network visualization. The output module receives the sentence segmentation results from the sentence segmentation module and the semantic analysis results from the semantic analysis module and outputs them.

10. The system for automatic sentence segmentation and semantic analysis of Classical Chinese based on deep learning according to claim 1, characterized in that, It also includes a feedback module connected to the output module. The feedback module is used to receive user feedback information on the sentence segmentation results and semantic analysis results, and to use the feedback information to optimize the deep learning models in the sentence segmentation module and the semantic analysis module. The feedback module forms a data interaction with the sentence segmentation module and the semantic analysis module.

Images