Chinese text correction method based on retrieval enhancement and candidate reordering

Abstract

The application discloses a Chinese text correction method based on retrieval enhancement and candidate reordering, and relates to the technical field of text correction, and comprises the following steps: constructing a high-quality structured knowledge base; inputting a to-be-corrected original sentence into two pre-trained mask language models to obtain a first and a second correction candidate sentence respectively, and recording the positions of error characters; replacing the characters at the corresponding positions in the to-be-corrected original sentence with mask identifiers for the error character positions in the error position set to generate a retrieval sentence with masks, and obtaining external reference knowledge through database retrieval; splicing the external reference knowledge and the to-be-corrected original sentence into a prompt word, and inputting the prompt word into a large language model to obtain a third correction candidate sentence; merging the three correction candidate sentences to obtain a multi-source correction candidate set; calculating the comprehensive scores of the candidate sentences in the set; and selecting the candidate sentence with the highest score as the final correction result output. The application solves the problems of semantic drift and uncontrollable generation when a large model corrects the existing Chinese text.

Description

Technical Field

[0001] This invention relates to the field of text correction technology, specifically to a Chinese text correction method based on retrieval enhancement and candidate reordering. Background Technology

[0002] Chinese Spelling Correction (CSC), as one of the core foundational tasks in the field of Natural Language Processing (NLP), aims to automatically detect and correct typos and non-standard expressions in text sequences. In actual digital information processing, front-end perception technologies such as Optical Character Recognition (OCR) and Automatic Speech Recognition (ASR) are highly susceptible to interference from environmental noise, illegible handwriting, or accent differences, inevitably introducing numerous errors of similar-looking or similar-sounding characters into the output text. This noise, outside the normal semantics, not only disrupts the logical coherence of the context but also causes a cascading performance collapse in downstream NLP tasks such as information retrieval and machine translation. For a long time, eliminating these transcriptional errors has heavily relied on manual proofreading, facing the severe bottleneck of high labor costs and low efficiency.

[0003] With the evolution of deep learning technology, error correction paradigms based on pre-trained language models (PLMs) have become mainstream. Researchers utilize the bidirectional feature extraction capabilities of masked language models (such as BERT) to model spell correction as a sequence labeling or character substitution task, and enhance the model's sensitivity to local errors by injecting multimodal features such as pinyin and radicals. However, single lightweight pre-trained models are limited by their finite parameter size and receptive field, often exhibiting high false negative rates when faced with complex misspellings with long-tailed distributions or obscure words in vertical domains. To break through the performance ceiling of single models, some studies have explored model ensemble strategies, attempting to integrate the prediction results of multiple base models.

[0004] In recent years, generative large language models, with their extremely large internal knowledge networks and excellent zero-shot reasoning capabilities, have opened up new technical routes for text correction tasks. However, in relatively demanding text proofreading scenarios, using large language models directly as "end-to-end generators" exposes a series of inherent flaws. Because large models tend to generate extremely fluent text that conforms to popular expression habits during the pre-training stage, they are prone to over-correction when processing input containing typos. The model often arbitrarily replaces words, restructures sentences, and even semantically alters the original sentences. This excessive polishing caused by the model's "illusion," while superficially improving the fluency of the text, seriously undermines the objective fidelity of the original text, making it difficult to record accurate information.

[0005] To address the challenge of excessive generational freedom in error correction tasks, existing research attempts to constrain the generational boundaries of large models by constructing complex instruction templates (Prompt Engineering), introducing decoding period length penalties, or injecting external examples into prompts using Retrieval Augmentation (RAG) techniques. While these approaches alleviate semantic drift to some extent, they fundamentally remain rooted in the underlying logic of "relying on the subjective generation of large models." In particular, traditional whole-sentence retrieval mechanisms, when faced with query text containing typos, suffer from severe semantic pollution of the retrieved vector features due to the erroneous characters, leading to biased external knowledge recall. Furthermore, relying solely on prompts still cannot provide precise word-level control over the output of large models. Summary of the Invention

[0006] To address the aforementioned shortcomings in existing technologies, this invention provides a Chinese text correction method based on retrieval enhancement and candidate re-ranking, which solves the semantic drift and uncontrollable generation problems in large-scale Chinese text correction.

[0007] To achieve the aforementioned objectives, the technical solution adopted by this invention is as follows: a Chinese text error correction method based on retrieval enhancement and candidate reordering, comprising the following steps: S1: Input the original corpus into a large-scale language model as a teacher model, and conduct in-depth analysis of each set of parallel samples through pre-designed prompt templates to extract fine-grained error types, error correction operation sequences and error cause explanations, and build a high-quality structured knowledge base to support subsequent retrieval and reasoning tasks; S2: Input the original sentence to be corrected into at least two pre-trained masked language models to obtain the first and second candidate sentences for correction, and record the position of each erroneous character; S3: Input the original sentence to be corrected into the BERT diagnostic tool, obtain the probability distribution of the query sentence belonging to each error type, retain the Top-K high-confidence error types and their probability values ​​as weighting factors, and perform weighted fusion of each error character position with the vector database retrieval results according to the error type probability to generate an error position set; S4: For each error character position in the error position set, replace the character at the corresponding position in the original sentence to be corrected with a mask identifier, generate a search sentence with a mask, and input the search sentence into the vector database for retrieval to obtain external reference knowledge; S5: Concatenate external reference knowledge with the original sentence to be corrected into structured prompt words, and input them into the large language model to obtain the third candidate sentence for correction; S6: Merge the first, second, and third error correction candidate sentences to obtain a multi-source error correction candidate set; S7: Concatenate and encode each candidate sentence in the multi-source error correction candidate set with the original sentence to be corrected, input it into the pre-trained Cross-Encoder reordering model, calculate the cross-attention reordering score of each candidate sentence, and select the Top-K candidate sentences to form a carefully selected candidate subset; S8: For each candidate sentence in the selected candidate subset, calculate the perplexity of the candidate sentence, the vector similarity between the candidate sentence and the original sentence to be corrected, and the correct character retention rate of the candidate sentence relative to the original sentence to be corrected. S9: The perplexity, vector similarity, and correct character retention rate are weighted and summed to obtain the comprehensive score of each candidate sentence; S10: Select the candidate sentence with the highest comprehensive score as the final error correction result output, realizing Chinese text error correction based on retrieval enhancement and candidate re-ranking.

[0008] Furthermore, the weighted fusion of the error detection of each erroneous character position and the vector database retrieval results according to the error type probability specifically includes: inputting the original sentence to be corrected into the BERT diagnostic tool and outputting the error probability distribution of each character position; inputting the original sentence to be corrected into the vector database for semantic retrieval and obtaining the retrieval similarity distribution; performing weighted fusion calculation on the error probability distribution and the retrieval similarity distribution to generate a comprehensive error detection confidence score, and selecting character positions with confidence scores exceeding a preset threshold to form an error position set.

[0009] Furthermore, the step of inputting the searched sentence into a vector database for retrieval to obtain external reference knowledge specifically includes: The search sentence is input into a vector database for similarity search, and at least one reference text that semantically matches the search sentence is returned. The reference texts are then aggregated as external reference knowledge.

[0010] Furthermore, the step of concatenating external reference knowledge with the original sentence to be corrected into structured prompt words, and inputting them into a large language model to obtain a third candidate sentence for error correction, specifically includes: Construct prompt words that include external reference knowledge and the original sentence to be corrected. The prompt words instruct the large language model to replace the typos in the original sentence one-to-one, and the sentence length remains unchanged before and after the replacement. Input the prompt words into the large language model and obtain the preliminary corrected sentences output by the large language model; The initially corrected sentence is input into the discriminator for rule verification. The rule verification includes: determining whether the corrected sentence still contains erroneous characters, determining whether the corrected sentence conforms to grammatical logic, and determining whether the sentence length before and after correction is the same. If the initial corrected sentence fails the rule verification, the verification feedback and prompt words are re-input into the large language model, instructing the large language model to regenerate the corrected sentence until it passes the rule verification or reaches the preset number of retries; The sentences that pass the rule validation will be used as the third candidate sentences for error correction.

[0011] Furthermore, the step of concatenating and encoding each candidate sentence in the multi-source error correction candidate set with the original sentence to be corrected, inputting it into the pre-trained Cross-Encoder reordering model, calculating the cross-attention reordering score of each candidate sentence, and selecting the Top-K candidate sentences to form a refined candidate subset specifically includes: Each candidate sentence is concatenated with the original sentence to be corrected to form an input sequence in the format [CLS] original sentence [SEP] candidate sentence [SEP]. The input sequence is input into a pre-trained Cross-Encoder reordering model, and the semantic matching score between the candidate sentence and the original sentence is calculated through the cross-attention mechanism. This score is then used as the cross-attention reordering score. The candidate sentences are sorted in descending order based on the cross-attention reordering scores, and the top K candidate sentences are selected to form a refined candidate subset. The formula for calculating the cross-attention reordering score is as follows:

[0012]

[0013] Where Q, K, and V are the query matrix, key matrix, and value matrix obtained by linear transformation of the concatenated sequence, respectively. For the dimension of attention head, The hidden state representation for the last layer of the Cross-Encoder marked [CLS]. and Here, σ represents the weight matrix and bias of the output layer. For the first The semantic matching score of each candidate sentence.

[0014] Furthermore, the formula for calculating the degree of confusion is:

[0015] in, Sentence perplexity calculated for large models. The number of tokens in a single sentence. For conditional probability, For the first One token, For the first All tokens preceding this token.

[0016] Furthermore, the formula for calculating the correct character retention rate is as follows:

[0017] in, This represents the retention rate of correct characters between the candidate sentence and the original sentence to be corrected. This represents the number of characters in the sentence that are identical to the original sentence after excluding erroneous characters detected by the lightweight model. This represents the total number of characters in the original sentence.

[0018] Furthermore, the comprehensive score calculation formula for the candidate sentences is as follows:

[0019] in, The overall score for the candidate sentences. Candidate sentences, The original sentence is to be corrected. The vector cosine similarity between the candidate sentence and the original sentence to be corrected is given. , , These are preset weighting coefficients.

[0020] Furthermore, it also includes steps for diagnosing common-sense errors: The final error correction result is queried in a vector database of a preset vertical domain to obtain reference knowledge that semantically matches the final error correction result; The final error correction result is concatenated with the reference knowledge and then input into the large language model. The large language model is instructed to determine whether there are common-sense errors in the final error correction result that violate the reference knowledge, and outputs the diagnostic results in a non-intrusive manner. The diagnostic results include a Boolean flag indicating whether there are common-sense errors and the correspondence between incorrect words and correct words. The diagnostic results are output along with the final error correction results, without modifying the final error correction results.

[0021] A Chinese text correction system based on retrieval enhancement and candidate reordering includes: The high-quality structured knowledge base construction module is used to input the original corpus into a large-scale language model as a teacher model. Through pre-designed prompt templates, it conducts in-depth analysis on each set of parallel samples, extracts fine-grained error types, error correction operation sequences and error cause explanations, and constructs a high-quality structured knowledge base to support subsequent retrieval and reasoning tasks. The initial error detection and generation module is used to input the original sentence to be corrected into at least two pre-trained masked language models to obtain the first and second error correction candidate sentences respectively, and record the position of each error character; The hybrid error detection fusion module is used to input the original sentence to be corrected into the BERT diagnostic tool, obtain the probability distribution of the query sentence belonging to each error type, retain the top-K high-confidence error types and their probability values ​​as weighting factors, and perform weighted fusion of each error character position with the vector database retrieval results according to the error type probability to generate a set of error positions; The retrieval enhancement generation module is used to replace the corresponding character in the original sentence to be corrected with a mask identifier for each error character position in the error position set, generating a retrieval sentence with a mask. The retrieval sentence is then input into a vector database for retrieval to obtain external reference knowledge. The external reference knowledge is then concatenated with the original sentence to be corrected to form structured prompt words, which are then input into a large language model to obtain a third candidate sentence for error correction. The candidate set construction module is used to merge the first, second, and third error correction candidate sentences to obtain a multi-source error correction candidate set. The Cross-Encoder reordering module is used to concatenate and encode each candidate sentence in the multi-source error correction candidate set with the original sentence to be corrected, input it into the pre-trained Cross-Encoder reordering model, calculate the cross-attention reordering score of each candidate sentence, and select the Top-K candidate sentences to form a carefully selected candidate subset. The scoring module is used to calculate the perplexity of each candidate sentence in the selected candidate subset, the vector similarity between the candidate sentence and the original sentence to be corrected, and the correct character retention rate of the candidate sentence relative to the original sentence to be corrected; and then the perplexity, vector similarity and correct character retention rate are weighted and summed to obtain the comprehensive score of each candidate sentence. The output module is used to select the candidate sentence with the highest comprehensive score as the final error correction result.

[0022] The beneficial effects of this invention are: (1) Enhanced Retrieval Generation: This framework introduces a generative model architecture that differs from other lightweight models when constructing the candidate set. This greatly enhances the system's ability to discover complex and obscure errors, directly leading to a significant improvement in recall. Although more reference information inevitably leads to a very small number of interference misjudgments (resulting in a slight decrease in precision), the significant breakthrough in recall achieved with a very small loss in precision makes it a very successful overall F1 score.

[0023] (2) Avoid large models directly generating error correction results: The framework does not allow large models to directly generate error correction results, but instead allows them to calculate the perplexity (semantic fluency) of candidate sentences and objectively score several alternative modification schemes. Compared with direct generation, this approach greatly reduces the space for large models to arbitrarily tamper with the original meaning and over-correct, firmly safeguarding the bottom line of accuracy.

[0024] (3) The comprehensive rearrangement mechanism plays a purifying role: Before the final output, the system will combine multi-dimensional indicators (such as the degree of preservation of the original sentence meaning) for final evaluation. This is equivalent to adding a "safety lock" to the model, directly eliminating the outrageous modifications that may have been generated due to the introduction of retrieval, and ensuring the reliability of the error correction results.

[0025] In summary, R2-CSC significantly improves recall by adding a retrieval enhancement generation module and maintaining the large model score, sacrificing some accuracy, and ultimately improving the model's overall error correction performance. Attached Figure Description

[0026] Figure 1 The flowchart shows the Chinese text correction method (S1-S10) based on retrieval enhancement and candidate re-ranking. Detailed Implementation

[0027] The present invention will be further described below with reference to the accompanying drawings and specific embodiments.

[0028] This invention proposes a Chinese text correction method based on retrieval enhancement and candidate re-ranking (R2-CSC). The method consists of two stages: multi-source heterogeneous candidate construction and joint re-ranking using a large language model.

[0029] The first stage involves feeding the input sentence to be corrected to two different BERT mini-models. While generating preliminary correction results, the positions of suspected typos are recorded. Simultaneously, the corresponding erroneous character positions in the original sentence are replaced one by one with a special mask. These masked sentences are then sent to a vertical medical vector database for retrieval. All accurate external knowledge data retrieved from these searches is aggregated and added to a prompt along with the original sentence. This prompt is then used by the larger language model to reference and modify the original sentence. After generation, a discriminator judges the results; if the result is unsatisfactory, it is returned for revision. In this way, RAG and the two BERT mini-models generate a total of three high-quality correction candidates, forming a multi-source candidate set. Fine-grained mask retrieval is employed during the candidate set construction to address the issue of large models easily generating illusions due to a lack of specific local context, thereby significantly improving the recall effect of the correction method.

[0030] In the second stage, abandoning the traditional approach of directly outputting text from a large model, all three error correction candidates generated in the first stage are input into the large language model, allowing the large model to act as an objective judge to calculate their sentence consistency re-scores. Simultaneously, the system calculates the correct character retention rate of each candidate sentence compared to the original sentence. Finally, the "sentence consistency re-score" and "correct character retention rate" scores are weighted and summed, and the candidate sentence with the highest overall score is selected as the final error correction answer. This re-ranking and selection mechanism, combined with objective rule constraints, solves the problem of large language models distorting the original meaning of medical diagnoses due to excessive freedom, greatly improving the accuracy and fidelity of error correction. The specific steps are as follows: Example 1, as Figure 1 As shown, a Chinese text correction method based on retrieval enhancement and candidate re-ranking includes the following steps: S1: Input the original corpus into a large-scale language model (such as DeepSeek-V3 671B) as a teacher model. Conduct in-depth analysis on each set of parallel samples using pre-designed prompt templates (as shown in Table 1) to extract fine-grained error types, error correction operation sequences, and explanations of error causes. Sample the same parallel sample multiple times and use a majority voting strategy to determine the final annotation. Randomly sample and review and correct high-frequency error types or complex cases. Construct a high-quality structured knowledge base that can support subsequent retrieval and reasoning tasks, and store the knowledge in a vector database for subsequent retrieval.

[0031] Table 1. Knowledge Distillation Structured Analysis Hints

[0032] S2: Input the original sentence to be corrected into at least two pre-trained masked language models to obtain the first and second candidate sentences for correction, and record the position of each erroneous character.

[0033] First, the system feeds the original sentence to be corrected into two different pre-trained masked language models (SCOPE and bert-chinese-base). These two lightweight models scan and predict the original sentence in parallel, outputting their respective corrected results. In this process, the system not only collects two initial candidate sentences for correction, but more importantly, by comparing the original sentence with the output of the small models, it accurately records the index positions of suspected erroneous characters in the original sentence. This approach of pre-detection using dual small models provides the error locations for subsequent retrieval enhancement.

[0034] For a sentence that needs correction :

[0035] in, The characters in the sentence.

[0036] There are two lightweight models, and the error correction results of these two models are used as part of a candidate set, which is then used... express:

[0037] in, Error correction results generated for the SCOPE model Error correction results generated for the Chinese BERT model. The error correction results generated for the RAG module. While generating error correction results, the two lightweight models also output the positions of the erroneous characters they detected. The error positions detected by the two models are merged into a single set M:

[0038] in, For the first One error location.

[0039] S3: Input the original sentence to be corrected into the BERT diagnostic tool, obtain the probability distribution of the query sentence belonging to each error type, retain the top-K high-confidence error types and their probability values ​​as weighting factors, and perform weighted fusion of each error character position with the vector database retrieval results according to the error type probability to generate an error position set.

[0040] Traditional single-model error detection mechanisms often suffer from high false negative rates when dealing with complex misspellings or obscure words in specific domains. To address this issue, this invention innovatively introduces a hybrid error detection weighted fusion mechanism. The system simultaneously inputs the original sentence to be corrected into the BERT diagnostic tool and a vector database: the BERT diagnostic tool outputs fine-grained error probability distributions for each character position, while the vector database returns the retrieval similarity distribution. Subsequently, the system performs weighted fusion calculations on these two heterogeneous signals to generate a comprehensive error detection confidence score, selecting only character positions with confidence scores exceeding a preset threshold to form the final set of error positions. This hybrid error detection method fully utilizes the local perception capabilities of the BERT diagnostic tool and the global semantic retrieval capabilities of the vector database, significantly reducing both false negative and false positive rates.

[0041] S4: For each error character position in the error position set, replace the character at the corresponding position in the original sentence to be corrected with a mask identifier, generate a search sentence with a mask, and input the search sentence into the vector database for retrieval to obtain external reference knowledge; Traditional retrieval enhancement techniques usually directly input the entire sentence into the vector database for similarity search. However, in the spelling correction task, if a sentence containing spelling mistakes is directly used for retrieval, the incorrect semantics of the spelling mistakes themselves will interfere with the vector matching process, resulting in irrelevant texts being retrieved.

[0042] To solve this problem, the present invention proposes a fine-grained masking retrieval mechanism. For each recorded position of the incorrect character, the system replaces it with a special masking identifier [MASK], while retaining the correct context of the rest of the sentence. Then, the system sends the sentence into the vector database for query. This approach eliminates the feature pollution of the incorrect character, enabling the database to accurately match and return one or two most appropriate standard reference texts based on the pure context. Subsequently, the system aggregates the retrieved high-quality data to form an external reference knowledge base specifically for the current original sentence.

[0043] Input the retrieval sentence into the vector database for retrieval to obtain external reference knowledge, specifically including: Input the retrieval sentence into the vector database for similarity search, return at least one reference text semantically matching the retrieval sentence, and aggregate the reference texts as external reference knowledge.

[0044] Suppose the sentence to be corrected is: "Let's go to the City God Temple for a stroll. I heard there's a new ice cream shop there. Their shaved ice is especially delicious, and the environment is also very pleasant." The final detection results of the two lightweight models are , that is, the "皇" at position 4, the "水" at position 27, and the "咦" at position 39 are detected as having mistakes. Replace each mistake with [MASK] one by one, and the final sentence obtained for vector database search is: "Let's go to the City [MASK] Temple for a stroll. I heard there's a new ice cream shop there. Their shaved [MASK] is especially delicious, and the environment is also very [MASK]." S5: Concatenate the external reference knowledge with the original sentence to be corrected into a structured prompt, and input it into the large language model to obtain the third correction candidate sentence, specifically including: Construct a prompt containing the external reference knowledge and the original sentence to be corrected, and the prompt instructs the large language model to perform one-to-one replacement of the spelling mistakes in the original sentence to be corrected, and the sentence lengths before and after replacement remain the same; Input the prompt into the large language model to obtain the preliminary corrected sentence output by the large language model; Input the preliminary corrected sentence into the discriminator for rule verification, and the rule verification includes: judging whether there are still incorrect characters in the corrected sentence, judging whether the corrected sentence conforms to grammar logic, and judging whether the sentence lengths before and after correction are consistent; If the initial corrected sentence fails the rule verification, the verification feedback and prompt words are re-input into the large language model, instructing the large language model to regenerate the corrected sentence until it passes the rule verification or reaches the preset number of retries; The sentences that pass the rule validation will be used as the third candidate sentences for error correction.

[0045] The prompt words are shown in Table 2. The large model then acts as a "reference generator," modifying the original sentence by incorporating external knowledge from the prompt words. To prevent the large model from over-diverging or becoming disorienting during generation (e.g., arbitrarily adding or deleting words), this module introduces a discriminator after the generation process. The discriminator performs strict rule-based validation on the sentences output by the large model (e.g., comparing the string length before and after modification). If a sentence fails, the discriminator returns the error reason as feedback, forcing the large language model to revise it again until a valid sentence is output. After this, through the combination of RAG and the large model, the system successfully obtained the third high-quality error-correction candidate sentence. Together with the candidates generated by the previous two BERT small models, the system ultimately constructs a multi-source error-correction candidate set containing three options.

[0046] Table 2. New Candidates Build by the Search Enhancement Module

[0047] S6: Merge the first, second, and third error correction candidate sentences to obtain a multi-source error correction candidate set; S7: Concatenate and encode each candidate sentence in the multi-source error correction candidate set with the original sentence to be corrected, input it into the pre-trained Cross-Encoder reordering model, calculate the cross-attention reordering score of each candidate sentence, and select the Top-K candidate sentences to form a carefully selected candidate subset; Traditional candidate sentence selection methods rely solely on simple rule filtering or a single scoring metric, lacking the ability to model the deep semantic interactions between candidate sentences and the original sentences, making it difficult to effectively distinguish between high-quality correction results and over-correction results. To address this issue, this invention innovatively introduces a pre-trained Cross-Encoder reordering model. Specifically, the system concatenates each candidate sentence with the original sentence to be corrected into an input sequence in the format [CLS]original sentence[SEP]candidate sentence[SEP], which is then input into the Cross-Encoder model. The Cross-Encoder deeply models the semantic interaction between candidate sentences and the original sentences through a bidirectional cross-attention mechanism, accurately capturing subtle semantic differences between candidate sentence pairs and calculating a semantic matching score. The formula for calculating the cross-attention reordering score is as follows:

[0048]

[0049] Where Q, K, and V are the query matrix, key matrix, and value matrix obtained by linear transformation of the concatenated sequence, respectively. For the dimension of attention head, The hidden state representation for the last layer of the Cross-Encoder marked [CLS]. and Here, σ represents the weight matrix and bias of the output layer. Let be the semantic matching score of the i-th candidate sentence.

[0050] Candidate sentences are sorted in descending order based on their matching scores, and the top K candidate sentences are selected to form a refined candidate subset. This cross-encoder-based reordering mechanism can effectively distinguish between high-quality error correction and over-correction, significantly improving the accuracy of candidate sentence selection.

[0051] Having a candidate set, the core problem of this framework is how to select the correct sentence from it. To this end, the second stage of this method designs a joint scoring re-ranking mechanism based on a large language model. In order to solve the problems of over-correction and uncontrollable generation that large language models often encounter when performing error correction tasks, this stage no longer allows the large language model to generate sentences based on the modifications of the candidate sentences.

[0052] S8: For each candidate sentence in the selected candidate subset, calculate the perplexity of the candidate sentence, the vector similarity between the candidate sentence and the original sentence to be corrected, and the correct character retention rate of the candidate sentence relative to the original sentence to be corrected. The formula for calculating perplexity is:

[0053] in, Sentence perplexity calculated for large models. The number of tokens in a single sentence. For conditional probability, For the first One token, For the first All tokens preceding this token.

[0054] The formula for calculating the correct character retention rate is:

[0055] in, This represents the retention rate of correct characters between the candidate sentence and the original sentence to be corrected. This represents the number of characters in the sentence that are identical to the original sentence after excluding erroneous characters detected by the lightweight model. This represents the total number of characters in the original sentence.

[0056] S9: The perplexity, vector similarity, and correct character retention rate are weighted and summed to obtain the comprehensive score of each candidate sentence; The formula for calculating the overall score of candidate sentences is as follows:

[0057] in, The overall score for the candidate sentences. Candidate sentences, The original sentence is to be corrected. The vector cosine similarity between the candidate sentence and the original sentence to be corrected is given. , , Preset weighting coefficients are used, as semantic preservation and minimal modification are important. and With values ​​set to 0.4 and 0.35, sentence fluency is relatively less important, therefore The value is set to 0.25.

[0058] Suppose we have the sentence to be corrected, "I run but don't exercise," and candidate sentences, "I run but don't exercise" and "I exercise." The lightweight model detects that the character at position 2 is an error and does not participate in the calculation. Therefore, the keep value for "I run but don't exercise" is 4 correct characters / 5 total characters, resulting in a value of 0.8; the keep value for "I exercise" is 1 correct character / 5 total characters, resulting in a value of 0.2. Finally, the candidate sentence with the highest score is selected as the final corrected output of the entire framework. Through this reordering and selection mechanism that strongly couples multi-source candidates with objective scoring rules, this method greatly constrains the problem of excessive freedom in large model generation.

[0059] S10: Select the candidate sentence with the highest comprehensive score as the final error correction result output, realizing Chinese text error correction based on retrieval enhancement and candidate re-ranking.

[0060] To make the framework more adaptable to vertical industries, the present invention also includes common-sense error diagnosis steps: The final error correction result is queried in a vector database of a preset vertical domain to obtain reference knowledge that semantically matches the final error correction result; The final error correction result is concatenated with the reference knowledge and then input into the large language model. The large language model is instructed to determine whether there are common-sense errors in the final error correction result that violate the reference knowledge, and outputs the diagnostic results in a non-intrusive manner. The diagnostic results include a Boolean flag indicating whether there are common-sense errors and the correspondence between incorrect words and correct words. The diagnostic results are output along with the final error correction results, without modifying the final error correction results.

[0061] The common-sense error diagnosis module prompts are shown in Table 3: Assuming the corrected sentence is "City A is the capital of Country A", the final corrected result will be: { "result": "City A is the capital of Country A". “commonsence_reasoning”:{ "is_mis":true, "diagnose": "Check for common-sense errors: City A -> City B" } } This non-intrusive approach ensures that the final correction remains true to the original meaning while also helping to correct common-sense errors in the sentence.

[0062] Table 3 Common Sense Error Detection Tips

[0063] Furthermore, this invention also includes an offline structured knowledge base construction step: inputting the original corpus into a large language model, extracting the error type, error correction operation, and cause explanation for each corpus, and constructing a structured knowledge base; constructing triples (a, p, n) based on type labels and syntactic tree distance, where a is the anchor sample, p is the positive sample, and n is the negative sample; training the encoder Esyn through contrastive learning, enabling the encoder to learn to distinguish the semantic representation of correct and incorrect text; and using the trained encoder to generate semantic vectors for the vector database.

[0064] In one embodiment of the present invention, in order to fully verify the error correction performance of the R2-CSC framework proposed in this invention under different difficulties and distributions, the experiment used a recognized benchmark dataset in the field of Chinese spelling error correction for evaluation.

[0065] The SIGHAN series datasets are the most authoritative benchmark evaluation sets in the field of Chinese spelling correction. Released by the SIGHAN workshops in 2013, 2014, and 2015, their corpora primarily consist of real error samples generated by non-native Chinese learners during writing exercises. The series is characterized by its rigorous annotation standards, including not only typical Chinese spelling errors such as homophones, near-homophones, and similar-looking characters, but also evolving gradually over time, from initially focusing on high-density word errors (SIGHAN 13) to later text distributions that more closely resemble natural language logic and possess more complex semantic backgrounds (SIGHAN 15). By providing standardized training and test sets, it offers a unified metric for evaluating the performance of error correction models in both error detection and correction dimensions, and is currently the core standard for measuring the generalization ability and robustness of Chinese error correction models.

[0066] The Wang271k dataset is a large-scale, automatically synthesized corpus built by Wang et al. for Chinese spelling correction tasks. Its core value lies in addressing the scarcity of labeled data in the field of Chinese spelling correction. Using general Chinese news and web page texts as a baseline, this dataset simulates the real human typing process by utilizing a Chinese character confusion set. Homophones, near-homophones, and similar-looking spelling errors are automatically injected into the original text at a certain proportion, ultimately generating a parallel corpus containing approximately 271,000 pairs of sentences to be corrected and correct sentences. Due to its extremely broad coverage of language expression scenarios and rich error patterns, Wang271k has become the preferred dataset for large-scale pre-training or supervised fine-tuning of mainstream Chinese spelling correction models, significantly improving the model's ability to capture underlying language rules and its recall rate for common spelling errors.

[0067] The training data for these two datasets are shown in Table 4: Table 4 Training Set

[0068] The test set data is shown in Table 5: Table 5 Test Set

[0069] The evaluation metrics used are precision, recall, and F1 score, which are commonly used in Chinese error correction tasks. The calculation formulas for the three metrics are as follows:

[0070]

[0071]

[0072] Here, TP represents the number of words that were originally misspelled, but the model successfully detected the error and correctly corrected them to the target character. FP represents the number of words that were originally correct, but the model misjudged or overcorrected them, incorrectly changing them to other characters. FN represents the number of words that were originally misspelled, but the model failed to detect them (false negatives), or detected them but failed to correct them correctly.

[0073] To ensure the rigor and comprehensiveness of the comparative experiments, this embodiment selected three mainstream baseline models that are representative of the Chinese spell checking field and cover different technological evolution directions during the multi-source candidate construction and performance comparison stages. Their specific architecture and design ideas are as follows: REALISE: This architecture approaches the problem from the perspective of multimodal feature alignment, and its core consists of independent semantic, phonetic, and glyph encoders operating in parallel. To avoid unnecessary modal information interfering with error correction, the model employs a heuristic gating mechanism that adaptively evaluates and filters noise during prediction, enabling dynamic selection and fusion of the three modal features.

[0074] DCN: This framework aims to solve the challenge of candidate combination search in scenarios with multiple misspellings. By introducing a dynamic connection network at the decoding end, DCN can generate multiple correction candidates in parallel for each error position in a sentence based on a global attention mechanism. Finally, it selects the globally optimal transition path using an end-to-end scoring function, which significantly enhances the coherence of the corrected syntactic structure.

[0075] MDCSpell: As a performance benchmark in pure deep learning architecture, this model proposes an efficient multi-task joint learning paradigm. The system not only utilizes Transformer to extract deep multimodal features, but more importantly, it adopts a "post-fusion strategy" in the prediction layer, directly aligning the hidden layer states of the error detection network to the error correction network. This significantly reduces the cascading propagation of detection errors while achieving excellent accuracy.

[0076] This embodiment conducted experiments on three datasets: SIGHAN13, SIGHAN14, and SIGHAN15. The experimental results are shown in Table 6.

[0077] Table 6 Experimental Results

[0078] As can be seen from the table above, simply letting a large model like Qwen1.5-7B directly generate error-corrected sentences yields very poor results. In contrast, the R2-CSC model of this invention achieves F1 scores on the SIGHAN 13, 14, and 15 datasets that are 23.3, 30.3, and 32.5 percentage points higher than the directly generated Qwen, respectively. This indicates that large models are extremely prone to deviating from the original meaning when generating sentences freely, and strict framework constraints must be imposed to ensure the accuracy of the corrections.

[0079] Furthermore, R2-CSC achieved recall rates of 85.2%, 74.0%, and 85.5% on the three test sets, comprehensively outperforming other models. This means that the framework can identify many deeply hidden typos that traditional models easily miss. Overall, although the introduction of the new mechanism resulted in a slightly lower precision (P) for R2-CSC compared to some models in certain situations, its superior recall ultimately led to R2-CSC surpassing other models in F1 scores, representing overall error correction capability, reaching 86.6%, 71.7%, and 83.0%, exceeding baselines such as MDCSpell and REALISE.

[0080] Based on experimental observations, the superior performance of R2-CSC is mainly attributed to the following core design features: (1) Enhanced Retrieval Generation: This framework introduces a generative model architecture that differs from other lightweight models when constructing the candidate set. This greatly enhances the system's ability to discover complex and obscure errors, directly leading to a significant improvement in recall. Although more reference information inevitably leads to a very small number of interference misjudgments (resulting in a slight decrease in precision), the significant breakthrough in recall achieved with a very small loss in precision makes it a very successful overall F1 score.

[0081] (2) Avoid large models directly generating error correction results: The framework does not allow large models to directly generate error correction results, but instead allows them to calculate the perplexity (semantic fluency) of candidate sentences and objectively score several alternative modification schemes. Compared with direct generation, this approach greatly reduces the space for large models to arbitrarily tamper with the original meaning and over-correct, firmly safeguarding the bottom line of accuracy.

[0082] (3) The comprehensive rearrangement mechanism plays a purifying role: Before the final output, the system will combine multi-dimensional indicators (such as the degree of preservation of the original sentence meaning) for final evaluation. This is equivalent to adding a "safety lock" to the model, directly eliminating the outrageous modifications that may have been generated due to the introduction of retrieval, and ensuring the reliability of the error correction results.

[0083] In summary, R2-CSC significantly improved recall by adding a retrieval enhancement generation module and maintaining the large model score, sacrificing some accuracy, and ultimately improving the overall error correction performance of the model, proving that this framework is indeed useful.

[0084] Example 2: A Chinese text correction system based on retrieval enhancement and candidate reordering, comprising: The high-quality structured knowledge base construction module is used to input the original corpus into a large-scale language model as a teacher model. Through pre-designed prompt templates, it conducts in-depth analysis on each set of parallel samples, extracts fine-grained error types, error correction operation sequences and error cause explanations, and constructs a high-quality structured knowledge base to support subsequent retrieval and reasoning tasks. The initial error detection and generation module is used to input the original sentence to be corrected into at least two pre-trained masked language models to obtain the first and second error correction candidate sentences respectively, and record the position of each error character; The hybrid error detection fusion module is used to input the original sentence to be corrected into the BERT diagnostic tool, obtain the probability distribution of the query sentence belonging to each error type, retain the top-K high-confidence error types and their probability values ​​as weighting factors, and perform weighted fusion of each error character position with the vector database retrieval results according to the error type probability to generate a set of error positions; The retrieval enhancement generation module is used to replace the corresponding character in the original sentence to be corrected with a mask identifier for each error character position in the error position set, generating a retrieval sentence with a mask. The retrieval sentence is then input into a vector database for retrieval to obtain external reference knowledge. The external reference knowledge is then concatenated with the original sentence to be corrected to form structured prompt words, which are then input into a large language model to obtain a third candidate sentence for error correction. The candidate set construction module is used to merge the first, second, and third error correction candidate sentences to obtain a multi-source error correction candidate set. The Cross-Encoder reordering module is used to concatenate and encode each candidate sentence in the multi-source error correction candidate set with the original sentence to be corrected, input it into the pre-trained Cross-Encoder reordering model, calculate the cross-attention reordering score of each candidate sentence, and select the Top-K candidate sentences to form a carefully selected candidate subset. The scoring module is used to calculate the perplexity of each candidate sentence in the selected candidate subset, the vector similarity between the candidate sentence and the original sentence to be corrected, and the correct character retention rate of the candidate sentence relative to the original sentence to be corrected; and then the perplexity, vector similarity and correct character retention rate are weighted and summed to obtain the comprehensive score of each candidate sentence. The output module is used to select the candidate sentence with the highest comprehensive score as the final error correction result.

[0085] Those skilled in the art will recognize that the embodiments described herein are intended to help the reader understand the principles of the invention, and should be understood that the scope of protection of the invention is not limited to such specific statements and embodiments. Those skilled in the art can make various other specific modifications and combinations based on the technical teachings disclosed in this invention without departing from the spirit of the invention, and these modifications and combinations are still within the scope of protection of the invention.

Claims

1. A Chinese text error correction method based on retrieval enhancement and candidate re-ranking, characterized in that, Includes the following steps: S1: Input the original corpus into a large-scale language model as a teacher model, and conduct in-depth analysis of each set of parallel samples through pre-designed prompt templates to extract fine-grained error types, error correction operation sequences and error cause explanations, and build a high-quality structured knowledge base to support subsequent retrieval and reasoning tasks; S2: Input the original sentence to be corrected into at least two pre-trained masked language models to obtain the first and second candidate sentences for correction, and record the position of each erroneous character; S3: Input the original sentence to be corrected into the BERT diagnostic tool, obtain the probability distribution of the query sentence belonging to each error type, retain the Top-K high-confidence error types and their probability values ​​as weighting factors, and perform weighted fusion of each error character position with the vector database retrieval results according to the error type probability to generate an error position set; S4: For each error character position in the error position set, replace the character at the corresponding position in the original sentence to be corrected with a mask identifier, generate a search sentence with a mask, and input the search sentence into the vector database for retrieval to obtain external reference knowledge; S5: Concatenate external reference knowledge with the original sentence to be corrected into structured prompt words, and input them into the large language model to obtain the third candidate sentence for correction; S6: Merge the first, second, and third error correction candidate sentences to obtain a multi-source error correction candidate set; S7: Concatenate and encode each candidate sentence in the multi-source error correction candidate set with the original sentence to be corrected, input it into the pre-trained Cross-Encoder reordering model, calculate the cross-attention reordering score of each candidate sentence, and select the Top-K candidate sentences to form a carefully selected candidate subset; S8: For each candidate sentence in the selected candidate subset, calculate the perplexity of the candidate sentence, the vector similarity between the candidate sentence and the original sentence to be corrected, and the correct character retention rate of the candidate sentence relative to the original sentence to be corrected. S9: The perplexity, vector similarity, and correct character retention rate are weighted and summed to obtain the comprehensive score of each candidate sentence; S10: Select the candidate sentence with the highest comprehensive score as the final error correction result output, realizing Chinese text error correction based on retrieval enhancement and candidate re-ranking.

2. The Chinese text error correction method based on retrieval enhancement and candidate re-ranking according to claim 1, characterized in that, The weighted fusion of the error detection method, which mixes the error character positions with the vector database retrieval results according to the error type probability, specifically includes: inputting the original sentence to be corrected into the BERT diagnostic tool and outputting the error probability distribution of each character position; inputting the original sentence to be corrected into the vector database for semantic retrieval and obtaining the retrieval similarity distribution; performing a weighted fusion calculation on the error probability distribution and the retrieval similarity distribution to generate a comprehensive error detection confidence score, and selecting character positions with confidence scores exceeding a preset threshold to form an error position set.

3. The Chinese text error correction method based on retrieval enhancement and candidate re-ranking according to claim 1, characterized in that, The step of inputting the search sentence into a vector database for retrieval to obtain external reference knowledge specifically includes: The search sentence is input into a vector database for similarity search, and at least one reference text that semantically matches the search sentence is returned. The reference texts are then aggregated as external reference knowledge.

4. The Chinese text error correction method based on retrieval enhancement and candidate re-ranking according to claim 1, characterized in that, The process of concatenating external reference knowledge with the original sentence to be corrected into structured prompt words, and inputting them into a large language model to obtain the third candidate sentence for correction, specifically includes: Construct prompt words that include external reference knowledge and the original sentence to be corrected. The prompt words instruct the large language model to replace the typos in the original sentence one-to-one, and the sentence length remains unchanged before and after the replacement. Input the prompt words into the large language model and obtain the preliminary corrected sentences output by the large language model; The initially corrected sentence is input into the discriminator for rule verification. The rule verification includes: determining whether the corrected sentence still contains erroneous characters, determining whether the corrected sentence conforms to grammatical logic, and determining whether the sentence length before and after correction is the same. If the initial corrected sentence fails the rule verification, the verification feedback and prompt words are re-input into the large language model, instructing the large language model to regenerate the corrected sentence until it passes the rule verification or reaches the preset number of retries; The sentences that pass the rule validation will be used as the third candidate sentences for error correction.

5. The Chinese text error correction method based on retrieval enhancement and candidate re-ranking according to claim 1, characterized in that, The process of concatenating and encoding each candidate sentence in the multi-source error correction candidate set with the original sentence to be corrected, inputting it into a pre-trained Cross-Encoder reordering model, calculating the cross-attention reordering score of each candidate sentence, and selecting the Top-K candidate sentences to form a refined candidate subset specifically includes: Each candidate sentence is concatenated with the original sentence to be corrected to form an input sequence in the format [CLS] original sentence [SEP] candidate sentence [SEP]. The input sequence is input into a pre-trained Cross-Encoder reordering model, and the semantic matching score between the candidate sentence and the original sentence is calculated through the cross-attention mechanism. This score is then used as the cross-attention reordering score. The candidate sentences are sorted in descending order based on the cross-attention reordering scores, and the top K candidate sentences are selected to form a refined candidate subset. The formula for calculating the cross-attention reordering score is as follows: Where Q, K, and V are the query matrix, key matrix, and value matrix obtained by linear transformation of the concatenated sequence, respectively. For the dimension of attention head, The hidden state representation for the last layer of the Cross-Encoder marked [CLS]. and Here, σ represents the weight matrix and bias of the output layer. For the first The semantic matching score of each candidate sentence.

6. The Chinese text error correction method based on retrieval enhancement and candidate re-ranking according to claim 1, characterized in that, The formula for calculating the degree of confusion is: in, Sentence perplexity calculated for large models. The number of tokens in a single sentence. For conditional probability, For the first One token, For the first All tokens preceding this token.

7. The Chinese text error correction method based on retrieval enhancement and candidate re-ranking according to claim 6, characterized in that, The formula for calculating the correct character retention rate is as follows: in, This represents the retention rate of correct characters between the candidate sentence and the original sentence to be corrected. This represents the number of characters in the sentence that are identical to the original sentence after excluding erroneous characters detected by the lightweight model. This represents the total number of characters in the original sentence.

8. The Chinese text error correction method based on retrieval enhancement and candidate re-ranking according to claim 7, characterized in that, The formula for calculating the comprehensive score of the candidate sentences is as follows: in, The overall score for the candidate sentences. As candidate sentences, The original sentence is to be corrected. The vector cosine similarity between the candidate sentence and the original sentence to be corrected is given. , , These are preset weighting coefficients.

9. The Chinese text error correction method based on retrieval enhancement and candidate re-ranking according to claim 1, characterized in that, It also includes common-sense error diagnosis steps: The final error correction result is queried in a vector database of a preset vertical domain to obtain reference knowledge that semantically matches the final error correction result; The final error correction result is concatenated with the reference knowledge and then input into the large language model. The large language model is instructed to determine whether there are common-sense errors in the final error correction result that violate the reference knowledge, and outputs the diagnostic results in a non-intrusive manner. The diagnostic results include a Boolean flag indicating whether there are common-sense errors and the correspondence between incorrect words and correct words. The diagnostic results are output along with the final error correction results, without modifying the final error correction results.

10. A system for Chinese text correction based on retrieval enhancement and candidate reordering as described in any one of claims 1-9, characterized in that, include: The high-quality structured knowledge base construction module is used to input the original corpus into a large-scale language model as a teacher model. Through pre-designed prompt templates, it conducts in-depth analysis on each set of parallel samples, extracts fine-grained error types, error correction operation sequences and error cause explanations, and constructs a high-quality structured knowledge base to support subsequent retrieval and reasoning tasks. The initial error detection and generation module is used to input the original sentence to be corrected into at least two pre-trained masked language models to obtain the first and second error correction candidate sentences respectively, and record the position of each error character; The hybrid error detection fusion module is used to input the original sentence to be corrected into the BERT diagnostic tool, obtain the probability distribution of the query sentence belonging to each error type, retain the top-K high-confidence error types and their probability values ​​as weighting factors, and perform weighted fusion of each error character position with the vector database retrieval results according to the error type probability to generate a set of error positions; The retrieval enhancement generation module is used to replace the corresponding character in the original sentence to be corrected with a mask identifier for each error character position in the error position set, generating a retrieval sentence with a mask. The retrieval sentence is then input into a vector database for retrieval to obtain external reference knowledge. The external reference knowledge is then concatenated with the original sentence to be corrected to form structured prompt words, which are then input into a large language model to obtain a third candidate sentence for error correction. The candidate set construction module is used to merge the first, second, and third error correction candidate sentences to obtain a multi-source error correction candidate set. The Cross-Encoder reordering module is used to concatenate and encode each candidate sentence in the multi-source error correction candidate set with the original sentence to be corrected, input it into the pre-trained Cross-Encoder reordering model, calculate the cross-attention reordering score of each candidate sentence, and select the Top-K candidate sentences to form a carefully selected candidate subset. The scoring module is used to calculate the perplexity of each candidate sentence in the selected candidate subset, the vector similarity between the candidate sentence and the original sentence to be corrected, and the correct character retention rate of the candidate sentence relative to the original sentence to be corrected; and then the perplexity, vector similarity and correct character retention rate are weighted and summed to obtain the comprehensive score of each candidate sentence. The output module is used to select the candidate sentence with the highest comprehensive score as the final error correction result.

Images