Chinese text correction methods, devices, and computer equipment based on spell checking
By extracting feature information from domain-specific documents and constructing case templates, and combining them with a pre-trained language model for text correction, the problem of existing spell checking models being difficult to adapt to specific domains is solved, achieving efficient and accurate text correction results.
Patent Information
- Application Number
- CN202411643354.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-11-18
- Publication Date
- 2025-10-28
- Estimated Expiration
- 2044-11-18
AI Technical Summary
Existing spell-checking models struggle to demonstrate substantial correction effects in specific domains, primarily due to a lack of labeled data specific to those domains, resulting in insufficient supervised training. Furthermore, developing models for multiple domains is time-consuming and labor-intensive.
By extracting domain-specific feature information from document texts in a specific domain, constructing domain-specific retrieval text and case templates, and combining them with a pre-trained large language model for text correction, the model can be adapted directly using unlabeled data, thus avoiding the cost of supervised training.
It improves the adaptability of the spell check model to specific domains, enhances the accuracy and efficiency of text correction, and reduces the cost of model fine-tuning.
Smart Images

Figure CN119886116B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of Chinese text correction technology, and in particular to a Chinese text correction method, apparatus, computer device, and storage medium based on spell checking. Background Technology
[0002] Spell checking technology aims to automatically identify and correct erroneous characters in Chinese text using intelligent methods. It is a fundamental task in natural language processing and is widely used in downstream tasks such as speech recognition, summarization, and machine translation. Therefore, spell correction technology has become an indispensable and important technical tool in the field of computer technology.
[0003] The development of spell checking technology is closely related to the performance of language models. Early spell checking methods mainly relied on linguistic knowledge to manually design rules. Subsequently, with the development of machine learning, spell checking began to use probabilistic language models to detect error locations and utilize confusion sets and character similarity to correct potential spelling errors and identify candidate correct characters. Finally, the optimal correction scheme was determined by scoring the replacement sentences. In recent years, with the rapid development of deep learning, benefiting from the powerful understanding capabilities of pre-trained language models, practitioners and researchers have adopted pre-trained language models as a foundation for further developing spell correction models. Research has found that the most important cause of spelling errors is the similarity of speech and morphology. Therefore, a series of studies have incorporated multimodal information such as speech, morphology, and context into spell correction models.
[0004] In real-world applications, users often operate within specific industries or domains, leading to domain-specific error types. However, most existing methods require supervised training on labeled datasets to develop Chinese spelling correction models on large, general, manually labeled datasets. Unfortunately, these models often fail to demonstrate substantial correction effectiveness in specific domains. Therefore, enabling error correction models to correct errors across different domains has become a pressing challenge. Specifically, this problem manifests in two ways: first, the scarcity of labeled data for specific domains limits the effectiveness of existing supervised training methods, thus restricting their performance; second, if multiple domains require error correction, developing separate supervised error correction models for each domain is undoubtedly time-consuming and labor-intensive. Summary of the Invention
[0005] Based on this, the purpose of this invention is to provide a Chinese text correction method, apparatus, computer device, and storage medium based on spell checking. This invention applies retrieval enhancement technology to spell checking, allowing the pre-trained model to directly access unlabeled data in a specific domain without supervised training, avoiding the costs of fine-tuning, and improving the model's adaptability in spell checking applications, thereby achieving the goal of improving text correction. The technical solution is as follows:
[0006] In a first aspect, embodiments of this application provide a Chinese text correction method based on spell checking, comprising the following steps:
[0007] Obtain the Chinese text to be corrected and the document text of the specific domain of the Chinese text to be corrected, wherein the document text includes several document block texts;
[0008] Feature extraction is performed on the Chinese text to be corrected and several document block texts to obtain feature vectors of the Chinese text to be corrected and several document block texts;
[0009] Based on the feature vectors of the Chinese text to be corrected and several document block texts, several target document block texts are extracted from the document block texts to construct the domain-specific retrieval text of the Chinese text to be corrected.
[0010] Construct a case pool for a specific domain of the Chinese text to be corrected, wherein the case pool includes several sets of specific text samples;
[0011] Based on the Chinese text to be corrected, several sets of target-specific text samples are extracted from the case pool of the specific domain to construct a specific domain case template for the Chinese text to be corrected.
[0012] The Chinese text to be corrected, along with its specific domain retrieval text and specific domain case template, are combined to construct a sequence of instructions for the Chinese text to be corrected. This sequence of instructions is then input into a pre-trained large language model for text correction to obtain the corrected result of the Chinese text.
[0013] Secondly, embodiments of this application provide a Chinese text correction device based on spell checking, comprising:
[0014] The text acquisition module is used to acquire the Chinese text to be corrected and the document text of the Chinese text to be corrected in a specific domain, wherein the document text includes several document block texts;
[0015] The feature extraction module is used to extract features from the Chinese text to be corrected and several document block texts to obtain feature vectors of the Chinese text to be corrected and several document block texts.
[0016] The text retrieval module is used to extract several target document block texts from the several document block texts based on the feature vectors of the Chinese text to be corrected and several document block texts, and to construct the specific domain retrieval text of the Chinese text to be corrected;
[0017] The case pool construction module is used to construct a case pool for a specific domain of the Chinese text to be corrected, wherein the case pool includes several sets of specific text samples;
[0018] The case template construction module is used to extract several sets of target-specific text samples from the case pool of the specific domain based on the Chinese text to be corrected, and construct a specific domain case template for the Chinese text to be corrected.
[0019] The text correction module is used to combine the Chinese text to be corrected, the domain-specific search text of the Chinese text to be corrected, and the domain-specific case template to construct a sequence of instructions for the Chinese text to be corrected; and input the sequence of instructions for the Chinese text to be corrected into a pre-trained large language model for text correction to obtain the correction result of the Chinese text to be corrected.
[0020] Thirdly, embodiments of this application provide a computer device, including: a processor, a memory, and a computer program stored in the memory and executable on the processor; when the computer program is executed by the processor, it implements the steps of the Chinese text correction method based on spell checking as described in the first aspect.
[0021] Fourthly, embodiments of this application provide a storage medium storing a computer program that, when executed by a processor, implements the steps of the Chinese text correction method based on spell checking as described in the first aspect.
[0022] In this embodiment, domain feature information is extracted from document text in a specific domain to construct domain-specific retrieval text, and relevant text cases are extracted from a case pool in a specific domain to construct a domain-specific case template. The Chinese text to be corrected, the domain-specific retrieval text, and the domain-specific case template are then used as input data and passed to a pre-trained large language model. By introducing external domain knowledge and relevant cases as demonstrations, the language model is guided to perform accurate text correction, preventing the language model from generating too much information unrelated to the text correction result, and improving the accuracy of the language model in recognizing and correcting errors in characters in a specific domain.
[0023] To better understand and implement this invention, the following detailed description is provided in conjunction with the accompanying drawings. Attached Figure Description
[0024] Figure 1A flowchart illustrating a Chinese text correction method based on spell checking, provided as an embodiment of this application;
[0025] Figure 2 This is a flowchart illustrating step S2 of a Chinese text correction method based on spell checking, provided in one embodiment of this application.
[0026] Figure 3 This is a flowchart illustrating step S3 of a Chinese text correction method based on spell checking, provided in one embodiment of this application.
[0027] Figure 4 This is a flowchart illustrating step S4 of a Chinese text correction method based on spell checking, provided in one embodiment of this application.
[0028] Figure 5 This is a flowchart illustrating step S41 of a Chinese text correction method based on spell checking, provided in one embodiment of this application.
[0029] Figure 6 This is a flowchart illustrating step S5 of a Chinese text correction method based on spell checking, provided in one embodiment of this application.
[0030] Figure 7 This is a flowchart illustrating step S55 of a Chinese text correction method based on spell checking provided in one embodiment of this application.
[0031] Figure 8 A schematic diagram of the structure of a Chinese text correction device based on spell checking provided in one embodiment of this application;
[0032] Figure 9 This is a schematic diagram of the structure of a computer device provided in one embodiment of this application. Detailed Implementation
[0033] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.
[0034] The terminology used in this application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. The singular forms “a,” “the,” and “the” used in this application and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used herein refers to and includes any and all possible combinations of one or more of the associated listed items.
[0035] It should be understood that although the terms first, second, third, etc., may be used in this application to describe various information, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, without departing from the scope of this application, first information may also be referred to as second information, and similarly, second information may also be referred to as first information. Depending on the context, the word "if" as used herein may be interpreted as "when," "when," or "in response to determination."
[0036] Please see Figure 1 , Figure 1 The flowchart illustrates a Chinese text correction method based on spell checking, provided in one embodiment of this application. The method includes the following steps:
[0037] S1: Obtain the Chinese text to be corrected and the document text of the specific domain of the Chinese text to be corrected.
[0038] The execution entity of the Chinese text correction method based on spell checking is a text correction device (hereinafter referred to as the correction device). In an optional embodiment, the correction device may be a computer device, which may be a server or a server cluster composed of multiple computer devices, used to establish a network connection with the client.
[0039] In this embodiment, the correction device can obtain the Chinese text to be corrected and a longer document in a specific field of the Chinese text to be corrected by connecting to a database.
[0040] To facilitate the retrieval of more relevant document text information, the correction device divides lengthy documents in a specific field into blocks according to specific rules, such as word count limits and punctuation marks, to obtain the document text of the specific field of the Chinese text to be corrected. The document text includes several document block texts, as detailed below:
[0041] The document W = [w1, w2, ..., w] in the specific domain is quite long. N Let N be the text length of the document, representing the total length of the characters contained. The correction device sets the document length of each document block text to be between [N / 2, 2N]. The correction device first divides the document into sentences of different lengths based on punctuation. If the text length of a sentence is between [N / 2, 2N], it is divided into a single document block text. If the text length of a sentence is less than N / 2, it is merged with subsequent sentences until the text length of the merged sentence is between [N / 2, 2N]. If the text length of a sentence is greater than 2N, it is split into two segments. The final document text Set is constructed from several document block texts. docs={D1,D2,…,D s}, where Set docs For document text, D s Let s be the text of the s-th document block.
[0042] The correction device encodes the dictionary data to obtain its pinyin and glyph representations. Specifically, the correction device uses a pinyin information extraction tool to obtain the pinyin information of several characters in the dictionary data, thus obtaining the pinyin representation of the dictionary data. This pinyin information extraction tool can use a Python library named "pinyin". The correction device also uses a glyph information extraction tool to decompose the glyph information of several characters in the dictionary data, thus obtaining the pinyin representation of the dictionary data. This glyph information extraction tool can use a Python library named "python_chaizi".
[0043] S2: Extract features from the Chinese text to be corrected and several document block texts to obtain feature vectors of the Chinese text to be corrected and several document block texts.
[0044] In this embodiment, the correction device performs feature extraction on the Chinese text to be corrected and several document block texts to obtain feature vectors of the Chinese text to be corrected and several document block texts, wherein the feature vectors include semantic feature vectors, pinyin feature vectors and visual morphological feature vectors.
[0045] Please see Figure 2 , Figure 2 The flowchart of S2 in the Chinese text correction method based on spell checking provided in one embodiment of this application includes steps S21 to S23, as follows:
[0046] S21: Extract semantic information from the Chinese text to be corrected and several document block texts to obtain a semantic information sequence of the Chinese text to be corrected and several document block texts; perform feature vector transformation on the semantic information sequence to obtain a semantic feature vector of the Chinese text to be corrected and several document block texts.
[0047] In this embodiment, the correction device extracts semantic information from the Chinese text to be corrected and several document block texts to obtain a semantic information sequence of the Chinese text to be corrected and several document block texts.
[0048] Specifically, the correction device can employ the BERT (Bidirectional Encoder Representation from Transformers) language model. The Chinese text to be corrected and several document block texts are input into the BERT language model, and the text is converted into a vector sequence E = [e1, e2, e3, ..., e] through an embedding layer. N ], where e i The vectorized representation of the i-th character is used to convert the sequence of vectors into a semantic information sequence, thereby obtaining the semantic information sequence of the Chinese text to be corrected and several document block texts, wherein the semantic information sequence is:
[0049] H = [h1, h2, h3, ..., h N ] = PLMEncoder(E)
[0050] Where H is the semantic information sequence, h N The PLMEncoder is a semantic encoding component in a language model that encodes input text into a hidden representation, representing the semantic information vector of the Nth character.
[0051] The correction device performs feature vector transformation on the semantic information sequence to obtain semantic feature vectors of the Chinese text to be corrected and several document block texts. Specifically, the correction device uses the mean method, linear transformation, or max pooling method to perform feature vector transformation on the obtained semantic information sequence representation to obtain semantic feature vectors of the Chinese text to be corrected and several document block texts, wherein the semantic feature vectors are:
[0052] h=tran(H)=tran([h1,h2,h3,…,h N ])
[0053] Where h is the semantic feature vector, and tran(*) represents the feature vector transformation function.
[0054] S22: Using a preset pinyin conversion tool, extract pinyin information from the Chinese text to be corrected and several document block texts to obtain the pinyin information sequence of the Chinese text to be corrected and several document block texts. Then, perform feature vector conversion on the pinyin information sequence to obtain the pinyin feature vector of the Chinese text to be corrected and several document block texts.
[0055] In this embodiment, the correction device uses a preset pinyin conversion tool to extract pinyin information from the Chinese text to be corrected and several document block texts, obtains the pinyin information sequence of the Chinese text to be corrected and several document block texts, and performs feature vector conversion on the pinyin information sequence to obtain the pinyin feature vector of the Chinese text to be corrected and several document block texts.
[0056] Specifically, the correction device uses a preset pinyin conversion tool to extract the pinyin information, such as initials, finals, and tones, of several characters in the Chinese text to be corrected and several document blocks, obtaining a pinyin information sequence of the Chinese text to be corrected and several document blocks. The correction device uses an LSTM (Long Short-Term Memory) model to perform feature vector transformation on the pinyin information sequence, obtaining the pinyin feature vectors of the Chinese text to be corrected and several document blocks.
[0057] S23: Using an image feature encoding method, visual morphological information is extracted from the Chinese text to be corrected and several document block texts to obtain a visual morphological information sequence of the Chinese text to be corrected and several document block texts. The visual morphological information sequence is then transformed into a feature vector to obtain a visual morphological feature vector of the Chinese text to be corrected and several document block texts.
[0058] In this embodiment, the correction device uses an image feature encoding method to extract visual morphological information from the Chinese text to be corrected and several document block texts, obtains a visual morphological information sequence of the Chinese text to be corrected and several document block texts, and performs feature vector transformation on the visual morphological information sequence to obtain the visual morphological feature vector of the Chinese text to be corrected and several document block texts.
[0059] Specifically, the correction device extracts visual morphological information from the n×n pixel images corresponding to each character in the Chinese text to be corrected and several document block texts. It then processes the character images using a CNN model with a width of f to obtain a sequence of visual morphological information of the Chinese text to be corrected and several document block texts. The correction device then generates morphological embeddings using a max-pooling operation and performs feature vector transformation on the sequence of visual morphological information to obtain the visual morphological feature vectors of the Chinese text to be corrected and several document block texts.
[0060] S3: Based on the feature vectors of the Chinese text to be corrected and several document block texts, extract several target document block texts from the several document block texts to construct the domain-specific retrieval text of the Chinese text to be corrected.
[0061] In this embodiment, the correction device extracts several target document block texts from the document block texts based on the feature vectors of the Chinese text to be corrected and several document block texts, and constructs a domain-specific retrieval text for the Chinese text to be corrected.
[0062] Please see Figure 3 , Figure 3 The flowchart of S3 in the Chinese text correction method based on spell checking provided in one embodiment of this application includes steps S31 to S32, as follows:
[0063] S31: Concatenate the semantic feature vector, pinyin feature vector, and visual morphological feature vector of the Chinese text to be corrected and the text of the same document block respectively to obtain the original multimodal vector of the Chinese text to be corrected and several document blocks. Perform dimensional transformation on the original multimodal vector to obtain the final multimodal vector of the Chinese text to be corrected and several document blocks.
[0064] In this embodiment, the correction device concatenates the semantic feature vector, pinyin feature vector, and visual morphological feature vector of the Chinese text to be corrected and the text of the same document block, respectively, to obtain the original multimodal vector of the Chinese text to be corrected and several document blocks.
[0065] The correction device uses a fully connected layer to perform dimensionality transformation on the original multimodal vector to obtain the final multimodal vector of the Chinese text to be corrected and several document block texts.
[0066] S32: Based on the final multimodal vectors of the Chinese text to be corrected and several document block texts, obtain the similarity between the Chinese text to be corrected and several document block texts. Based on the similarity, extract several target document block texts from the several document block texts to construct the domain-specific retrieval text of the Chinese text to be corrected.
[0067] In this embodiment, the correction device obtains the similarity between the Chinese text to be corrected and the several document block texts based on the final multimodal vectors of the Chinese text to be corrected and the several document block texts.
[0068] Specifically, the correction device uses the Euclidean distance calculation method to obtain the Euclidean distance between the Chinese text to be corrected and the several document block texts based on the final multimodal vectors of the Chinese text to be corrected and the several document block texts, which is used as the similarity between the Chinese text to be corrected and the several document block texts.
[0069] The correction device sorts the document block texts in ascending order according to the relevance, identifies the top f document block texts as target document block texts, extracts several target document block texts from the document block texts, and constructs the domain-specific retrieval text of the Chinese text to be corrected.
[0070] S4: Construct a case pool for the specific domain of the Chinese text to be corrected.
[0071] In this embodiment, the correction device constructs a case pool for a specific domain of the Chinese text to be corrected, wherein the case pool includes several sets of specific text samples.
[0072] Please see Figure 4 , Figure 4 The flowchart of S4 in the Chinese text correction method based on spell checking provided in one embodiment of this application includes steps S41 to S44, as follows:
[0073] S41: Obtain several correct texts in a specific domain of the Chinese text to be corrected, and construct several types of confusion sets.
[0074] In this embodiment, the correction device obtains several correct texts in a specific domain of the Chinese text to be corrected, and constructs several types of confusion sets. The confusion set includes several confusion subsets corresponding to several characters, and the confusion subset includes the confusion probability between the character and several corresponding similar characters.
[0075] Specifically, the confusion set includes a pinyin confusion set and a visual morphology confusion set; the pinyin confusion set includes a pinyin confusion subset corresponding to several characters, and the pinyin confusion subset includes the pinyin confusion probability between the character and several corresponding similar characters; the visual morphology confusion set includes a visual morphology confusion subset corresponding to several characters, and the visual morphology confusion subset includes the visual morphology confusion probability between the character and several corresponding similar characters. Please refer to [link to relevant documentation]. Figure 5 , Figure 5 The flowchart of S41 in the Chinese text correction method based on spell checking provided in one embodiment of this application includes steps S411 to S413, as follows:
[0076] S411: Traverse the characters in the preset dictionary to obtain the pinyin information vector and visual morphological feature vector of several characters.
[0077] In this embodiment, the correction device traverses the characters in a preset dictionary to obtain the pinyin information vectors and visual morphological feature vectors of several characters. Specific implementation details can be found in steps S22-S23, and will not be repeated here.
[0078] S412: Calculate the similarity based on the pinyin feature vector and visual morphology feature vector of several characters respectively to obtain the pinyin similarity and visual morphology similarity between several characters; based on the pinyin similarity, visual morphology similarity and the corresponding similarity threshold, obtain several similar characters corresponding to several characters, and construct a pinyin similar character set and a visual morphology similar character set corresponding to several characters.
[0079] In this embodiment, the correction device employs a similarity calculation method, performing similarity calculations based on the pinyin feature vectors and visual morphological feature vectors of several characters respectively, to obtain the pinyin similarity and visual morphological similarity between the several characters, wherein the pinyin similarity is:
[0080]
[0081] In the formula, Let be the similarity in pinyin between the i-th character and the j-th character. Let be the pinyin feature vector of the i-th character. Let be the pinyin feature vector of the j-th character.
[0082] The visual morphological similarity is:
[0083]
[0084] In the formula, Let be the visual morphological similarity between the i-th character and the j-th character. Let be the visual morphological feature vector of the i-th character. Let be the visual morphological feature vector of the j-th character.
[0085] The correction device obtains several similar characters corresponding to several characters based on the pinyin similarity, visual morphological similarity, and corresponding similarity thresholds, and constructs a pinyin similar character set and a visual morphological similar character set corresponding to several characters. Specifically, if the pinyin similarity is greater than the corresponding similarity threshold, it obtains several pinyin similar characters corresponding to several characters and constructs a pinyin similar character set corresponding to several characters; if the visual morphological similarity is greater than the corresponding similarity threshold, it obtains several visual morphological similar characters corresponding to several characters and constructs a visual morphological similar character set corresponding to several characters.
[0086] S413: Calculate the confusion probability between the characters in the pinyin similarity set and the visual morphology similarity set corresponding to the characters and the corresponding similar characters, respectively, to obtain the pinyin confusion probability and visual morphology confusion probability between the characters and the similar characters, and construct the pinyin confusion set and the visual morphology confusion set.
[0087] In this embodiment, the correction device calculates the confusion probability based on the pinyin similarity and visual morphological similarity between characters in the pinyin similarity set and the visual morphological similarity set corresponding to several characters and their corresponding several similar characters, respectively, to obtain the pinyin confusion probability and visual morphological confusion probability between several characters and several similar characters, and constructs the pinyin confusion set and the visual morphological confusion set, wherein the pinyin confusion probability is:
[0088]
[0089] In the formula, p pe (char k |char i Let be the probability of confusion between the i-th character and its corresponding k-th similar character in Pinyin. Let m be the pinyin similarity between the i-th character and its corresponding k-th similar character, and m be the number of similar characters. Let be the pinyin similarity between the i-th character and its corresponding t-th similar character.
[0090] The visual morphological confusion probability is:
[0091]
[0092] In the formula, p ge (char k |char i The visual morphological confusion probability between the i-th character and its corresponding k-th similar character. Let be the visual morphological similarity between the i-th character and its corresponding k-th similar character. Let be the visual morphological similarity between the i-th character and its corresponding t-th similar character.
[0093] The phonetic confusion probability and visual morphological confusion probability between the same character and several similar characters are combined to construct the phonetic confusion set and the visual morphological confusion set. Through similarity calculation methods, several phonetic similar characters and visual morphological similar characters corresponding to several characters are determined. Based on the phonetic similarity and visual morphological similarity, the corresponding phonetic confusion probability and visual morphological confusion probability are calculated respectively. This realizes the construction of confusion sets driven by similarity probability theory, so that each similar character in the confusion set has a separate confusion probability that better reflects the similarity with the corresponding character. This allows characters with high similarity to be selected with a higher probability to construct case templates and improve the accuracy of text correction.
[0094] S42: Obtain the first sampling probability of several characters in the correct text, and determine several characters to be replaced in the correct text based on the first sampling probability and the preset first sampling probability threshold.
[0095] In this embodiment, the correction device obtains a first sampling probability of several characters in the correct text, wherein the first sampling probability is used to determine whether a character has been replaced.
[0096] The correction device is based on the first sampling probability A preset first sampling probability threshold is used to determine several characters to be replaced in the correct text. Specifically, to ensure that the number of spelling errors contained in different texts has a certain variation, the first sampling probability threshold is a dynamically changing threshold. If the first sampling probability of the character is greater than or equal to the first sampling probability threshold, the character is determined to be used to replace the corresponding character in the confusion set.
[0097] S43: Obtain the second sampling probability of several characters to be replaced, and determine the confusion subset corresponding to several characters to be replaced based on the second sampling probability and the preset second sampling probability threshold.
[0098] In this embodiment, the correction device obtains a second sampling probability of several of the characters to be replaced. The second sampling probability is used to determine the type of confusion set used for character replacement.
[0099] The correction device determines several confusion subsets corresponding to the characters to be replaced based on the second sampling probability and a preset second sampling probability threshold. Specifically, the correction device determines that the confusion subsets corresponding to the characters to be replaced are either pinyin confusion subsets or visual morphological confusion subsets, as follows:
[0100]
[0101] In the formula, D * The confusion set used for character substitution, D pe For the Pinyin confusion set, D ge Let β be the visual morphological confusion set, and let β be the second sampling probability threshold.
[0102] The correction device takes the obfuscated subset of characters in the obfuscation set that are the same as the character to be replaced as the obfuscated subset corresponding to the character to be replaced, and determines several obfuscated subsets corresponding to the character to be replaced.
[0103] S44: Obtain the third sampling probability of several characters in the confusion subset corresponding to several characters to be replaced, and based on the third sampling probability and the confusion probability between the characters in the corresponding confusion subset and several similar characters, confirm the target characters corresponding to several characters to be replaced.
[0104] In this embodiment, the correction device obtains a third sampling probability of several characters in a confusion subset corresponding to several characters to be replaced, wherein the third sampling probability is used to determine similar characters in the confusion subset corresponding to the characters to be replaced;
[0105] The correction device is based on the third sampling probability. And based on the confusion probability between the character in the corresponding confusion subset and several similar characters, the correction device identifies target characters in the confusion set corresponding to several characters to be replaced. Specifically, the correction device, based on the third sampling probability, the confusion probability between the character in the corresponding confusion subset and several similar characters, and a judgment formula, identifies target characters corresponding to several characters to be replaced if the third sampling probability of the character to be replaced satisfies the judgment formula, wherein the judgment formula is:
[0106]
[0107] In the formula, Let be the target character in the obfuscation subset corresponding to the i-th character to be replaced, char k For the k-th similar character in the confusion subset, p(char k |char i ) represents the confusion probability between the i-th character and its corresponding k-th similar character.
[0108] S45: Replace the target character with the corresponding character to be replaced in the correct text, construct several replacement texts corresponding to the correct text, and combine the several correct texts with the corresponding replacement texts to construct several sets of specific text samples.
[0109] S5: Based on the Chinese text to be corrected, extract several sets of target-specific text samples from the case pool of the specific domain, and construct a specific domain case template for the Chinese text to be corrected.
[0110] In this embodiment, the correction device extracts several sets of target-specific text samples from the case pool of the specific domain based on the Chinese text to be corrected, and constructs a specific domain case template for the Chinese text to be corrected.
[0111] Please see Figure 6 , Figure 6 The flowchart of step S5 in the Chinese text correction method based on spell checking provided in one embodiment of this application includes steps S51 to S54, as follows:
[0112] S51: Calculate the context probability of several groups of specific text samples in the case pool of the specific domain to obtain the context probability data of several groups of specific text samples.
[0113] In this embodiment, the correction device uses the same language model as in step S21 to input several sets of specific text samples from the case pool of the specific domain into the language model for context probability calculation, thereby obtaining context probability data of several sets of specific text samples. The context probability data of the specific text samples includes the probability between several context characters of the correct text and the replacement text.
[0114] S52: Obtain the text length of several sets of specific text samples, and based on the text length of several sets of specific text samples, context probability data, and a preset fluency calculation algorithm, obtain the fluency of the correct text and the replacement text in several sets of specific text samples, and subtract the fluency of the replacement text and the correct text in the same set of specific text samples to obtain the fluency difference of several sets of specific text samples.
[0115] In this embodiment, the correction device obtains the text lengths of several sets of specific text samples. Specifically, the text lengths of the correct text and the replacement text in the specific text samples are the same, and the text length of the correct text or the replacement text can be used as the text length of the corresponding specific text sample.
[0116] The correction device obtains the fluency of the correct text and the replacement text in several sets of specific text samples based on the text length, context probability data, and a preset fluency calculation algorithm. It then subtracts the fluency of the replacement text and the correct text in the same set of specific text samples to obtain the fluency difference among the several sets of specific text samples. The fluency calculation algorithm is as follows:
[0117]
[0118] In the formula, fluency is the fluency level, x is the text length, and x i Let x be the i-th character in the text. <i Let P(x) be the preceding character of the i-th character in the text. i |x <i ) is the given context character x i and x <i The probability between them.
[0119] S53: Perform preliminary correction processing on the Chinese text to be corrected to obtain preliminary corrected Chinese text; combine the Chinese text to be corrected and the preliminary corrected Chinese text to obtain a corrected text sample; perform context probability calculation on the corrected text sample to obtain context probability data of the corrected text sample.
[0120] In this embodiment, the correction device performs preliminary correction processing on the Chinese text to be corrected to obtain preliminary corrected Chinese text. The Chinese text to be corrected and the preliminary corrected Chinese text are then combined to obtain a corrected text sample. Specifically, the correction device inputs the Chinese text to be corrected into the same language model as S51 for preliminary correction processing to obtain the preliminary corrected Chinese text.
[0121] The correction device performs context probability calculation on the corrected text sample to obtain context probability data of the corrected text sample. The context probability data of the corrected text sample includes the probability between several context characters of the Chinese text to be corrected and the Chinese text to be initially corrected. For specific embodiments, please refer to step S51, which will not be repeated here.
[0122] S54: Obtain the text length of the corrected text sample; based on the text length of the corrected text sample, context probability data, and the fluency calculation algorithm, obtain the fluency of the Chinese text to be corrected and the initially corrected Chinese text in the corrected text sample; subtract the fluency of the initially corrected Chinese text and the Chinese text to be corrected to obtain the fluency difference of the corrected text sample.
[0123] In this embodiment, the correction device obtains the text length of the corrected text sample, and obtains the fluency of the Chinese text to be corrected and the initially corrected Chinese text in the corrected text sample based on the text length of the corrected text sample, the context probability data, and the fluency calculation algorithm. The fluency difference between the initially corrected Chinese text and the Chinese text to be corrected is obtained by subtracting the fluency of the Chinese text to be corrected. For a specific embodiment, please refer to step S52, which will not be repeated here.
[0124] S55: Based on the fluency difference and text length of the corrected text sample and several sets of specific text samples, extract several sets of target specific text samples from the several sets of specific text samples to construct a specific domain case template for the Chinese text to be corrected.
[0125] In this embodiment, the correction device extracts several sets of target specific text samples from the several sets of specific text samples based on the fluency difference and text length of the corrected text sample and several sets of specific text samples, and constructs a specific domain case template for the Chinese text to be corrected.
[0126] Please see Figure 7 , Figure 7 The flowchart of S55 in the Chinese text correction method based on spell checking provided in one embodiment of this application includes steps S551 to S552, as follows:
[0127] S551: If the text length of the corrected text sample differs from that of the specific text sample by a preset text length threshold, the specific text sample is used as a candidate specific text sample to obtain several candidate specific text samples.
[0128] If the text length of the corrected text sample differs from that of the specific text sample by a preset text length threshold, in this embodiment, the correcting device uses the specific text sample as a candidate specific text sample to obtain several candidate specific text samples.
[0129] S552: Using a proximity distance calculation method, based on the fluency difference between the corrected text sample and several candidate specific text samples, the proximity distance between the corrected text sample and several candidate specific text samples is obtained; based on the proximity distance, several groups of candidate specific text samples that are closest to the corrected text sample are obtained as target specific text samples, and several groups of target specific text samples are extracted from several groups of specific text samples.
[0130] In this embodiment, the correction device employs a proximity distance calculation method to obtain the proximity distance between the corrected text sample and the candidate specific text samples based on the fluency difference between the corrected text sample and the candidate specific text samples. The proximity distance is:
[0131] distance ir =|Δfluency i -Δfluency r |
[0132] In the formula, distance irΔfluency is the nearest neighbor distance between the i-th corrected text sample and the r-th candidate specific text sample. i Let Δfluency be the fluency difference for the i-th corrected text sample. r Let be the fluency difference for the r-th candidate specific text sample.
[0133] That is, when the two have similar relative fluency, the closer the proximity distance between the two text samples is, the closer the value is to 0; otherwise, the value is larger. The correction device obtains several groups of candidate specific text samples that are closest to the proximity distance of the corrected text sample, as target specific text samples, and extracts several groups of target specific text samples from the several groups of specific text samples.
[0134] S6: Combine the Chinese text to be corrected, the domain-specific search text of the Chinese text to be corrected, and the domain-specific case template to construct a sequence of instructions for the Chinese text to be corrected; input the sequence of instructions for the Chinese text to be corrected into a pre-trained large language model for text correction to obtain the correction result of the Chinese text to be corrected.
[0135] The pre-trained large language model can employ advanced language models such as ChatGPT and Qwen, enabling text correction based on spell check task prompts. By introducing external domain knowledge and relevant case studies for demonstration, the language model is trained, effectively enhancing its knowledge background and compensating for any lack of domain-specific knowledge. This further ensures the language model generates error-correcting text according to standard paradigms, preventing the generation of excessive information irrelevant to standard results and improving its ability to identify and correct errors in specific domain characters.
[0136] In this embodiment, the correction device combines the Chinese text to be corrected, the domain-specific search text of the Chinese text to be corrected, and the domain-specific case template to construct a command sequence for the Chinese text to be corrected; the command sequence for the Chinese text to be corrected is input into a pre-trained large language model for text correction to obtain the correction result of the Chinese text to be corrected.
[0137] By extracting domain-specific feature information from document text in a specific domain, constructing domain-specific retrieval text, and extracting relevant text cases from a domain-specific case pool to construct domain-specific case templates, the Chinese text to be corrected, the domain-specific retrieval text, and the domain-specific case templates are used as input data to be fed into a pre-trained model. By introducing external domain knowledge and relevant cases as demonstrations, the language model is guided to perform accurate text correction, preventing the language model from generating too much information unrelated to the text correction results, and improving the accuracy of the language model in recognizing and correcting errors in characters in a specific domain.
[0138] Please refer to Figure 8 , Figure 8 This is a schematic diagram of a Chinese text correction device based on spell checking, provided in one embodiment of this application. The device can be implemented entirely or partially through software, hardware, or a combination of both. The device 8 includes:
[0139] The text acquisition module 81 is used to acquire the Chinese text to be corrected and the document text of the Chinese text to be corrected in a specific domain, wherein the document text includes several document block texts;
[0140] The feature extraction module 82 is used to extract features from the Chinese text to be corrected and several document block texts to obtain feature vectors of the Chinese text to be corrected and several document block texts.
[0141] The text retrieval module 83 is used to extract several target document block texts from the several document block texts based on the feature vectors of the Chinese text to be corrected and several document block texts, and to construct a specific domain retrieval text for the Chinese text to be corrected.
[0142] The case pool construction module 84 is used to construct a case pool for a specific domain of the Chinese text to be corrected, wherein the case pool includes several sets of specific text samples;
[0143] The case template construction module 85 is used to extract several sets of target-specific text samples from the case pool of the specific domain based on the Chinese text to be corrected, and construct a specific domain case template for the Chinese text to be corrected.
[0144] The text correction module 86 is used to combine the Chinese text to be corrected, the domain-specific search text of the Chinese text to be corrected, and the domain-specific case template to construct the instruction sequence of the Chinese text to be corrected; input the instruction sequence of the Chinese text to be corrected into the pre-trained large language model for text correction, and obtain the correction result of the Chinese text to be corrected.
[0145] In this embodiment, a text acquisition module obtains the Chinese text to be corrected and document text of a specific domain of the Chinese text to be corrected, wherein the document text includes several document block texts; a feature extraction module extracts features from the Chinese text to be corrected and the several document block texts to obtain feature vectors of the Chinese text to be corrected and the several document block texts; a text retrieval module extracts several target document block texts from the several document block texts based on the feature vectors of the Chinese text to be corrected and the several document block texts to construct the specific domain retrieval text of the Chinese text to be corrected; and a case pool construction module constructs... The specific domain case pool of the Chinese text to be corrected includes several sets of specific text examples. A case template construction module extracts several sets of target specific text examples from the specific domain case pool based on the Chinese text to be corrected, constructing a specific domain case template for the Chinese text to be corrected. A text correction module combines the Chinese text to be corrected, its specific domain retrieval text, and the specific domain case template to construct a command sequence for the Chinese text to be corrected. This command sequence is then input into a pre-trained large language model for text correction to obtain the correction result. By extracting domain feature information from specific domain document text to construct specific domain retrieval text, and extracting relevant text examples from the specific domain case pool to construct specific domain case templates, these are combined with the Chinese text to be corrected, the specific domain retrieval text, and the specific domain case templates as input data to the pre-trained model. By introducing external domain knowledge and relevant examples as demonstrations, the language model is guided to perform accurate text correction, preventing the language model from generating too much information unrelated to the text correction result, and improving the accuracy of the language model in recognizing and correcting errors in specific domain characters.
[0146] Please refer to Figure 9 , Figure 9 This is a schematic diagram of the structure of a computer device provided in one embodiment of this application. The computer device 9 includes: a processor 91, a memory 92, and a computer program 93 stored in the memory 92 and executable on the processor 91; the computer device can store multiple instructions, which are adapted to be loaded and executed by the processor 91. Figures 1 to 7 The method steps of the illustrated embodiment can be specifically executed as follows: Figures 1 to 7 Specific details will not be elaborated here.
[0147] The processor 91 may include one or more processing cores. The processor 91 connects to various parts of the server using various interfaces and lines. It executes various functions and processes data of the Chinese text correction device 8 based on spell checking by running or executing instructions, programs, code sets, or instruction sets stored in memory 92, and by calling data from memory 92. Optionally, the processor 91 may be implemented using at least one hardware form of Digital Signal Processing (DSP), Field-Programmable Gate Array (FPGA), or Programmable Logic Array (PLA). The processor 91 may integrate one or a combination of several of the following: a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), and a modem. The CPU primarily handles the operating system, user interface, and applications; the GPU is responsible for rendering and drawing the content required to be displayed on the touch screen; and the modem handles wireless communication. It is understood that the modem may also be implemented as a separate chip without being integrated into the processor 91.
[0148] The memory 92 may include random access memory (RAM) or read-only memory. Optionally, the memory 92 may include a non-transitory computer-readable storage medium. The memory 92 can be used to store instructions, programs, code, code sets, or instruction sets. The memory 92 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function (such as touch instructions), instructions for implementing the various method embodiments described above, etc.; the data storage area may store data involved in the various method embodiments described above, etc. Optionally, the memory 92 may also be at least one storage device located remotely from the aforementioned processor 91.
[0149] This application embodiment also provides a storage medium that can store multiple instructions, which are adapted to be loaded and executed by a processor as described above. Figures 1 to 7 For the method steps and specific execution process, please refer to [link / reference]. Figures 1 to 7 Specific details will not be elaborated here.
[0150] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.
[0151] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.
[0152] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.
[0153] In the embodiments provided by this invention, it should be understood that the disclosed apparatus / terminal devices and methods can be implemented in other ways. For example, the apparatus / terminal device embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.
[0154] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.
[0155] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.
[0156] If the integrated module / unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments of the present invention can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms.
[0157] This invention is not limited to the above-described embodiments. If any modifications or variations to this invention do not depart from the spirit and scope of this invention, and if such modifications and variations fall within the scope of the claims and equivalent technologies of this invention, then this invention also intends to include such modifications and variations.
Claims
1. A Chinese text correction method based on spell checking, characterized in that, Includes the following steps: Obtain the Chinese text to be corrected and the document text of the specific domain of the Chinese text to be corrected, wherein the document text includes several document block texts; Feature extraction is performed on the Chinese text to be corrected and several document block texts to obtain feature vectors of the Chinese text to be corrected and several document block texts, wherein the feature vectors include semantic feature vectors, pinyin feature vectors and visual morphological feature vectors; Based on the feature vectors of the Chinese text to be corrected and several document block texts, the semantic feature vector, pinyin feature vector, and visual morphology feature vector of the Chinese text to be corrected and the same document block text are concatenated to obtain the original multimodal vector of the Chinese text to be corrected and several document block texts. The original multimodal vector is then transformed to obtain the final multimodal vector of the Chinese text to be corrected and several document block texts. Based on the final multimodal vectors of the Chinese text to be corrected and several document block texts, the similarity between the Chinese text to be corrected and several document block texts is obtained. Based on the similarity, several target document block texts are extracted from the several document block texts to construct the domain-specific retrieval text of the Chinese text to be corrected. Construct a case pool for a specific domain of the Chinese text to be corrected, wherein the case pool includes several sets of specific text samples; Based on the Chinese text to be corrected, several sets of target-specific text samples are extracted from the case pool of the specific domain to construct a specific domain case template for the Chinese text to be corrected. The Chinese text to be corrected, along with its specific domain retrieval text and specific domain case template, are combined to construct a sequence of instructions for the Chinese text to be corrected. This sequence of instructions is then input into a pre-trained large language model for text correction to obtain the corrected result of the Chinese text.
2. The Chinese text correction method based on spell checking according to claim 1, characterized in that, The step of extracting features from the Chinese text to be corrected and several document block texts to obtain feature vectors for the Chinese text to be corrected and several document block texts includes the following steps: Semantic information is extracted from the Chinese text to be corrected and several document block texts to obtain a semantic information sequence of the Chinese text to be corrected and several document block texts; the semantic information sequence is transformed into a feature vector to obtain a semantic feature vector of the Chinese text to be corrected and several document block texts. Using a preset pinyin conversion tool, pinyin information is extracted from the Chinese text to be corrected and several document block texts to obtain the pinyin information sequence of the Chinese text to be corrected and several document block texts. The pinyin information sequence is then converted into a feature vector to obtain the pinyin feature vector of the Chinese text to be corrected and several document block texts. An image feature encoding method is used to extract visual morphological information from the Chinese text to be corrected and several document block texts to obtain a sequence of visual morphological information of the Chinese text to be corrected and several document block texts. The sequence of visual morphological information is then transformed into a feature vector to obtain the visual morphological feature vector of the Chinese text to be corrected and several document block texts.
3. The Chinese text correction method based on spell checking according to claim 2, characterized in that, The process of constructing a domain-specific case pool for the Chinese text to be corrected includes the following steps: Obtain several correct texts in a specific domain of the Chinese text to be corrected, and construct several types of confusion sets, wherein the confusion set includes several confusion subsets corresponding to several characters, and the confusion subset includes the confusion probability between the character and several corresponding similar characters; Obtain the first sampling probability of several characters in the correct text, and determine several characters to be replaced in the correct text based on the first sampling probability and a preset first sampling probability threshold, wherein the first sampling probability is used to determine whether the character is replaced; A second sampling probability of several characters to be replaced is obtained, and based on the second sampling probability and a preset second sampling probability threshold, a confusion subset corresponding to several characters to be replaced is determined, wherein the second sampling probability is used to determine the type of confusion set used for character replacement; Obtain the third sampling probability of several characters in the confusion subset corresponding to several characters to be replaced; based on the third sampling probability and the confusion probability between the characters in the corresponding confusion subset and several similar characters, confirm the target characters corresponding to several characters to be replaced. Replace the target character with the corresponding character to be replaced in the correct text to construct several replacement texts corresponding to the correct text. Combine the several correct texts with their corresponding replacement texts to construct several sets of specific text samples.
4. The Chinese text correction method based on spell checking according to claim 3, characterized in that: The confusion set includes a pinyin confusion set and a visual morphology confusion set; the pinyin confusion set includes a pinyin confusion subset corresponding to several characters, and the pinyin confusion subset includes the pinyin confusion probability between the character and several corresponding similar characters; The visual morphological confusion set includes a visual morphological confusion subset corresponding to several characters, and the visual morphological confusion subset includes the visual morphological confusion probability between the character and several corresponding similar characters. The construction of several types of obfuscation sets includes the following steps: Traverse the characters in the preset dictionary to obtain the pinyin information vector and visual morphological feature vector of several characters; Similarity is calculated based on the pinyin feature vector and visual morphology feature vector of several characters respectively to obtain the pinyin similarity and visual morphology similarity between several characters; based on the pinyin similarity, visual morphology similarity and the corresponding similarity threshold, several similar characters corresponding to several characters are obtained, and a pinyin similar character set and a visual morphology similar character set corresponding to several characters are constructed. The confusion probability is calculated based on the pinyin similarity and visual morphological similarity between characters in the pinyin similarity set and visual morphological similarity set corresponding to several characters and their corresponding several similar characters, respectively, to obtain the pinyin confusion probability and visual morphological confusion probability between several characters and several similar characters, and to construct the pinyin confusion set and visual morphological confusion set.
5. The Chinese text correction method based on spell checking according to claim 2, characterized in that, The step of extracting several sets of target-specific text samples from the case pool of the specific domain based on the Chinese text to be corrected, and constructing a specific domain case template for the Chinese text to be corrected, includes the following steps: Context probability calculations are performed on several sets of specific text samples in the case pool of the specific domain to obtain context probability data of several sets of specific text samples. The context probability data of the specific text samples includes the probability between several context characters of the correct text and the replacement text. The text lengths of several sets of specific text samples are obtained. Based on the text lengths of the several sets of specific text samples, context probability data, and a preset fluency calculation algorithm, the fluency of the correct text and the replacement text in the several sets of specific text samples is obtained. The fluency of the replacement text and the correct text in the same set of specific text samples is subtracted to obtain the fluency difference of the several sets of specific text samples. The fluency calculation algorithm is as follows: In the formula, For fluency, For text length, For the first in the text i One character, For the first in the text i The preceding character of a character, For a given context character as well as The probability between; The Chinese text to be corrected is subjected to preliminary correction processing to obtain preliminary corrected Chinese text. The Chinese text to be corrected and the preliminary corrected Chinese text are combined to obtain a corrected text sample. The corrected text sample is subjected to context probability calculation to obtain context probability data of the corrected text sample. The context probability data of the corrected text sample includes the probability between several context characters of the Chinese text to be corrected and the preliminary corrected Chinese text. The text length of the corrected text sample is obtained. Based on the text length of the corrected text sample, the context probability data, and the fluency calculation algorithm, the fluency of the Chinese text to be corrected and the Chinese text to be initially corrected in the corrected text sample is obtained. The fluency of the initially corrected Chinese text and the Chinese text to be corrected are subtracted to obtain the fluency difference of the corrected text sample. Based on the fluency difference and text length of the corrected text sample and several sets of specific text samples, several sets of target specific text samples are extracted from the several sets of specific text samples to construct a specific domain case template for the Chinese text to be corrected.
6. The Chinese text correction method based on spell checking according to claim 5, characterized in that, The step of extracting several sets of target specific text samples from several sets of specific text samples based on the fluency difference and text length of the corrected text samples includes the following steps: If the text length of the corrected text sample differs from that of the specific text sample by a preset text length threshold, the specific text sample is used as a candidate specific text sample to obtain several candidate specific text samples; Using a proximity distance calculation method, the proximity distance between the corrected text sample and several candidate specific text samples is obtained based on the fluency difference between the corrected text sample and several candidate specific text samples. Based on the proximity distance, several groups of candidate specific text samples that are closest to the corrected text sample are obtained as target specific text samples, and several groups of target specific text samples are extracted from several groups of specific text samples.
7. A Chinese text correction device based on spell checking, characterized in that, include: The text acquisition module is used to acquire the Chinese text to be corrected and the document text of the Chinese text to be corrected in a specific domain, wherein the document text includes several document block texts; The feature extraction module is used to extract features from the Chinese text to be corrected and several document block texts to obtain feature vectors of the Chinese text to be corrected and several document block texts, wherein the feature vectors include semantic feature vectors, pinyin feature vectors and visual morphological feature vectors. The text retrieval module is used to extract several target document block texts from the several document block texts based on the feature vectors of the Chinese text to be corrected and several document block texts, construct the domain-specific retrieval text of the Chinese text to be corrected, and concatenate the semantic feature vector, pinyin feature vector and visual morphology feature vector of the Chinese text to be corrected and the same document block text respectively to obtain the original multimodal vector of the Chinese text to be corrected and several document block texts, and perform dimensional transformation on the original multimodal vector to obtain the final multimodal vector of the Chinese text to be corrected and several document block texts. Based on the final multimodal vectors of the Chinese text to be corrected and several document block texts, the similarity between the Chinese text to be corrected and several document block texts is obtained. Based on the similarity, several target document block texts are extracted from the several document block texts to construct the domain-specific retrieval text of the Chinese text to be corrected. The case pool construction module is used to construct a case pool for a specific domain of the Chinese text to be corrected, wherein the case pool includes several sets of specific text samples; The case template construction module is used to extract several sets of target-specific text samples from the case pool of the specific domain based on the Chinese text to be corrected, and construct a specific domain case template for the Chinese text to be corrected. The text correction module is used to combine the Chinese text to be corrected, the domain-specific search text of the Chinese text to be corrected, and the domain-specific case template to construct a sequence of instructions for the Chinese text to be corrected; and input the sequence of instructions for the Chinese text to be corrected into a pre-trained large language model for text correction to obtain the correction result of the Chinese text to be corrected.
8. A computer device, characterized in that, include: A processor, a memory, and a computer program stored in the memory and executable on the processor; the computer program, when executed by the processor, implements the steps of the Chinese text correction method based on spell checking as described in any one of claims 1 to 6.
9. A storage medium, characterized in that: The storage medium stores a computer program that, when executed by a processor, implements the steps of the Chinese text correction method based on spell checking as described in any one of claims 1 to 6.
Citation Information
Patent Citations
Selling error correction model training method, spelling error correction method and storage medium
CN115455948A
Video retrieval method and device based on multiple modes, electronic equipment and storage medium
CN116628263A