A document proofreading method, device and equipment based on word matching

Abstract

This application discloses a document proofreading method, apparatus, and device based on word matching. The method includes: collecting standard word expressions from multiple documents and constructing a positive lexicon, which includes basic vocabulary and special vocabulary; constructing a prefix index and a feature character index for each word in the positive lexicon; performing reverse similarity matching on a target text sentence sequence based on the positive lexicon, the prefix index, and the feature character index to obtain a candidate word sequence, which includes word position and weighted similarity; using a pre-set large language model to verify the candidate word sequence based on level and similarity to obtain a confirmed word sequence; and mapping the confirmed word sequence back to the target text sentence to generate a proofread document. This application can solve the technical problems of insufficient error coverage of existing error lexicons; high computational cost, high latency, and insufficient utilization of document structure information in direct proofreading using large models; and the lack of timeliness, accuracy, and reliability in document proofreading.

Description

Technical Field

[0001] This application relates to the field of large language model technology, and in particular to a document proofreading method, apparatus and device based on word matching. Background Technology

[0002] Chinese text correction is a core foundational technology for intelligent office work and is a rigid requirement in large-scale daily office operations. Due to the lack of explicit morphological changes in Chinese, the high density of homophones and similar-looking characters, and the heavy reliance on contextual semantics, coupled with the fact that office documents involve a large number of professional terms such as legally serious corporate organizational structure names, the difficulty of text correction is far greater than in general text processing scenarios, placing extremely high demands on accuracy, compliance, and real-time performance.

[0003] Existing Chinese text correction technologies mainly fall into two categories: one is the rule-based and error lexicon-based forward matching method, which compares strings using predefined error patterns to achieve simple error detection; the other is the generative error correction method based on large language models, which uses semantic understanding to handle complex contextual errors. Some studies have attempted to combine the two into a cascaded scheme, using a small model for initial filtering before feeding it into a large model, but this has not yet overcome the core technical bottlenecks and cannot adapt to the special proofreading needs of office documents.

[0004] Currently, existing technologies face three major problems: First, forward matching relies on an error-prone vocabulary, which cannot exhaustively list infinite error variations such as similar-looking word replacements and word order reversals in customized enterprise names, resulting in severely insufficient coverage. Second, large-scale models face high costs and long delays in full-text proofreading and reasoning, and the generation process is uncontrollable, making it easy to arbitrarily modify unchangeable terms and cause compliance risks. Third, there is a lack of a hierarchical protection mechanism for unchangeable names, and the failure to fully utilize document structure information leads to vague error location and inaccurate computing power allocation, becoming the core obstacle to the implementation of intelligent proofreading projects for office documents. Summary of the Invention

[0005] This application provides a document proofreading method, apparatus, and device based on word matching to address the insufficient error coverage of existing error lexicons; the high computational cost, high latency, and uncontrollable nature of direct proofreading using large models; and the lack of full utilization of document structure information, resulting in ambiguous error location; thus leading to the current technical problems of lack of timeliness, accuracy, and reliability in document proofreading.

[0006] In view of this, the first aspect of this application provides a document proofreading method based on word matching, including:

[0007] Standard vocabulary expressions are collected from various documents and a standard vocabulary database is constructed, which includes basic vocabulary and special vocabulary.

[0008] Construct initial prefix indexes and feature character indexes for the words in the orthographic dictionary;

[0009] Based on the positive lexicon, the initial prefix index, and the feature character index, reverse similarity matching is performed on the target text sentence sequence to obtain a candidate word sequence, which includes word position and weighted similarity.

[0010] The candidate word sequence is reviewed and confirmed based on level and similarity using a pre-set large language model to obtain a confirmed word sequence;

[0011] The confirmation word sequence is mapped back to the target text sentence to generate a proofreading document.

[0012] Preferably, constructing a first-character prefix index and a feature character index for each word in the orthographic dictionary includes:

[0013] A prefix index is created for the first character in the orthographic dictionary;

[0014] Based on the prefix index, the words in the orthographic dictionary are classified to obtain a set of initial character categories;

[0015] The first character classification set is sorted in descending order based on word length to obtain a descending first character sequence, thus completing the construction of the first character prefix index;

[0016] Extract the feature characters corresponding to the positive words in the positive word library to generate a feature character set;

[0017] An inverted mapping is established based on multiple feature characters in the feature character set to generate a feature character index.

[0018] Preferably, the step of performing reverse similarity matching on the target text sentence sequence based on the positive lexicon, the initial prefix index, and the feature character index to obtain a candidate word sequence includes:

[0019] Based on the initial prefix index, a candidate word matching a certain character in the target text sentence is searched in the orthography database to generate an initial candidate orthography set;

[0020] The first candidate substring is extracted from the initial candidate positive word set through a preset dynamic window, and the original similarity is calculated based on the first candidate substring. The change of the preset dynamic window follows the dynamic length window constraint condition.

[0021] An equal-length character weighting mechanism and a length-adaptive weighting mechanism are introduced to calculate weighted similarity based on the original similarity.

[0022] The candidate word sequences with the first character are selected by combining the weighted similarity and the preset level threshold;

[0023] The target text sentence is searched and matched a second time based on the feature character index. A substring of equal length within a preset length range is extracted from the positive word library to obtain a second candidate substring.

[0024] Based on the second candidate substring, length matching, punctuation constraint analysis and similarity calculation are performed respectively, and feature candidate word sequences are selected by combining similarity thresholds;

[0025] The candidate word sequence is formed by combining the first character candidate word sequence and the feature candidate word sequence.

[0026] Preferably, the step of performing reverse similarity matching on the target text sentence sequence based on the positive lexicon, the initial prefix index, and the feature character index to obtain a candidate word sequence further includes:

[0027] The target text stream is parsed in a paragraph-level manner to obtain structured text, which includes paragraph identifiers, paragraph text content, paragraph length, and paragraph position.

[0028] The structured text is robustly segmented using a preset segmentation mechanism to obtain the target text sentence.

[0029] Preferably, the step of performing reverse similarity matching on the target text sentence sequence based on the positive lexicon, the initial prefix index, and the feature character index to obtain a candidate word sequence further includes:

[0030] A greedy algorithm is used to perform deduplication and filtering on the candidate word sequence.

[0031] Preferably, the step of performing reverse similarity matching on the target text sentence sequence based on the positive lexicon, the initial prefix index, and the feature character index to obtain a candidate word sequence further includes:

[0032] A long word expansion mechanism is used to perform an extended screening analysis on the matching words in the candidate word sequence based on the original similarity.

[0033] Preferably, the step of using a preset large language model to review and confirm the candidate word sequence based on level and similarity to obtain a confirmed word sequence includes:

[0034] Cluster analysis is performed on the candidate word sequence to obtain a clustered word sequence;

[0035] Differentiated confirmation strategies for review and approval based on level and similarity;

[0036] Based on the preset structured prompts and the differentiated confirmation strategy, the clustered word sequence is reviewed and confirmed using a preset large language model to obtain a confirmed word sequence.

[0037] A second aspect of this application provides a word-matching-based document proofreading apparatus, comprising:

[0038] The lexicon building unit is used to collect standard word expressions from various documents and build a positive lexicon, which includes basic vocabulary and special vocabulary;

[0039] The index building unit is used to build a first-character prefix index and a feature character index for the words in the orthographic dictionary, respectively;

[0040] The reverse matching unit is used to perform reverse similarity matching on the target text sentence sequence based on the positive lexicon, the initial prefix index and the feature character index to obtain a candidate word sequence, wherein the candidate word sequence includes word position and weighted similarity.

[0041] The review and confirmation unit is used to review and confirm the candidate word sequence based on level and similarity using a preset large language model to obtain a confirmed word sequence;

[0042] The text proofreading unit is used to map the confirmation word sequence back to the target text sentence to generate a proofreading document.

[0043] Preferably, the reverse matching unit is specifically used for:

[0044] Based on the initial prefix index, a candidate word matching a certain character in the target text sentence is searched in the orthography database to generate an initial candidate orthography set;

[0045] The first candidate substring is extracted from the initial candidate positive word set through a preset dynamic window, and the original similarity is calculated based on the first candidate substring. The change of the preset dynamic window follows the dynamic length window constraint condition.

[0046] An equal-length character weighting mechanism and a length-adaptive weighting mechanism are introduced to calculate weighted similarity based on the original similarity.

[0047] The candidate word sequences with the first character are selected by combining the weighted similarity and the preset level threshold;

[0048] The target text sentence is searched and matched a second time based on the feature character index. A substring of equal length within a preset length range is extracted from the positive word library to obtain a second candidate substring.

[0049] Based on the second candidate substring, length matching, punctuation constraint analysis and similarity calculation are performed respectively, and feature candidate word sequences are selected by combining similarity thresholds;

[0050] The candidate word sequence is formed by combining the first character candidate word sequence and the feature candidate word sequence.

[0051] A third aspect of this application provides a document proofreading device based on word matching, the device including a processor and a memory;

[0052] The memory is used to store program code and transmit the program code to the processor;

[0053] The processor is configured to execute the word-matching-based document proofreading method described in the first aspect according to instructions in the program code.

[0054] As can be seen from the above technical solutions, the embodiments of this application have the following advantages:

[0055] This application provides a document proofreading method based on word matching, comprising: collecting standard word expressions from multiple documents and constructing a positive lexicon, the positive lexicon including basic vocabulary and special vocabulary; constructing a prefix index and a feature character index for each word in the positive lexicon; performing reverse similarity matching on the target text sentence sequence based on the positive lexicon, the prefix index, and the feature character index to obtain a candidate word sequence, the candidate word sequence including word position and weighted similarity; using a pre-set large language model to review and confirm the candidate word sequence based on level and similarity to obtain a confirmed word sequence; mapping the confirmed word sequence back to the target text sentence to generate a proofreading document.

[0056] The document proofreading method based on word matching provided in this application promotes a reverse similarity matching mechanism for misspelled words by constructing a positive word library and multi-level indexes, thereby replacing the forward matching operation based on the misspelled word library in the prior art. In the reverse similarity matching operation, preliminary matching can be performed through the first character prefix index, and the feature character index can achieve fallback matching, ensuring that the coverage of misspelled word matching is as large as possible, and even achieving full matching of errors. The input to the large language model is not the initial complete document information, but the candidate word sequence obtained by matching, which greatly reduces the information processing volume of the model. Moreover, the process of obtaining the candidate word sequence is controllable and adjustable, which can ensure the reduction of model cost and response latency. In addition, the text position information is introduced in the process of processing the target text sentence, so that the words in the candidate word sequence include attribute information such as word position, which can ensure the accuracy and reliability of error location, making subsequent document confirmation and proofreading more accurate. Therefore, this application can solve the problem that the error coverage of the error lexicon in the prior art is insufficient; while the computational load, cost and latency of direct proofreading of large models are high and uncontrollable, and the document structure information is not fully utilized, so the error location is vague; resulting in the technical problems of the lack of timeliness, accuracy and reliability of current document proofreading. Attached Figure Description

[0057] Figure 1 A flowchart illustrating a word-matching-based document proofreading method provided in this application embodiment;

[0058] Figure 2 This is a schematic diagram of a document proofreading device based on word matching, provided in an embodiment of this application. Detailed Implementation

[0059] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present application.

[0060] For easier understanding, please refer to Figure 1 This application provides an embodiment of a document proofreading method based on word matching, comprising:

[0061] Step 101: Collect standard word expressions from various documents and build a standard vocabulary library, which includes basic vocabulary and special vocabulary.

[0062] If this embodiment is applied to a specific office document proofreading scenario, it is necessary to collect and organize the standard expressions that cannot be changed in the office documents, and build a lexicon based on this. The lexicon in this embodiment can be mainly divided into basic vocabulary and special vocabulary. The special vocabulary can include the name of the enterprise's organizational structure, professional terms, and mandatory protected expressions.

[0063] Specifically, basic vocabulary covers common idioms, fixed collocations, and standardized expressions. These words are used frequently, so the error rate is also high, but changes will not cause serious consequences. Corporate organizational structure names include the full legal name and standardized abbreviation of the group company, subsidiaries, business units, and functional departments. These terms relate to the company's legal identity and must be completely consistent in contracts, official documents, and external materials; at least the abbreviations must be consistent, meaning standardized abbreviations can be used in different scenarios. Professional terminology includes theoretical innovations requiring fixed collocations. This type of terminology is highly standardized; any addition, deletion, order adjustment, or substitution of synonyms is considered an error. Mandatory protected expressions include absolutely unchangeable content such as names; errors in these expressions must be tolerated without exception.

[0064] Step 102: Construct a first-character prefix index and a feature character index for each word in the lexicon.

[0065] This embodiment constructs a multi-level index structure for the orthographic vocabulary database, reducing the computational complexity of subsequent matching processes and avoiding excessive traversal of massive orthographic words. Constructing a first-character prefix index allows for the creation of a prefix index for the first character of each orthographic word in the database. This index then retrieves all orthographic words beginning with that first character, forming a set. Similarly, a feature character index works in the same way, but instead of selecting the first character, it selects two or more special characters from the orthographic words to construct a combined index, indicating the word's position. The reason for constructing a feature character index after the first-character prefix index is to fill in any gaps. While the first-character prefix index is fast, it cannot identify words with incorrect first characters. Therefore, the feature character index serves as a fallback, ensuring that no matches are missed during index-based matching, meaning all errors are found.

[0066] Further, step 102 includes:

[0067] Create a prefix index for the first character in the orthographic dictionary;

[0068] Based on the prefix index, words in the orthographic dictionary are classified to obtain a set of initial character categories;

[0069] The first character classification set is sorted in descending order based on word length to obtain the descending first character sequence, thus completing the construction of the first character prefix index;

[0070] Extract the feature characters corresponding to the positive words in the positive word library to generate a feature character set;

[0071] An inverted mapping is established based on multiple feature characters in the feature character set to generate a feature character index.

[0072] Specifically, if the orthography is expressed as ,in, The total number of positive words. This represents the i-th positive word, and each positive word... The length or number of characters is denoted as In the process of constructing a first-letter prefix index, the first-letter prefix index can be expressed as... For all the orthogonal words in the orthogonal dictionary This index can be built in any of these cases, allowing words to be categorized according to their first character. Words with the same first character are grouped together, forming a dataset and resulting in a set of words categorized by their first character.

[0073]

[0074] in, Then it indicates a positive word. The first character, The first position; the overall meaning of the formula is that for any character c, the index... Return the set of all positive words that begin with the letter 'c'.

[0075] The positive words in the resulting set of initial character categories can be sorted in descending order of word length to obtain a descending sequence of initial characters. Longer words in this sequence will be prioritized for matching. This mechanism is designed because longer positive words usually carry more specific information and have higher value in error detection; at the same time, if shorter words are matched first, longer words that should be matched may be truncated, leading to missed detections. For example, when both "artificial intelligence" and "human" exist, prioritizing the matching of "artificial intelligence" can avoid incorrectly splitting "artificial intelligence" into two fragments: "human" + "intelligent".

[0076] In the process of building a feature character index, the feature character index can be expressed as: This index can quickly filter and perform fallback matching on areas not covered by the initial prefix index. For each positive word... The following feature character sets can be extracted:

[0077]

[0078] in, As the first character, For the last character, This is a middle character; it can only be extracted when the word length is at least 5 characters. The second-to-last character is extracted only when the word length is not less than 7. Based on these extracted feature characters, an inverted index can be constructed, that is, these feature characters point to the set of positive words containing that feature character. Therefore, the feature character index can be expressed as:

[0079]

[0080] This formula means that for any character c, the index... This can return a set of all positive words whose characteristic character is 'c'. Similarly, positive words can be classified in this way to obtain a set of categorized words.

[0081] The hierarchical indexing mechanism of the first-character prefix index and feature character index constructed in this embodiment can ensure the comprehensiveness of the misspelling search and further compress the candidate space. Moreover, for areas not covered by the first-character prefix index, the feature character index can serve as a fallback mechanism to initiate secondary matching, ensuring that high-value positive words are not missed due to differences in the first character. The dual-indexing mechanism in this embodiment can reduce the number of candidates from an average of 50 per position to 3 per position during the matching process, which can significantly reduce the amount of computation in the subsequent confirmation and proofreading operations and improve proofreading efficiency.

[0082] In addition, to avoid excessive traversal, this embodiment also establishes a dynamic length window constraint to constrain the change in the dynamic window size during the matching process. It is not an unlimited dynamic change, but an adjustment within a range, thereby limiting the length range of the substrings participating in the comparison during the reverse similarity matching with the positive word.

[0083] For positive words Define minimum matching length and maximum matching length for:

[0084]

[0085]

[0086] in, , These represent rounding down and rounding up, respectively; this formula is the dynamic length window constraint.

[0087] The mathematical principle of the dynamic length window constraint in this embodiment is based on the geometric properties of edit distance to achieve distance constraint; specifically, it uses edit distance... To measure strings , The difference between them is defined as a string. Convert to The minimum number of single-character editing operations required, such as insertion, deletion, and replacement. When the edit distance between two strings is d, their length difference satisfies the following inequality:

[0088]

[0089] If the required similarity exceeds the threshold Then we have:

[0090]

[0091] Then the range of length ratios can be derived:

[0092]

[0093] like Therefore, we have:

[0094]

[0095] It can be approximated as This is the theoretical principle behind the dynamic length window constraint mentioned above; it is also possible to set the absolute lower limit to 3 simultaneously to avoid meaningless matching of excessively short segments.

[0096] Step 103: Perform reverse similarity matching on the target text sentence sequence based on the positive lexicon, the first character prefix index, and the feature character index to obtain the candidate word sequence, which includes word position and weighted similarity.

[0097] In this embodiment, the target text sentence sequence is extracted from the target text that needs to be proofread. The target text can be a text stream identified in an office document. It can be understood that performing structured parsing of the text stream at the paragraph level can not only preserve the hierarchical structure of the original document, but also obtain the position information of the text. If the position information of the text is fully considered during the matching process, the accuracy of matching, confirmation, and proofreading can be ensured, and the proofreading effect can be improved.

[0098] Words in the target text sentence can be matched against a standard lexicon. The matching is based on the initial prefix index and the feature character index. The principle of the reverse similarity matching mechanism is to use a standard lexicon as a benchmark, with each standard word as an anchor point, actively searching for all segments similar to that standard word in the text to be proofread. Similarity calculations determine whether these segments are incorrect variants corresponding to that standard word. This mechanism can cover an infinite error space with a finite number of standard words. Regardless of how an error occurs, as long as its similarity to the standard standard word reaches a certain level, it can be detected, fundamentally solving the problems of missed detections and insufficient coverage in existing technologies that rely on forward matching with a standard error word lexicon.

[0099] Further, step 103 includes:

[0100] Based on the first-character prefix index, the system searches for matching candidate words in the orthography database according to a certain character in the target text sentence, and generates an initial candidate orthography set.

[0101] The first candidate substring is extracted from the initial candidate positive word set through a preset dynamic window, and the original similarity is calculated based on the first candidate substring. The change of the preset dynamic window follows the dynamic length window constraint condition.

[0102] We introduce a weighted similarity mechanism based on equal-length characters and an adaptive length weighted similarity mechanism, and calculate the weighted similarity based on the original similarity.

[0103] Candidate word sequences with the first character are selected by combining weighted similarity and preset level thresholds;

[0104] The target text sentence is searched and matched a second time based on the feature character index. The substrings of equal length within the preset length range are extracted from the positive word library to obtain the second candidate substring.

[0105] Based on the second candidate substring, length matching, punctuation constraint analysis and similarity calculation are performed respectively, and feature candidate word sequences are selected by combining similarity thresholds;

[0106] A candidate word sequence is constructed by combining the first character candidate word sequence and the feature candidate word sequence.

[0107] Specifically, this embodiment uses a positive lexicon as a benchmark to reverse-detect erroneous variants in the text sentence to be tested, calculates similarity, and selects a set of candidate words. This is the core matching mechanism. The matching process can be divided into two parts according to the index structure: primary character index-driven matching and feature index-based fallback matching. Assuming the target text sentence to be processed is... , where n is the sentence length, which is the number of characters. The sentence can be viewed as a sequence of multiple characters.

[0108] For each character position , with characters By querying the first-character prefix index, multiple related words can be selected from the orthographic dictionary to form an initial candidate orthographic word set:

[0109]

[0110] That is, all characters starting from the i-th character of the sentence All words that begin with a positive word will be included in the word set.

[0111] For each candidate positive word, there is You can set the size of the dynamic window. Furthermore, the size of this window changes in accordance with the dynamic length window constraint obtained above. Within this dynamic window, a substring can be extracted, which is the first candidate substring, expressed as... Where j is the sliding position of the window; the starting point of the window is... This ensures a matching result of at least three characters, avoiding meaningless short fragments; and the window endpoint is taken as... This can prevent the match from exceeding the end of the sentence.

[0112] This embodiment uses sequence comparison normalization to calculate the original similarity:

[0113]

[0114] in, This indicates that the similarity is calculated based on the Ratcliff / Obershelp pattern recognition algorithm by finding the longest matching subsequence, and it has good tolerance for changes in character order and partial insertions and deletions. This is the original similarity. During the calculation of the original similarity, the string similarity can be normalized to... The interval is defined as follows: a value of 1 indicates that the two parts are completely identical, and a value of 0 indicates that they are completely unrelated.

[0115] Building upon this, this embodiment also introduces an equal-length character weighting mechanism and a length-adaptive weighting mechanism. The equal-length character weighting mechanism works when the substring t to be matched and the positive word w have the same length and identical character sets, indicating that they only differ in character order, which is a typical input order error. In this case, the original similarity is weighted and improved.

[0116]

[0117] in, , These represent all the characters in the substring t to be matched and the positive word w, respectively. In other words, when characters completely overlap but are in different orders, the probability of a word error is higher than that of a typical editing error, thus an additional confidence bonus is given. The original similarity weight of 0.6 preserves the order information; if the order is completely reversed, the original similarity is lower. A fixed bonus of 0.4 ensures a high similarity even with a completely reversed order.

[0118] The length-adaptive weighting mechanism first requires defining a weighting function. Map the length of the positive word to the weight coefficient:

[0119]

[0120] When the length of a positive word is not less than 5, it can carry highly specific information, resulting in extremely high false positive detection. Furthermore, the error variation space for long words is relatively limited, so the detection sensitivity should be increased. Short positive words with a length less than 4 are easily confused with other word segments; for example, "big data" and "large number" might match "big mathematics." Therefore, appropriately increasing the weight can balance false negatives and false positives. Positive words with a length between 4 and 5 can maintain their standard weight. For short positive words, a length constraint can be added, meaning the length of the substring to be matched must not be less than the length of the positive word. This prevents meaningless matching caused by excessively short segments.

[0121] Based on this, the weighted similarity can be calculated by combining the calculated weight coefficients:

[0122]

[0123] By applying a differentiated threshold filtering mechanism based on the level, the level of positive words can be configured. Each of these levels corresponds to a threshold. , , , , This is the preset level threshold. If and only if:

[0124]

[0125] In other words, the weighted similarity must reach or exceed the corresponding threshold of the level to which the positive word belongs before it can be selected into the first character candidate word sequence.

[0126] Among them, the L4 level has the highest threshold, which can ensure the rigor of detection of mandatory protection expressions; while the L1 level has the lowest threshold, allowing for loose matching of general vocabulary.

[0127] Since the error might occur in the first character, the initial prefix index matching may not cover all text. Therefore, this embodiment requires a secondary search matching as a fallback to ensure that no omissions occur during the matching process. The secondary search matching is based on the feature character index. Specifically, it first traverses each character position i in the target text sentence that is not covered by the initial prefix index matching result. For each positive word w in the positive word library whose word length is within the threshold length range [4,12], it extracts a substring of equal length. This is the second candidate substring; moreover, the secondary search matching process strictly follows the equal-length constraint, for example, the lengths must be strictly equal, i.e. Fragments are not allowed to participate in the matching; there is no punctuation constraint, that is, the second candidate substring does not contain any punctuation marks; and the similarity threshold has been reached, i.e. Candidate words that meet these conditions can form a sequence of feature candidate words.

[0128] By combining the first-character candidate word sequence and the feature candidate word sequence, a candidate word sequence can be constructed. These word sequences can be organized into a suitable dictionary format, which is convenient for subsequent unified input into a large language model for review and confirmation. The dictionary-formatted word sequence should include attributes such as the starting position, similarity, and word length.

[0129] Furthermore, step 103, preceding the following, also includes:

[0130] The target text stream is parsed in a paragraph-level manner to obtain structured text, which includes paragraph identifiers, paragraph text content, paragraph length, and paragraph position.

[0131] A pre-defined segmentation mechanism is used to robustly segment the structured text to obtain the target text sentences.

[0132] It should be noted that it can receive office documents, identify their format, extract the target text stream, and obtain structured text by parsing according to paragraph hierarchy. Assuming the target document P consists of a sequence of paragraphs, it can be expressed as follows: Where n is the total number of paragraphs in the document. This represents the text of the i-th paragraph. Each paragraph's text... The mapping to structured text can be expressed as:

[0133]

[0134] in, Paragraph identifiers are unique and are typically represented by the paragraph's ordinal number within the document or a globally unique code. Paragraph text content, i.e., plain text after removing formatting marks; Paragraph length, which is the number of characters in the paragraph content; This represents the global starting position of the paragraph within the document, also known as the character offset or paragraph position. The global starting position follows a recursive relationship: , ,in, Indicates the start position of the document. The length of characters used to control formatting, such as paragraph separators and line breaks, depends on the specific file format. For DOCX format, the python-docx library is used to extract paragraph objects; for PDF format, the PyPDF2 or pdfplumber library is used to extract text blocks and identify paragraph boundaries; for WPS format, it is obtained through conversion or SDK interface; for TXT format, consecutive line breaks are used as paragraph separators.

[0135] Structured text in segment form cannot be directly matched; it needs to be robustly segmented to obtain individual text sentences, i.e., target text sentences. The preset segmentation mechanism is the sentence segmentation mechanism, which can be selected or designed according to actual needs. In this embodiment, the paragraph text is directly segmented based on the pairing symbol stack mechanism to obtain sentence sequences, which can ensure the accuracy of sentence boundaries, especially in handling nested structures such as quotation marks and parentheses.

[0136] Specifically, a set of paired symbols can be predefined:

[0137]

[0138] It includes common pairing symbols such as Chinese quotation marks, English quotation marks, various parentheses, and book titles; it can establish mapping relationships. This maps the left-hand symbol to the corresponding right-hand symbol, for example, $M(``)=''$, M('(')=')'.

[0139] Maintain a stack structure S, initially empty, and iterate through the character sequence of the paragraph. If the current character If the current character is the left sign, it will be pushed onto the stack S; if the current character is... That is, the right sign is true, and stack S is not an empty set, and the top element of the stack is true. If the character matches, the top of the stack is popped; otherwise, the character is treated as a normal character, and no stack operation is triggered for a mismatched right sign. If the current character... If all elements are sentence-ending punctuation marks and stack S is an empty set, then segmentation is performed; additionally, a set of noise punctuation marks can follow the sentence-ending punctuation marks. The characters in the string, these noisy punctuation marks, together with the sentence-ending punctuation, form the segmentation boundary, but do not trigger segmentation independently. An empty stack S indicates that the current element is not inside any paired symbol, and the sentence-ending punctuation encountered at this point is the true end of the sentence.

[0140] The paragraph text The set of splitting points is The resulting sequence of target text sentences is: , ,..., Each sentence Offset from its position within the paragraph Assuming This offset is relative to the global starting position. The relationship is:

[0141]

[0142] in, Indicates the global location.

[0143] Based on the above method, paragraph text can be segmented into sentences to obtain target text sentences, and the words in the target text sentences are the targets to be matched and proofread.

[0144] Furthermore, step 103, followed by:

[0145] A greedy algorithm is used to perform deduplication and screening of candidate word sequences.

[0146] To avoid duplicate matching, i.e., word overlap in the candidate word sequence, this embodiment can also design a greedy algorithm for non-overlapping filtering. This algorithm iterates through the sorted candidate words, and if the character position range of the current candidate does not intersect with the already selected word sequence, it is included in the result, and its overlapping character position is marked. This strategy ensures that the word with the highest weighted similarity and longest word length is selected for matching among overlapping candidates.

[0147] Furthermore, step 103, followed by:

[0148] A long word expansion mechanism is used to perform extended screening analysis on matching words in the candidate word sequence based on the original similarity.

[0149] This embodiment can also design a long word expansion mechanism to expand short words, provided that the short word has a longer and more standard expression; assuming that a certain matched positive word is Its word length is Starting at position i, the query's first-letter prefix index is expressed as follows: Filter longer headings that meet the following conditions. : ; for A continuous substring, i.e. For those that meet the conditions In the extended window Extracting substrings from the inner part Calculate their original similarity The weighting function in the expansion phase is adjusted as follows:

[0150]

[0151] Extended similarity can then be calculated as:

[0152]

[0153] The filtering criteria are That is, the original similarity is not lower than the corresponding level threshold. Two-thirds; through the extended filtering operation, the extended filtering result is merged with the original candidate word sequence. If the matching process of the first character prefix index is the main matching process, then the extended filtering result is merged with the word sequence obtained by matching the first character prefix index. The specific choice can be made according to the actual operation focus. All of these are optional operation cases in this embodiment. They are not limited here, but only the implementation idea is provided.

[0154] The long word expansion mechanism designed in this embodiment is to avoid missing detections due to truncation of long words in short word matching. For example, when artificial intelligence technology applications appear in the text, if the positive lexicon contains both artificial intelligence and artificial intelligence technology applications, this embodiment will prioritize matching the latter to obtain more complete error location and more accurate contextual information.

[0155] Step 104: Use a pre-set large language model to review and confirm the candidate word sequence based on level and similarity to obtain the confirmed word sequence.

[0156] The pre-built large language model in this embodiment is prepared in advance and can be configured with necessary text review and confirmation prompts according to the nature of the task in this embodiment; specific details are not limited here. Furthermore, the large language model here is mainly used to review and confirm already matched candidate word sequences, rather than directly proofreading them; that is, the candidate word sequences input to the model have already had their information content significantly reduced, lowering the model's computational load and directly improving the model's processing speed and confirmation efficiency. Different levels correspond to different level thresholds and specific weighted similarity; based on these factors, a dedicated strategy for the large language model can be designed to help it perform text judgment more quickly and accurately.

[0157] Further, step 104 includes:

[0158] Cluster analysis is performed on the candidate word sequences to obtain clustered word sequences;

[0159] Differentiated confirmation strategies for review and approval based on level and similarity;

[0160] Based on the preset structured prompts and differentiated confirmation strategies, a preset large language model is used to review and confirm the clustered word sequences, resulting in a confirmed word sequence.

[0161] Specifically, the candidate word sequence is input into a pre-defined large language model, which can then utilize its contextual understanding capabilities to perform semantic-level review and achieve intelligent confirmation. Prior to this, the candidate word sequence needs to be clustered according to the sentence context. Assuming the sentence... If we associate the candidate word sequence, then we have Based on this, clustering analysis can be used to generate clustered word sequences; then, structured hints can be constructed, denoted as preset structured hints:

[0162]

[0163] in, To establish an expert's identity, for example, "You are a professional zero-tolerance enterprise name review expert." To clarify the review objective, for example, "Please identify any errors in the meaning of the sentence." This indicates a complete sentence that is pending review. The k-th candidate quadruple contains the error text. Correct words ,level and similarity , To specify the output format, it is generally output in JSON structure.

[0164] For example, we can define a pre-defined structured prompt case, specifically: You are a professional expert in zero-tolerance review of Chinese unit and department names. Your core mission is to strictly review whether errors in the original text should be converted into unit names. You only perform replacements on errors that are 100% confirmed to be "functional organizational entity unit / department names." Non-unit names, non-functional entities, and content other than errors (including punctuation, tone, and other typos) are strictly prohibited. Your judgment is based on the ironclad rule of "better safe than sorry," ensuring that the output is completely consistent with the original text except for the possible differences in error replacements.

[0165] The presupposed large language model M is expressed as:

[0166]

[0167] in, To confirm the results, this indicates whether the large model agrees that the candidate word is a true error; Classify errors by type, such as errors based on similar form, similar sound, incorrect sequence, or incorrect keywords; For severity assessment, Provide detailed reasons and explanations for the basis of confirmation or rejection.

[0168] The differentiation confirmation strategy in this embodiment is based on level. Similarity configuration mainly refers to the calculated weighted similarity. Based on this strategy, the processing path for each word in the candidate word sequence can be determined. Specifically:

[0169] like and Confirm directly. This skips the call to the large language model. Level 4 is a mandatory protection statement; any similarity match that reaches its corresponding threshold is considered an error, without consuming model resources for confirmation.

[0170] like The system executes a pre-defined large language model for review and confirmation. The detection of L3-level professional terms and L2-level enterprise names requires contextual semantic judgment, such as the distinction between "organization" and "organization department," which is handled by the large model.

[0171] like and If the condition is met, it will be directly confirmed and marked as automatically approved, without calling the large language model for judgment.

[0172] like If so, mark it as pending manual review.

[0173] Errors in L1-level general terms have a relatively small impact; they are handled automatically at high confidence levels and switched to manual verification at low confidence levels, without calling the model at all. This differentiated verification strategy allows for focused review and verification of high-value and high-uncertainty L2 and L3 level scenarios, not only meeting review requirements but also significantly reducing model processing costs. Real-world testing shows that this differentiated verification strategy can reduce the number of model calls by more than 70% while maintaining the detection quality of key terms.

[0174] Step 105: Map the confirmation word sequence back to the target text sentence to generate a proofreading document.

[0175] The confirmation word sequence obtained in this embodiment needs to be mapped back to the target text sentence, or in other words, the original target document, so as to form a proofreading document with proofreading traces and complete positioning information, supporting accurate review and correction.

[0176] For each candidate word in the confirmation word sequence Both can be used to construct the output structure:

[0177]

[0178] The four levels of coordinates are calculated as follows:

[0179]

[0180] Document Identifier Typically, it is a globally unique number assigned to the task. .

[0181]

[0182]

[0183] Paragraph identifier Indicates the paragraph number where the candidate word is located. ,satisfy The position falls in paragraph Within the range.

[0184]

[0185]

[0186] Sentence identifier Indicates the sentence ordinal number of the candidate word. ,satisfy The position falls in the sentence Within the range.

[0187]

[0188] Character-level global start and end positions, among which For sentences The document-level starting position is i, where i is the starting offset of the candidate word in the sentence, and j-1 is the ending offset. Since the interval is left-closed and right-open, it is converted to a closed interval during output.

[0189] Other fields The complete sentence context facilitates human understanding of the language; t represents the detected erroneous text, the original fragment from the document; w represents the suggested corrected words, standard expressions obtained from a lexicon; meta represents the meta-information structure, containing levels. Original similarity and weighted similarity .

[0190] Severity Error text length Length of the main word The output sequence is sorted according to its document-level position, i.e. ; Generate a structured proofreading report.

[0191] The generated structured report supports applications such as highlighting and locating errors in a visual interface, one-click jump to review, and batch export of correction suggestions. It can be seen that this embodiment, through a reverse word matching strategy, maps the infinite error space to a finite word space, fundamentally solving the problem of incomplete coverage caused by the inability of traditional forward matching to exhaustively enumerate error variants; through dynamic window constraints and a multi-level index structure, the matching complexity is reduced from being related to the size m of the word library. Reduced to the sentence length n and average word length L Related This achieves near-linear efficiency improvements; through a length-adaptive weighted mechanism, it establishes a quantitative relationship between term importance and detection sensitivity, optimizing the balance between long word recall and short word precision, making the detection of long positive words carrying key information more sensitive and the detection of easily confused short positive words more cautious; through an equal-length character set weighted mechanism, it effectively identifies specific error patterns such as character order errors, improving the detection capability for input order reversal errors; through a long word expansion mechanism, it avoids missed detections caused by short word matching truncation, ensuring that the positive word level of inclusion relationships is completely identified; through feature index fallback matching, it fills the coverage blind spot of the first character index, achieving effective capture of variants with correct first characters but incorrect subsequent characters; through a hierarchical scheduling strategy for large model semantic confirmation, it concentrates high-cost calls on high-value, high-uncertainty L2 and L3 level scenarios, achieving the optimal trade-off between accuracy and cost, and in actual tests, it can reduce model call costs by more than 70%; through a four-level positioning system, it establishes a complete coordinate mapping from document to character, meeting the engineering needs of accurate review, and supporting one-click positioning and batch processing. This embodiment is particularly suitable for automated proofreading scenarios involving massive amounts of official documents in large corporations, achieving an order-of-magnitude increase in proofreading efficiency while ensuring security and compliance.

[0192] The document proofreading method based on word matching provided in this application replaces the forward matching operation based on the word database in the prior art by constructing a positive word database and a multi-level index to promote the reverse similarity matching mechanism of misspelled words. In the reverse similarity matching operation, the initial matching can be performed by the first character prefix index, and the feature character index can achieve the fallback matching, ensuring that the coverage of misspelled word matching is as large as possible, or even achieving full matching of errors. The input to the large language model is not the initial complete document information, but the candidate word sequence obtained by matching, which greatly reduces the information processing volume of the model. Moreover, the process of obtaining the candidate word sequence is controllable and adjustable, which can ensure the reduction of model cost and response latency. In addition, the text position information is introduced in the process of processing the target text sentence, so that the words in the candidate word sequence include attribute information such as word position, which can ensure the accuracy and reliability of error location, making subsequent document confirmation and proofreading more accurate. Therefore, the embodiments of this application can solve the problem of insufficient error coverage in the existing error dictionary; while direct proofreading of large models involves large computational load, high cost and latency, and is uncontrollable, and does not fully utilize document structure information, resulting in vague error location; thus causing the current document proofreading to lack timeliness, accuracy and reliability.

[0193] For easier understanding, please refer to Figure 2 This application provides an embodiment of a word-matching-based document proofreading device, comprising:

[0194] The lexicon building unit 201 is used to collect standard word expressions from various documents and build a standard lexicon, which includes basic vocabulary and special vocabulary.

[0195] Index building unit 202 is used to build first-character prefix index and feature character index for words in the positive lexicon respectively;

[0196] The reverse matching unit 203 is used to perform reverse similarity matching on the target text sentence sequence based on the positive vocabulary, the first character prefix index and the feature character index to obtain a candidate word sequence, which includes word position and weighted similarity.

[0197] The review and confirmation unit 204 is used to review and confirm the candidate word sequence based on level and similarity using a preset large language model to obtain a confirmed word sequence;

[0198] The text proofreading unit 205 is used to map the sequence of confirmation words back to the target text sentence and generate a proofreading document.

[0199] Furthermore, the reverse matching unit 203 is specifically used for:

[0200] Based on the first-character prefix index, the system searches for matching candidate words in the orthography database according to a certain character in the target text sentence, and generates an initial candidate orthography set.

[0201] The first candidate substring is extracted from the initial candidate positive word set through a preset dynamic window, and the original similarity is calculated based on the first candidate substring. The change of the preset dynamic window follows the dynamic length window constraint condition.

[0202] We introduce a weighted similarity mechanism based on equal-length characters and an adaptive length weighted similarity mechanism, and calculate the weighted similarity based on the original similarity.

[0203] Candidate word sequences with the first character are selected by combining weighted similarity and preset level thresholds;

[0204] The target text sentence is searched and matched a second time based on the feature character index. The substrings of equal length within the preset length range are extracted from the positive word library to obtain the second candidate substring.

[0205] Based on the second candidate substring, length matching, punctuation constraint analysis and similarity calculation are performed respectively, and feature candidate word sequences are selected by combining similarity thresholds;

[0206] A candidate word sequence is constructed by combining the first character candidate word sequence and the feature candidate word sequence.

[0207] This application also provides a word-matching-based document proofreading device, which includes a processor and a memory;

[0208] The memory is used to store program code and transfer the program code to the processor;

[0209] The processor is configured to execute the word-matching-based document proofreading method of any one of claims 1-7 according to instructions in the program code.

[0210] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0211] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0212] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0213] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for executing all or part of the steps of the methods described in the various embodiments of this application through a computer device (which may be a personal computer, server, or network device, etc.). The aforementioned storage medium includes: USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, optical disks, and other media capable of storing program code.

[0214] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A document proofreading method based on word matching, characterized in that, include: Standard vocabulary expressions are collected from various documents and a standard vocabulary database is constructed, which includes basic vocabulary and special vocabulary. Construct initial prefix indexes and feature character indexes for the words in the orthographic dictionary; Based on the positive lexicon, the initial prefix index, and the feature character index, reverse similarity matching is performed on the target text sentence sequence to obtain a candidate word sequence, which includes word position and weighted similarity. The candidate word sequence is reviewed and confirmed based on level and similarity using a pre-set large language model to obtain a confirmed word sequence; The confirmation word sequence is mapped back to the target text sentence to generate a proofreading document.

2. The document proofreading method based on word matching according to claim 1, characterized in that, The step of constructing a first-character prefix index and a feature character index for each word in the orthographic dictionary includes: A prefix index is created for the first character in the orthographic dictionary; Based on the prefix index, the words in the orthographic dictionary are classified to obtain a set of initial character categories; The first character classification set is sorted in descending order based on word length to obtain a descending first character sequence, thus completing the construction of the first character prefix index; Extract the feature characters corresponding to the positive words in the positive word library to generate a feature character set; An inverted mapping is established based on multiple feature characters in the feature character set to generate a feature character index.

3. The document proofreading method based on word matching according to claim 1, characterized in that, The reverse similarity matching of the target text sentence sequence based on the positive lexicon, the initial prefix index, and the feature character index yields a candidate word sequence, including: Based on the initial prefix index, a candidate word matching a certain character in the target text sentence is searched in the orthography database to generate an initial candidate orthography set; The first candidate substring is extracted from the initial candidate positive word set through a preset dynamic window, and the original similarity is calculated based on the first candidate substring. The change of the preset dynamic window follows the dynamic length window constraint condition. An equal-length character weighting mechanism and a length-adaptive weighting mechanism are introduced to calculate weighted similarity based on the original similarity. The candidate word sequences with the first character are selected by combining the weighted similarity and the preset level threshold; The target text sentence is searched and matched a second time based on the feature character index. A substring of equal length within a preset length range is extracted from the positive word library to obtain a second candidate substring. Based on the second candidate substring, length matching, punctuation constraint analysis and similarity calculation are performed respectively, and feature candidate word sequences are selected by combining similarity thresholds; The candidate word sequence is formed by combining the first character candidate word sequence and the feature candidate word sequence.

4. The document proofreading method based on word matching according to claim 1, characterized in that, The step of performing reverse similarity matching on the target text sentence sequence based on the positive lexicon, the initial prefix index, and the feature character index to obtain a candidate word sequence, before which the following steps are also included: The target text stream is parsed in a paragraph-level manner to obtain structured text, which includes paragraph identifiers, paragraph text content, paragraph length, and paragraph position. The structured text is robustly segmented using a preset segmentation mechanism to obtain the target text sentence.

5. The document proofreading method based on word matching according to claim 1, characterized in that, The step of performing reverse similarity matching on the target text sentence sequence based on the positive lexicon, the initial prefix index, and the feature character index to obtain a candidate word sequence, further includes: A greedy algorithm is used to perform deduplication and filtering on the candidate word sequence.

6. The document proofreading method based on word matching according to claim 1, characterized in that, The step of performing reverse similarity matching on the target text sentence sequence based on the positive lexicon, the initial prefix index, and the feature character index to obtain a candidate word sequence, further includes: A long word expansion mechanism is used to perform an extended screening analysis on the matching words in the candidate word sequence based on the original similarity.

7. The document proofreading method based on word matching according to claim 1, characterized in that, The candidate word sequence is reviewed and confirmed based on level and similarity using a preset large language model to obtain a confirmed word sequence, including: Cluster analysis is performed on the candidate word sequence to obtain a clustered word sequence; Differentiated confirmation strategies for review and approval based on level and similarity configuration; Based on the preset structured prompts and the differentiated confirmation strategy, the clustered word sequence is reviewed and confirmed using a preset large language model to obtain a confirmed word sequence.

8. A document proofreading device based on word matching, characterized in that, include: The lexicon building unit is used to collect standard word expressions from various documents and build a positive lexicon, which includes basic vocabulary and special vocabulary; The index building unit is used to build a first-character prefix index and a feature character index for the words in the orthographic dictionary, respectively; The reverse matching unit is used to perform reverse similarity matching on the target text sentence sequence based on the positive lexicon, the initial prefix index and the feature character index to obtain a candidate word sequence, wherein the candidate word sequence includes word position and weighted similarity. The review and confirmation unit is used to review and confirm the candidate word sequence based on level and similarity using a preset large language model to obtain a confirmed word sequence; The text proofreading unit is used to map the confirmation word sequence back to the target text sentence to generate a proofreading document.

9. The document proofreading device based on word matching according to claim 8, characterized in that, The reverse matching unit is specifically used for: Based on the initial prefix index, a candidate word matching a certain character in the target text sentence is searched in the orthography database to generate an initial candidate orthography set; The first candidate substring is extracted from the initial candidate positive word set through a preset dynamic window, and the original similarity is calculated based on the first candidate substring. The change of the preset dynamic window follows the dynamic length window constraint condition. An equal-length character weighting mechanism and a length-adaptive weighting mechanism are introduced to calculate weighted similarity based on the original similarity. The candidate word sequences with the first character are selected by combining the weighted similarity and the preset level threshold; The target text sentence is searched and matched a second time based on the feature character index. A substring of equal length within a preset length range is extracted from the positive word library to obtain a second candidate substring. Based on the second candidate substring, length matching, punctuation constraint analysis and similarity calculation are performed respectively, and feature candidate word sequences are selected by combining similarity thresholds; The candidate word sequence is formed by combining the first character candidate word sequence and the feature candidate word sequence.

10. A document proofreading device based on word matching, characterized in that, The device includes a processor and a memory; The memory is used to store program code and transmit the program code to the processor; The processor is configured to execute the word-matching-based document proofreading method according to any one of claims 1-7 according to instructions in the program code.

Images