Large model-based long text correction method, system, device and storage medium

Abstract

This invention relates to the field of large model technology, specifically providing a method, system, device, and storage medium for long text correction based on a large model. The method includes: receiving long text content to be corrected; performing preprocessing and adaptive segmentation on the long text content to obtain multiple compliant paragraphs; preparing context information for each compliant paragraph; combining each compliant paragraph and its context information with a predefined set of review rules as input to a large model; using the large model to obtain correction information for each compliant paragraph, wherein the correction information includes at least a judgment on whether correction is needed, a correction explanation, and text content before and after correction; accurately locating the text in the original long text based on the corrected text content in the correction information; and returning and displaying the processing result to the user. This invention significantly improves the efficiency and quality of long text correction.

Description

Technical Field

[0001] This invention belongs to the field of large model technology, specifically relating to a long text correction method, system, device and storage medium based on a large model. Background Technology

[0002] With the development of large-scale language-based text processing technologies, automatic text correction using large models has become an important means of improving text quality. Existing solutions for processing long texts generally segment the text into fragments based on fixed lengths or simple punctuation marks, then process each fragment individually using a large model. This method has significant drawbacks: mechanical segmentation easily damages the semantic integrity of sentences or paragraphs, affecting the accuracy of the large model's judgment; sequential processing of massive amounts of fragments leads to low efficiency and long user wait times; and it typically only returns the corrected text, lacking the basis for modification and precise location of the original text, which is detrimental to user understanding and review. Therefore, how to achieve efficient, accurate, and user-friendly intelligent correction of long texts is a problem that urgently needs to be solved in this field. Summary of the Invention

[0003] In view of the above-mentioned shortcomings of the prior art, the present invention provides a long text correction method, system, device and storage medium based on a large model to solve the above-mentioned technical problems.

[0004] In a first aspect, the present invention provides a long text correction method based on a large model, comprising:

[0005] Receive long text content to be corrected;

[0006] The long text content is preprocessed and adaptively segmented to obtain multiple compliant paragraphs;

[0007] Prepare context information for each of the compliant paragraphs;

[0008] Each compliant paragraph and its context information are combined with a predefined set of review rules as input to a large model. The large model is then used to obtain correction information for each compliant paragraph. The correction information includes at least a judgment on whether correction is required, a correction explanation, and the text content before and after correction.

[0009] Based on the corrected text content in the correction information, the error is accurately located in the original long text, and the processing result is returned and displayed to the user.

[0010] In one optional implementation, the long text content undergoes preprocessing and adaptive segmentation to obtain multiple compliant paragraphs, including:

[0011] The received long text content is formatted and cleaned to remove special characters that may interfere with subsequent processing;

[0012] The preprocessed long text content is initially segmented based on line breaks to obtain multiple preliminary paragraphs;

[0013] For initial paragraphs exceeding a preset threshold in length, a recursive segmentation algorithm based on priority delimiters is executed. This recursive segmentation algorithm includes:

[0014] Based on a preset priority order, candidate segmentation points are found in the initial paragraph using the delimiters in the delimiter list in turn;

[0015] For each candidate segmentation point, the length of the prefix content from the beginning of the paragraph to the segmentation point is evaluated, and the prefix content whose length does not exceed the threshold and is closest to the threshold is selected as the optimal segmentation segment.

[0016] Store the optimal segment as a compliant paragraph;

[0017] The remaining content after segmentation is treated as a new processing object, and the threshold judgment and segmentation are repeated until its length does not exceed the threshold, and it is stored as the last compliant paragraph.

[0018] If no matching split point is found after traversing all delimiters, the initial paragraph is forcibly split according to the threshold.

[0019] In one optional implementation, the long text content undergoes preprocessing and adaptive segmentation to obtain multiple compliant paragraphs, including:

[0020] The preprocessed long text content is initially segmented based on line breaks to obtain multiple preliminary paragraphs;

[0021] The initial paragraph is decomposed into multiple atomic units, wherein the atomic units are the smallest semantic segments identified based on separators or lightweight models that are more granular than periods.

[0022] Using atomic units as nodes, a graph structure representing the possibility of merging between adjacent atomic units is constructed, and a merging cost is assigned to each edge in the graph. The merging cost is calculated by weighting semantic coherence cost and length compliance cost.

[0023] Using dynamic programming, under the constraint that the final paragraph length does not exceed the preset threshold, we search for the atomic unit segmentation scheme that minimizes the total merging cost of all paragraphs.

[0024] Based on the optimal segmentation scheme obtained by the dynamic programming algorithm, the atomic units are aggregated into the final compliant paragraphs.

[0025] In an optional implementation, the initial segment is decomposed into a plurality of atomic units, including:

[0026] Syntactic analysis and semantic role labeling are performed on the preliminary paragraphs to identify predicates in the text and multiple semantic role components surrounding the predicates in order to extract preliminary semantic blocks;

[0027] Based on the dependency syntax tree, the initial semantic block is checked for integrity and then finely segmented according to preset segmentation rules. The segmentation rules include: segmentation based on the independence of the core predicate, ensuring that the words in the semantic block form a complete dependency subtree, and logical segmentation based on conjunctions or punctuation.

[0028] Output the sequence of atomic units obtained after verification and segmentation.

[0029] In an optional implementation, context information is prepared for each of the compliant paragraphs, including:

[0030] For the currently compliant paragraph to be processed, the context range parameters to be extracted are dynamically determined based on its own content and / or position attributes. The range parameters include the number of preceding paragraphs n and the number of following paragraphs m.

[0031] Based on the determined range parameters, extract the text content of the first n compliant paragraphs and / or the last m compliant paragraphs adjacent to the current unit from the sequence of the plurality of compliant paragraphs;

[0032] The extracted context text is associated and combined with the text of the currently compliant paragraph to be processed to form a processing unit containing the current paragraph and its associated context.

[0033] In an optional implementation, for the currently compliant paragraph to be processed, the context scope parameters to be extracted are dynamically determined based on its own content and / or location attributes, including:

[0034] If the current compliant sub-paragraph unit is a title, the start of a list item, or a semantically significant independence is detected, then the extraction of the forward context n is reduced or canceled.

[0035] If the current compliant sub-paragraph unit contains pronouns or omitted elements, increase the extraction range of the forward context n to cover possible referents;

[0036] If the content of the current compliant sub-paragraph is short or a transitional sentence, then the extraction range of its subsequent context m is increased to obtain more complete subsequent semantics.

[0037] In an optional implementation, based on the corrected text content in the correction information, precise location is performed in the original long text, and the processing result is returned and displayed to the user, including:

[0038] For each piece of correction information containing the corrected text content, a fuzzy matching algorithm is used to search in the original long text to calculate the similarity between the corrected text content and each substring in the original long text. The start and end position indices of the substring with the highest similarity in the original long text are determined as the precise positioning information corresponding to the correction result.

[0039] According to the order of the compliant sub-paragraph units in the original long text, the correction information and precise positioning information corresponding to each paragraph are assembled in a streaming manner and returned; in the user interface, the content to be modified in the original long text is highlighted based on the precise positioning information, and the corresponding correction explanation and comparison of the text content before and after the correction are displayed.

[0040] Secondly, the present invention provides a long text correction system based on a large model, comprising:

[0041] The text receiving module is used to receive long text content to be corrected;

[0042] The text segmentation module is used to preprocess and adaptively segment the long text content to obtain multiple compliant paragraphs;

[0043] The information preparation module is used to prepare context information for each of the compliant paragraphs;

[0044] The paragraph correction module is used to combine each compliant paragraph and its context information with a predefined set of review rules as input to a large model, and use the large model to obtain correction information for each compliant paragraph. The correction information includes at least a judgment on whether correction is needed, a correction explanation, and the text content before and after correction.

[0045] The text correction module is used to accurately locate errors in the original long text based on the corrected text content in the correction information, and return and display the processing results to the user.

[0046] Thirdly, a device is provided, comprising:

[0047] Memory is used to store long text correction procedures based on large models;

[0048] A processor is configured to implement the steps of the long text correction method based on a large model as provided in the first aspect when executing the long text correction procedure based on the large model.

[0049] Fourthly, a computer-readable storage medium is provided, on which a long text correction program based on a large model is stored, wherein when the long text correction program based on a large model is executed by a processor, the long text correction program based on a large model implements the steps of the long text correction method based on a large model provided in the first aspect.

[0050] The beneficial effects of this invention are as follows: the long text correction method, system, device, and storage medium based on a large model provided by this invention efficiently segment long texts while ensuring semantic integrity by introducing adaptive segmentation processing, and improve the accuracy of large model correction by combining contextual information. The concurrent processing mechanism significantly improves the overall throughput of long text processing and shortens user waiting time. Simultaneously, through precise positioning technology and the return of detailed correction explanations, the basis for modification is clear and transparent, and it supports streaming result display and highlighted comparison with the original text, greatly improving the efficiency and experience of user review and operation. Attached Figure Description

[0051] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0052] Figure 1 This is a schematic flowchart of a method according to an embodiment of the present invention.

[0053] Figure 2 This is a schematic block diagram of a system according to an embodiment of the present invention.

[0054] Figure 3 This is a schematic diagram of the structure of a device provided in an embodiment of the present invention. Detailed Implementation

[0055] To enable those skilled in the art to better understand the technical solutions of this invention, the technical solutions of the embodiments of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this invention, and not all embodiments. Based on the embodiments of this invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of this invention.

[0056] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. The terminology used herein in the description of the invention is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention.

[0057] The long text correction method based on a large model provided in this embodiment of the invention is executed by a computer device, and correspondingly, the long text correction system based on a large model runs on the computer device.

[0058] Figure 1 This is a schematic flowchart illustrating a method according to an embodiment of the present invention. Wherein, Figure 1 The implementing entity can be a long text correction system based on a large model. Depending on different requirements, the order of steps in this flowchart can be changed, and some can be omitted.

[0059] like Figure 1 As shown, the method includes:

[0060] S1. Receive the long text content to be corrected;

[0061] S2. Perform preprocessing and adaptive segmentation on the long text content to obtain multiple compliant paragraphs;

[0062] S3. Prepare contextual information for each of the compliant paragraphs;

[0063] S4. Combine each compliant paragraph and its context information with a predefined set of review rules as input to a large model, and use the large model to obtain correction information for each compliant paragraph. The correction information includes at least a judgment on whether correction is needed, a correction explanation, and the text content before and after correction.

[0064] S5. Based on the corrected text content in the correction information, accurately locate the error in the original long text, and return and display the processing result to the user.

[0065] In one embodiment of the present invention, based on step S1, the following will provide a possible embodiment and describe its specific implementation in a non-limiting manner.

[0066] Direct text input: Receive plain text strings submitted by the user directly through application programming interface (API), user interface (UI) text boxes, or command-line parameters.

[0067] File Upload Parsing: This function receives user-uploaded document files (such as .docx, .pdf, .txt, .md, etc.) and calls the corresponding document parser or library (such as python-docx, PyPDF2, pdfplumber) to parse the files and extract the plain text content. For example, for a .docx file, the parser will traverse the document paragraphs to extract text; for a .pdf file, the parser will identify and extract the text layer information within the page.

[0068] Web page or rich text processing: If the input is HTML or rich text with formatting, first use a relevant library (such as BeautifulSoup) to strip away non-text elements such as HTML tags and style code, and extract the core text content.

[0069] In one embodiment of the present invention, based on step S2, the following will provide a possible embodiment and describe its specific implementation in a non-limiting manner.

[0070] In one implementation, the long text content undergoes preprocessing and adaptive segmentation to obtain multiple compliant paragraphs, including:

[0071] 1. Text preprocessing (formatting and cleaning):

[0072] First, the received raw long text content is standardized. If the input is a file (such as PDF or Word), the corresponding parsing library (such as PyPDF2 or python-docx) is called to extract the plain text.

[0073] Next, a cleaning operation is performed: special characters that may interfere with subsequent character positioning and processing are removed, such as zero-width spaces (U+200B), invisible control characters, and redundant tab characters. Simultaneously, basic formatting can be performed, such as converting full-width punctuation to half-width.

[0074] 2. Preliminary segmentation based on newline characters:

[0075] The preprocessed plain text is segmented by newline characters (\n or \r\n) to obtain a series of preliminary paragraph units. This step is based on a fundamental assumption: newline characters have natural paragraph-separating semantics in most text formats and can form initial, relatively independent text blocks.

[0076] Recursive intelligent segmentation of extremely long paragraphs (core algorithm):

[0077] Iterate through all initial paragraph units and determine their length. A preset length threshold, chunk_size (e.g., 500 characters), is used.

[0078] For "extra-long segment units" exceeding the length of chunk_size, the following recursive segmentation algorithm is initiated instead of simple mechanical cutting:

[0079] a. Definition and Initialization: Predefine a list S of delimiters ordered by priority, for example, S=[“.”,“!”,“?”,“;”,“,”]. The period has the highest priority because it usually marks the end of a complete semantic unit.

[0080] b. Candidate split point search: Starting from the first delimiter (highest priority) in list S, search for all occurrences of that delimiter within the current super-long paragraph. Each occurrence is considered a candidate split point.

[0081] c. Optimal Prefix Selection: For each candidate split point generated by the current priority delimiter, calculate the length of the "prefix content" from the beginning of the paragraph to that point (including the delimiter). From all prefix contents with a length not exceeding chunk_size, select the one with the closest length to chunk_size as the "optimal prefix segment". If multiple candidates with the same length and the closest length exist, prioritize the segment split by the higher priority delimiter.

[0082] d. Paragraph Storage and Recursion: The selected "optimal prefix segment" is output and stored as a compliant paragraph. This segment ends with a complete delimiter, ensuring semantic closure.

[0083] e. Processing the remaining content: Treat the remaining suffix content after segmentation as a new text block and determine its length. If its length is still greater than chunk_size, treat it as a new "oversized paragraph unit" and jump to step b, using the same delimiter list S for the next round of recursive segmentation; if its length is less than or equal to chunk_size, store it directly as a compliant paragraph, and the current recursive branch ends.

[0084] f. Fallback: If, in the current recursive level, traversing the entire delimiter list S fails to find any split point that can produce prefix content with a length not exceeding chunk_size (e.g., encountering an extremely long sentence without punctuation), then an alternative is initiated: the text block is hard-cut according to the number of characters in chunk_size to ensure that the process continues.

[0085] 3. Results organization (page summary):

[0086] All compliant paragraphs generated in the above steps (including unsegmented short paragraphs and segmented sub-paragraphs) are organized sequentially. In a real system, to further optimize processing and display, a certain number of compliant paragraphs (e.g., ensuring their total character count does not exceed a higher page threshold) can be aggregated into a logical unit called a "page". This organization method facilitates subsequent concurrent scheduling and streaming result return, avoiding processing excessively large amounts of data at once.

[0087] Through the above implementation methods, this method achieves adaptive segmentation under strict length constraints, prioritizing segmentation at high-level semantic boundaries (such as periods), and making the length of each paragraph as close to the ideal value as possible, thereby maximizing the preservation of the semantic coherence of the text while meeting the length constraints of large model processing.

[0088] In another implementation, the long text content undergoes preprocessing and adaptive segmentation to obtain multiple compliant paragraphs, including:

[0089] 1. Text preprocessing and initial segmentation:

[0090] First, perform routine preprocessing on the input long text content (such as formatting and cleaning up interfering characters).

[0091] Subsequently, an initial coarse division is performed based on line breaks, segmenting the text into multiple "preliminary paragraphs." This step utilizes the paragraph boundary information inherent in the document format as the first level of structural division.

[0092] 2. Fine-grained atomic unit decomposition:

[0093] For each "preliminary paragraph," instead of simple segmentation based on punctuation, natural language processing techniques are used for deep analysis, breaking it down into more fundamental atomic units. The specific process is as follows:

[0094] a. Syntactic and Semantic Analysis: Invoke pre-trained natural language processing tools (such as Stanford CoreNLP, SpaCy, or corresponding Chinese tools) to perform dependency parsing and semantic role labeling on paragraphs simultaneously.

[0095] b. Semantic Block Extraction: Based on the semantic role labeling results, the core predicate (verb) in the sentence and its associated semantic role components (such as agent, patient, time, place adverbs, etc.) are identified. Each "predicate-role" combination and all its modifiers are initially extracted into a semantic block.

[0096] c. Integrity Verification and Fine-Grained Segmentation: Using the dependency syntax tree obtained in the previous step, structural verification is performed on the extracted semantic blocks. Necessary segmentation is performed according to preset rules to ensure that each atomic unit is a relatively complete subtree syntactically. Key rules include: mandatory segmentation when a semantic block contains an independent core predicate structure; ensuring that words within a unit are continuous in the dependency tree; and prioritizing segmentation at the detection of strong adversative conjunctions or specific punctuation marks to preserve logical relationships.

[0097] d. Output sequence: After the above processing, the original "preliminary paragraph" is transformed into a sequence of atomic units [AU1,AU2,...,AUn] composed of the smallest semantic fragments.

[0098] 3. Semantic Relationship Graph Construction and Cost Calculation:

[0099] The atomic unit sequence is modeled as a graph optimization problem.

[0100] a. Node: Each atomic unit AU_i in the sequence is a graph node, and the node attributes include its text content and length.

[0101] b. Edges and Costs: Consider merging consecutive atomic units AU_{i+1} to AU_j into the segment containing AU_i. Define a merging cost C(i,j) for each possible merging edge (i,j). This cost is a weighted sum of two terms:

[0102] Semantic coherence cost: This is measured by calculating the cosine similarity between the semantic vectors of AU_i and AU_j (e.g., generated using models like Sentence-BERT). The lower the similarity, the higher the cost. If the starting point of AU_j is identified as a strong semantic boundary in step 2c, an additional penalty cost is incurred.

[0103] Length compliance cost: Measures the deviation between the total length of the merged segments and the preset target length L. When the length does not exceed L, the cost is (L - length). 2 It encourages merging close to L; when the length exceeds L, the cost is set to a very large constant M, thus prohibiting this merging in optimization.

[0104] c. Objective: The objective is to find a way to divide atomic units such that the length of each segment (i.e., the final paragraph) does not exceed L, and the total cost of merging all segments is minimized.

[0105] 4. Dynamic Programming Optimal Search:

[0106] The above optimal partitioning problem can be solved efficiently using dynamic programming.

[0107] a. State definition: Let dp[j] represent the minimum cumulative cost of completing the optimal partitioning of the first j atomic units.

[0108] b. State transition:

[0109] ;

[0110] in, C(i,j) represents the text length of the fragment after merging the (i+1)th to the jth atomic unit. C(i,j) must be valid (i.e., fragment length ≤ L).

[0111] c. Calculation and backtracking: Calculate all values ​​of dp[j] from bottom to top and record the predecessor i when the minimum value is obtained. After the calculation is completed, backtracking from dp[n] will yield the globally optimal sequence of split points.

[0112] 5. Generating and outputting compliant paragraphs:

[0113] Based on the optimal splitting point obtained by backtracking through dynamic programming, the atomic unit sequence [AU1,AU2,...,AUn] is re-aggregated into several consecutive segments [P1,P2,...,Pk].

[0114] Each aggregated paragraph P_x is a compliant paragraph, and its length does not exceed the preset threshold L. It achieves an optimal balance between semantic coherence and length compliance at the algorithm level.

[0115] The beneficial effects of this implementation method are as follows: This method achieves sub-sentence level text understanding through advanced natural language processing technology, and uses graph models and dynamic programming for global optimization decisions. Thus, under the premise of strictly adhering to length constraints, it generates paragraph segmentation results with semantic integrity and coherence that are far superior to those based on simple punctuation or fixed rule segmentation, laying a high-quality input foundation for the accurate correction of subsequent large models.

[0116] In one embodiment of the present invention, based on step S3, the following will provide a possible embodiment and describe its specific implementation in a non-limiting manner.

[0117] S301. Adaptive context window determination:

[0118] For the currently compliant paragraph to be processed, the system does not extract a fixed number of paragraphs before and after it, but dynamically calculates the required number of preceding paragraphs n and following paragraphs m based on its content characteristics and positional attributes.

[0119] The specific adaptive rules include:

[0120] Context reduction: If the current compliant paragraph is a heading, a list start item, or is determined to have significant semantic independence (i.e., starting a new topic) through rule-based or lightweight model detection, the system will significantly reduce (e.g., set to 0 or 1) the value of its preceding context n. This avoids introducing irrelevant preceding content into the model and prevents interference with the judgment.

[0121] Forward context expansion: Using pronoun resolution tools or pattern matching (such as detecting words like "this," "its," and "the above"), if the analysis finds that the current paragraph contains explicit pronouns or omitting elements, the system will automatically increase the extraction range of the forward context n. The purpose is to ensure that the context submitted to the larger model includes the preceding text object to which the pronoun or omitted content points, thus ensuring the coherence of the model's understanding.

[0122] Backward context expansion: If the current paragraph is brief, or identified as a transitional or introductory sentence through conjunction analysis (such as "however," "next," "in short"), the system will correspondingly increase the extraction range of its backward context m. This aims to provide a more complete subsequent semantic background, helping the model predict the current paragraph's role in the overall logical flow of the text.

[0123] S302. Contextual content extraction:

[0124] Based on the n and m parameters dynamically determined by S301, the system accurately locates the current paragraph within the entire generated sequence of compliant paragraphs.

[0125] As instructed by the parameters, extract the text from the sequence that is immediately adjacent to the current paragraph and counts n paragraphs backward, and / or the text that is immediately adjacent to the current paragraph and counts m paragraphs backward. If a text boundary is reached (such as before the first paragraph or after the last paragraph), only the portion that actually exists is extracted.

[0126] S303. Processing Unit Assembly:

[0127] The complete text of the current paragraph, together with the preceding and following context texts extracted in step 2, are concatenated or structurally combined in the logical order of [previous text 1]...[previous text n]<current paragraph>[following text 1]...[following text m].

[0128] The assembled whole is treated as a complete processing unit, accompanied by metadata indicating the exact location of the current paragraph within this unit. This processing unit serves as the core input data for invoking the larger model for correction processing.

[0129] By using a content-aware dynamic window mechanism, each paragraph is intelligently adapted to a "just right" context. This avoids information redundancy or insufficiency caused by mechanically fixed windows, and accurately supplements the key background information (such as the referent) necessary to understand the current semantics. Thus, while improving the processing accuracy of large models, the overall scale of input data is effectively controlled, achieving the optimal balance between efficiency and effectiveness.

[0130] In one embodiment of the present invention, based on step S4, the following will provide a possible embodiment and describe its specific implementation in a non-limiting manner.

[0131] S401. Model Call Configuration and Input Assembly:

[0132] The system is configured to connect to a large language model service, such as by accessing a large model deployed in the cloud or locally via an API.

[0133] For each processing unit generated in step S3 (containing the current compliant paragraph and its dynamic context), the system combines it with a predefined set of review rules to construct the final model input. The set of review rules can include various configurable instructions such as syntax checking, punctuation standardization, terminology consistency, logical coherence, and factual verification.

[0134] S402. Engineering Prompt Design and Task Guidance:

[0135] The system uses a carefully designed structured Prompt template to guide large models. This template clearly defines the model's role, the structure of the input data, the specific tasks to be performed, and the output format that must be followed.

[0136] An example of a Prompt core is as follows:

[0137] "You are a professional text proofreading assistant. Please carefully analyze the following text paragraph enclosed in '###' ('current paragraph') and fully understand it in conjunction with its context. Context: {dynamically extracted context content inserted here};"

[0138] Current paragraph: {Insert the corrected paragraph text here};

[0139] Task:

[0140] Determine whether the above "current paragraph" violates the following proofreading rules: {Insert specific rules here, such as "Check and correct incorrect punctuation usage" or "Ensure technical terms are consistent with the context"}.

[0141] If no rules have been violated, please output: {"needs_correction": false}.

[0142] If a rule violation is found, please output a JSON object that must contain the following fields:

[0143] needs_correction: true,

[0144] correction_explanation: Provides a detailed explanation of the error type and the reason for the correction.

[0145] original_text: The original sentence or paragraph that needs to be modified, extracted from the "current paragraph".

[0146] `corrected_text`: The corrected complete sentence or fragment. Please ensure your output is only valid JSON.

[0147] Model Invocation and Raw Response Acquisition: Send the assembled complete Prompt to the large model API and receive the text response returned by the model.

[0148] Response parsing and correction information extraction: The system attempts to parse the text returned by the model into structured data (such as JSON).

[0149] Result determination: Based on the parsed needs_correction field, determine whether the paragraph needs correction.

[0150] Information extraction: If correction is required, the three core pieces of information, correction_explanation, original_text, and corrected_text, are accurately extracted from the parsed object to form the correction information of the paragraph.

[0151] Fault tolerance: If the model returns an incorrect format or cannot be parsed, the system will trigger a retry mechanism (such as a re-call) or a downgrade process (such as marking it as requiring manual review) to ensure the robustness of the process.

[0152] Through standardized Prompt engineering, the open-ended text generation task is transformed into a controllable, structured rule-based judgment and editing task, ensuring that the output of the large model strictly conforms to the format requirements of subsequent precise localization and display stages. Combined with dynamically prepared context, the model's ability to understand referents, tone, and logical relationships is significantly improved, resulting in more accurate correction judgments and more persuasive generated explanations.

[0153] In one embodiment of the present invention, based on step S5, a possible embodiment will be given below, and its specific implementation will be described in a non-limiting manner.

[0154] S501. Fuzzy matching for precise positioning:

[0155] For each entry that needs correction returned by the large model (including original_text and corrected_text), the system initiates a fuzzy search locator.

[0156] The core of this locator is to perform a global scan of the original long text using string similarity algorithms (such as the Levenshtein edit distance algorithm based on dynamic programming or the longest common subsequence algorithm).

[0157] Specific operation: The algorithm uses original_text (text before correction) as the target pattern and calculates its similarity with all possible consecutive substrings of the original long text. The similarity is quantified using formula 1 - (edit distance / max(pattern length, substring length)).

[0158] Location Determination: The algorithm finds the substring with the highest similarity to the original text and records the absolute start and end indices of this substring within the original long text. These two index values ​​constitute the precise location information for the corrected result. This step effectively overcomes the location failure problem caused by minor differences between the model output and the original text in spaces, line breaks, or individual punctuation marks.

[0159] Assembly and return of streaming results:

[0160] The processing results are organized and sent according to the natural order of all compliant paragraphs in the original text. A streaming technology is used, meaning that as soon as one or more paragraphs are processed (including model calls and location calculations), the results are immediately encapsulated into structured data blocks (such as JSON format) and returned to the client, without waiting for all processing to complete.

[0161] Each data block contains: paragraph number, original paragraph content, and all correction entries corresponding to that paragraph (each entry contains correction_explanation, original_text, corrected_text, and its start_index and end_index in the original text).

[0162] S502. Dynamic display of user interface:

[0163] After receiving the streamed data blocks, the client (such as a web application) dynamically updates the user interface.

[0164] Original text highlighting: In the area where the original long text is displayed, the interface marks the corresponding original text in real time and accurately with highlighting, underlining or different background colors according to the start_index and end_index of each correction information, intuitively indicating where modifications are needed.

[0165] Related Comparison Panel: When a user clicks or hovers over a highlighted area, a related details panel pops up. This panel is displayed side-by-side:

[0166] Correction Explanation: Clearly explain why the correction was made (error type, rule basis).

[0167] Text comparison: The specific changes are clearly displayed side by side or with highlights showing the differences between "before correction" and "after correction".

[0168] This presentation method allows users to receive and review documents simultaneously, and intuitively understand the context and reasons for each modification.

[0169] Robust alignment between the model output and the original text was achieved through a fuzzy matching algorithm, ensuring accurate positioning. Combined with streaming return and an interactive highlighting comparison interface, this not only significantly reduced the user's perceived waiting time but also provided a highly transparent, user-friendly, and smooth environment for reviewing proofreading results, greatly improving the method's practicality and user acceptance.

[0170] In some embodiments, the large-model-based long text correction system may include multiple functional modules composed of computer program segments. The computer programs for each program segment in the large-model-based long text correction system may be stored in the memory of a computer device and executed by at least one processor to perform (see details). Figure 1 (Description) Functionality for long text correction based on a large model.

[0171] In this embodiment, the long text correction system based on a large model can be divided into multiple functional modules according to the functions it performs, such as... Figure 2 As shown. The module referred to in this invention is a series of computer program segments that can be executed by at least one processor and perform a fixed function, and is stored in memory. In this embodiment, the functions of each module will be described in detail in subsequent embodiments.

[0172] The text receiving module is used to receive long text content to be corrected;

[0173] The text segmentation module is used to preprocess and adaptively segment the long text content to obtain multiple compliant paragraphs;

[0174] The information preparation module is used to prepare context information for each of the compliant paragraphs;

[0175] The paragraph correction module is used to combine each compliant paragraph and its context information with a predefined set of review rules as input to a large model, and use the large model to obtain correction information for each compliant paragraph. The correction information includes at least a judgment on whether correction is needed, a correction explanation, and the text content before and after correction.

[0176] The text correction module is used to accurately locate errors in the original long text based on the corrected text content in the correction information, and return and display the processing results to the user.

[0177] Figure 3The long text correction method based on a large model provided in this application embodiment can be applied to devices. Those skilled in the art will understand that the device structure involved in the embodiments of this invention does not constitute a limitation on the device; a device may include more or fewer components than illustrated, or combine certain components, or have different component arrangements. In the embodiments of this invention, the device includes, but is not limited to, laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the embodiments of this application described and / or claimed herein.

[0178] The device 300 may include a processor 310, a memory 320, and a communication unit 330. These components communicate via one or more buses. Those skilled in the art will understand that the server structure shown in the figure does not constitute a limitation of the present invention. It may be a bus topology or a star topology, and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0179] The memory 320 can be used to store execution instructions of the processor 310. The memory 320 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. When the execution instructions in the memory 320 are executed by the processor 310, the device 300 is able to perform some or all of the steps in the above method embodiments.

[0180] The processor 310 serves as the control center of the storage device, connecting various parts of the electronic device via various interfaces and lines. It executes software programs and / or modules stored in the memory 320, and calls data stored in the memory to perform various functions of the electronic device and / or process data. The processor can be composed of integrated circuits (ICs), such as a single packaged IC or multiple packaged ICs with the same or different functions connected together. For example, the processor 310 may consist only of a central processing unit (CPU). In this embodiment of the invention, the CPU may have a single processing core or include multiple processing cores.

[0181] The communication unit 330 is used to establish a communication channel, enabling the storage device to communicate with other devices. It can receive user data sent by other devices or send user data to other devices.

[0182] The present invention also provides a computer storage medium, wherein the computer storage medium may store a program, which, when executed, may include some or all of the steps provided in the embodiments of the present invention. The storage medium may be a magnetic disk, an optical disk, read-only memory (ROM), or random access memory (RAM), etc.

[0183] Those skilled in the art will clearly understand that the techniques in the embodiments of the present invention can be implemented using software plus necessary general-purpose hardware platforms. Based on this understanding, the technical solutions in the embodiments of the present invention, or the parts that contribute to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium such as a USB flash drive, a portable hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, or any other medium capable of storing program code. It includes several instructions to cause a computer device (which may be a personal computer, a server, or a second device, network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention.

[0184] The same or similar parts between the various embodiments in this specification can be referred to mutually. In particular, the device embodiments are basically similar to the method embodiments, so the description is relatively simple, and the relevant parts can be referred to the description in the method embodiments.

[0185] In the embodiments provided by this invention, it should be understood that the disclosed systems and methods can be implemented in other ways. For example, the system embodiments described above are merely illustrative. For instance, the division of modules is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple modules or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between systems or modules may be electrical, mechanical, or other forms.

[0186] The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical modules; that is, they may be located in one place or distributed across multiple network modules. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0187] In addition, the functional modules in the various embodiments of the present invention can be integrated into one processing module, or each module can exist physically separately, or two or more modules can be integrated into one module.

[0188] Although the present invention has been described in detail with reference to the accompanying drawings and preferred embodiments, the present invention is not limited thereto. Various equivalent modifications or substitutions can be made to the embodiments of the present invention by those skilled in the art without departing from the spirit and essence of the invention, and such modifications or substitutions should all be within the scope of the present invention. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should also be covered within the protection scope of the present invention.

Claims

1. A long text correction method based on a large model, characterized in that, include: Receive long text content to be corrected; The long text content is preprocessed and adaptively segmented to obtain multiple compliant paragraphs; Prepare context information for each of the compliant paragraphs; Each compliant paragraph and its context information are combined with a predefined set of review rules as input to a large model. The large model is then used to obtain correction information for each compliant paragraph. The correction information includes at least a judgment on whether correction is required, a correction explanation, and the text content before and after correction. Based on the corrected text content in the correction information, the error is accurately located in the original long text, and the processing result is returned and displayed to the user. The long text content is preprocessed and adaptively segmented to obtain multiple compliant paragraphs, including: The preprocessed long text content is initially segmented based on line breaks to obtain multiple preliminary paragraphs; The initial paragraph is decomposed into multiple atomic units, wherein the atomic units are the smallest semantic segments identified based on separators or lightweight models that are more granular than periods. Using atomic units as nodes, a graph structure representing the possibility of merging between adjacent atomic units is constructed, and a merging cost is assigned to each edge in the graph. The merging cost is calculated by weighting semantic coherence cost and length compliance cost. Using dynamic programming, under the constraint that the final paragraph length does not exceed a preset threshold, we search for the atomic unit segmentation scheme that minimizes the total merging cost of all paragraphs. Based on the optimal segmentation scheme obtained by the dynamic programming algorithm, the atomic units are aggregated into the final compliant paragraphs; The initial paragraph is decomposed into multiple atomic units, including: Syntactic analysis and semantic role labeling are performed on the preliminary paragraphs to identify predicates in the text and multiple semantic role components surrounding the predicates in order to extract preliminary semantic blocks; Based on the dependency syntax tree, the initial semantic block is checked for integrity and then finely segmented according to preset segmentation rules. The segmentation rules include: segmentation based on the independence of the core predicate, ensuring that the words in the semantic block form a complete dependency subtree, and logical segmentation based on conjunctions or punctuation. Output the sequence of atomic units obtained after verification and segmentation.

2. The method according to claim 1, characterized in that, The long text content is preprocessed and adaptively segmented to obtain multiple compliant paragraphs, including: The received long text content is formatted and cleaned to remove special characters that may interfere with subsequent processing; The preprocessed long text content is initially segmented based on line breaks to obtain multiple preliminary paragraphs; For initial paragraphs exceeding a preset threshold in length, a recursive segmentation algorithm based on priority delimiters is executed. This recursive segmentation algorithm includes: Based on a preset priority order, candidate segmentation points are found in the initial paragraph using the delimiters in the delimiter list in turn; For each candidate segmentation point, the length of the prefix content from the beginning of the paragraph to the segmentation point is evaluated, and the prefix content whose length does not exceed the threshold and is closest to the threshold is selected as the optimal segmentation segment. Store the optimal segment as a compliant paragraph; The remaining content after segmentation is treated as a new processing object, and the threshold judgment and segmentation are repeated until its length does not exceed the threshold, and it is stored as the last compliant paragraph. If no matching split point is found after traversing all delimiters, the initial paragraph is forcibly split according to the threshold.

3. The method according to claim 1, characterized in that, Prepare contextual information for each of the compliant paragraphs, including: For the currently compliant paragraph to be processed, the context range parameters to be extracted are dynamically determined based on its own content and / or position attributes. The range parameters include the number of preceding paragraphs n and the number of following paragraphs m. Based on the determined range parameters, extract the text content of the first n compliant paragraphs and / or the last m compliant paragraphs adjacent to the current unit from the sequence of the plurality of compliant paragraphs; The extracted context text is associated and combined with the text of the currently compliant paragraph to be processed to form a processing unit containing the current paragraph and its associated context.

4. The method according to claim 3, characterized in that, For the currently pending compliant paragraph, the contextual scope parameters to be extracted are dynamically determined based on its content and / or location attributes, including: If a compliant paragraph is a heading, the start of a list item, or is found to have significant semantic independence, then the extraction of the forward context n is reduced or canceled. If the compliant paragraph contains pronouns or omitted elements, the extraction scope of the forward context n is increased to cover possible referents; If the compliant paragraph is short or a transitional sentence, increase the extraction range of its subsequent context m to obtain more complete subsequent semantics.

5. The method according to claim 1, characterized in that, Based on the corrected text content in the correction information, precise location is performed in the original long text, and the processing result is returned and displayed to the user, including: For each piece of correction information containing the corrected text content, a fuzzy matching algorithm is used to search in the original long text to calculate the similarity between the corrected text content and each substring in the original long text. The start and end position indices of the substring with the highest similarity in the original long text are determined as the precise positioning information corresponding to the correction result. According to the order of the compliant paragraphs in the original long text, the correction information and precise positioning information corresponding to each paragraph are assembled in a streaming manner and returned; in the user interface, the content to be modified in the original long text is highlighted based on the precise positioning information, and the corresponding correction explanation and comparison of the text content before and after the correction are displayed.

6. A long text correction system based on a large model, characterized in that, include: The text receiving module is used to receive long text content to be corrected; The text segmentation module is used to preprocess and adaptively segment the long text content to obtain multiple compliant paragraphs; The information preparation module is used to prepare context information for each of the compliant paragraphs; The paragraph correction module is used to combine each compliant paragraph and its context information with a predefined set of review rules as input to a large model, and use the large model to obtain correction information for each compliant paragraph. The correction information includes at least a judgment on whether correction is needed, a correction explanation, and the text content before and after correction. The text correction module is used to accurately locate errors in the original long text based on the corrected text content in the correction information, and return and display the processing results to the user.

7. A long text correction device based on a large model, characterized in that, include: Memory is used to store long text correction procedures based on large models; A processor, configured to implement the steps of the long text correction method based on a large model as described in any one of claims 1-5 when executing the long text correction procedure based on a large model.

8. A computer-readable storage medium storing a computer program, characterized in that, The readable storage medium stores a long text correction program based on a large model, which, when executed by a processor, implements the steps of the long text correction method based on a large model as described in any one of claims 1-5.

Images