File verification method, system and device, readable storage medium and processor

Abstract

The invention discloses a file verification method, system and device, a readable storage medium and a processor. The method comprises the following steps: in response to a file verification instruction, obtaining a to-be-verified file indicated by the file verification instruction; key text information in the to-be-verified file is extracted, and the key text information is used for representing text content in the to-be-verified file; the key text information is used for conducting format conversion on the to-be-verified file to obtain a target file, the file format of the target file is a plain text format, and the plain text format is different from the initial file format of the to-be-verified file; checking operation is executed on the target file, a checking result is obtained, and the checking result is used for representing the integrity degree of the text content in the target file. The technical problem of low document verification efficiency is solved.

Description

Technical Field

[0001] This application relates to the field of document processing, and more specifically, to a document verification method, system, apparatus, readable storage medium, and processor. Background Technology

[0002] Currently, document review typically relies on manual or automated methods. However, the results of manual review are easily influenced by the reviewer's experience and attention, leading to low efficiency. Automated review, on the other hand, may encounter documents with excessive redundant information, such as formatting and style, causing automated review to malfunction and resulting in low document verification efficiency.

[0003] There is currently no effective solution to the above problems. Summary of the Invention

[0004] This application provides a document verification method, system, apparatus, readable storage medium, and processor to at least solve the technical problem of low document verification efficiency.

[0005] According to one aspect of the embodiments of this application, a file verification method is provided. The method may include: in response to a file verification instruction, obtaining a file to be verified indicated by the file verification instruction; extracting key text information from the file to be verified, wherein the key text information is used to characterize the text content in the file to be verified; using the key text information to perform format conversion on the file to be verified to obtain a target file, wherein the target file is in plain text format, and the plain text format is different from the initial file format of the file to be verified; performing a verification operation on the target file to obtain a verification result, wherein the verification result is used to characterize the completeness of the text content in the target file.

[0006] Furthermore, key text information is extracted from the file to be verified, including: removing redundant information from the file to be verified to obtain a simplified file; and extracting key text information from the simplified document.

[0007] Furthermore, extracting key text information from the simplified document includes: extracting at least one object information from at least one line of string in the simplified document; and using a preset key, traversing at least one object information to obtain the key text information.

[0008] Further, a verification operation is performed on the target file to obtain the verification result, including: inputting the target file into a language model for verification and obtaining the verification result, wherein the language model is used to verify the target file based on a preset verification standard.

[0009] Furthermore, the method may also include: obtaining an initial document generated by a text editor in the front-end module; and converting the initial document into a file to be verified according to a preset format.

[0010] According to another aspect of the embodiments of this application, a file verification system is also provided. This system may include: a front-end module for generating an initial document in response to an input instruction; a back-end service module for converting the initial document into a file to be verified according to a preset format and storing it; a conversion service module for obtaining the file to be verified indicated by the file verification instruction in response to receiving a file verification instruction; extracting key text information from the file to be verified, wherein the key text information is used to characterize the text content in the file to be verified; using the key text information to perform format conversion on the file to be verified to obtain a target file, wherein the target file's file format is plain text, and the plain text format is different from the initial file format of the file to be verified; and a verification service module for performing verification operations on the target file to obtain a verification result, wherein the verification result is used to characterize the completeness of the text content in the target file.

[0011] According to another aspect of the embodiments of this application, a file verification device is also provided. The device may include: an acquisition unit, configured to acquire a file to be verified indicated by a file verification instruction in response to a file verification instruction; an extraction unit, configured to extract key text information from the file to be verified, wherein the key text information is used to characterize the text content in the file to be verified; a conversion unit, configured to use the key text information to perform format conversion on the file to be verified to obtain a target file, wherein the file format of the target file is plain text format, and the plain text format is different from the initial file format of the file to be verified; and a verification unit, configured to perform a verification operation on the target file to obtain a verification result, wherein the verification result is used to characterize the completeness of the text content in the target file.

[0012] According to another aspect of the embodiments of this application, an electronic device is also provided, including: a memory storing an executable program; and a processor for running the program, wherein the program executes the method of any one of the above when it runs.

[0013] According to another aspect of the embodiments of this application, a computer-readable storage medium is also provided, the computer-readable storage medium including a stored executable program, wherein, when the executable program is running, it controls the device where the computer-readable storage medium is located to perform the methods of various embodiments of this application.

[0014] According to another aspect of the embodiments of this application, a computer program product is also provided, including a computer program that, when executed by a processor, implements the methods of various embodiments of this application.

[0015] According to another aspect of the embodiments of this application, a computer program product is also provided, including a non-volatile computer-readable storage medium storing a computer program, which, when executed by a processor, implements the methods in various embodiments of this application.

[0016] According to another aspect of the embodiments of this application, a computer program is also provided, which, when executed by a processor, implements the methods of the various embodiments of this application.

[0017] In this embodiment, in response to a file verification command, the file to be verified as indicated by the command is obtained; key text information is extracted from the file to be verified, wherein the key text information is used to characterize the text content in the file to be verified; the file to be verified is converted into a target file using the key text information, wherein the target file is in plain text format, which differs from the initial file format of the file to be verified; a verification operation is performed on the target file to obtain a verification result, wherein the verification result is used to characterize the completeness of the text content in the target file. In other words, in this embodiment, the key text information in the file to be verified is used to convert the file to be verified into a target file; the target file obtained after format conversion is verified to obtain a verification result, which can be used to determine the completeness of the text content in the target file, thereby achieving the technical effect of improving document proofreading efficiency and solving the technical problem of low document verification efficiency. Attached Figure Description

[0018] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments of this application and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0019] Figure 1 This is a flowchart of a document verification method according to an embodiment of this application;

[0020] Figure 2 This is a schematic diagram of an optional document verification system according to an embodiment of this application;

[0021] Figure 3 This is a schematic diagram of an optional document verification device according to an embodiment of this application. Detailed Implementation

[0022] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present application.

[0023] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0024] According to an embodiment of this application, an embodiment of a file verification method is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0025] Figure 1 This is a flowchart of a document verification method according to an embodiment of this application, such as... Figure 1 As shown, the method may include the following steps.

[0026] Step S102: In response to the file verification command, obtain the file to be verified as indicated by the file verification command.

[0027] In the technical solution provided by step S102 of the embodiments of this application, the file verification instruction can be a signal requesting verification from a user or a file verification system (hereinafter referred to as the system), which can be used to request verification of a file. This verification can refer to reviewing the problems existing in the file. The file to be verified can be used to represent the document that needs to be verified. It can be a JSONLines (JSONL) file, or a file generated by an editor that interacts with the user and obtained after specific conversion processing.

[0028] Optionally, the document review process can be initiated by the aforementioned document verification command. This command can be generated by user interaction, such as triggering a review task after a user submits a document to be verified, or it can be triggered periodically by the document verification system. The document to be verified can be stored in various formats, such as JSONL.

[0029] Optionally, verification can be triggered manually. For example, after editing and saving a document, the user can manually send a file verification command by clicking the "Review" button on the interface or using a shortcut key. Alternatively, verification can be performed automatically and periodically. For instance, the document review system can be set up with a scheduled task to automatically send file verification commands to documents stored in the system at regular intervals, thus triggering the verification of the files to be checked.

[0030] Optionally, analyzing the received file verification instruction may include: analyzing the information in the received file verification instruction, determining the path to the target document contained in the information in the file verification instruction, so as to locate the document that needs to be reviewed.

[0031] By following the steps above, it can be ensured that the file verification system responds accurately to the review requirements. That is, the file verification system can accurately find the corresponding file to be verified in the storage system based on the information in the received file verification instruction.

[0032] Step S104: Extract key text information from the file to be verified, wherein the key text information is used to characterize the text content in the file to be verified.

[0033] In the technical solution provided by step S104 in the embodiments of this application, the key text information can be used to represent the text content in the file to be verified, and can be key structural elements, including but not limited to: titles and subtitles, body text, lists and bullet points, etc.

[0034] Optionally, after obtaining the file to be verified as indicated by the file verification command, key text information in the file to be verified can be extracted.

[0035] Optionally, the text content in the file to be verified may include: document title and subtitle, paragraph text, lists and bullets, text content in tables, quoted content, code blocks or programming-related text, comments and footnotes.

[0036] Optionally, if the text to be verified is a JSONL file, extracting key text information from the file may include: reading each line of JSONL data to find specific fields that represent the text content; identifying and filtering out non-text information, such as style, color, font size and other format attributes; and extracting text elements, including but not limited to headings, paragraphs, lists, quotes, code blocks and so on.

[0037] By following the steps above, the efficiency and accuracy of document review can be significantly improved. Non-textual information, such as styles and colors, is removed from the original JSONL file to reduce the amount of data, while preserving the core structure of the file, such as headings, lists, and code blocks.

[0038] Step S106: Using key text information, the format of the file to be verified is converted to obtain the target file. The target file is in plain text format, which is different from the initial file format of the file to be verified.

[0039] In the technical solution provided by step S106 of this application embodiment, the target file can be used to represent a file after format conversion. It can be a plain text file, such as a Markdown (Markdown) format file. The file format can be used to represent the specific storage and presentation form of the file at different stages. The initial file format can be JSONL format.

[0040] Optionally, after extracting the key text information from the file to be verified, the key text information can be used to convert the format of the file to be verified to obtain the target file.

[0041] Optionally, the target file can be in a markup syntax format. This target file retains the structure and content of the file to be verified, but removes redundant formatting information and metadata.

[0042] Optionally, key text information in the document to be validated, such as headings, paragraphs, lists, tables, quotations, and code blocks, can be mapped to their equivalent representations in Markdown format to obtain the target file. For example, headings... <h1>Convert to # to convert an unordered list to - or Etc.; Extract the actual text content from JSONL; Identify and remove style tags and attributes contained in JSONL, such as font, color, alignment, etc., and retain only the pure text and structural information.

[0043] Optionally, the reconstructed Markdown text content can be saved as a target file, which is a new file relative to the file to be validated. This target file can be in plain text format, containing only the core structure and text information of the initial file. It differs significantly in form from the original JSONL format file, but retains the essential content of the file.

[0044] By following the steps above, the format information of the file to be verified can be removed, while retaining the key structure of the file, such as titles, lists, and references. This allows for an accurate grasp of the file's logic and framework during review, thereby enhancing the depth and comprehensiveness of the review.

[0045] Step S108: Perform a verification operation on the target file to obtain the verification result, wherein the verification result is used to characterize the integrity of the text content in the target file.

[0046] Optionally, after obtaining the target file, a verification operation can be performed on the target file to obtain the verification result.

[0047] Optionally, the above verification results may include: evaluating whether the file covers the necessary parts and information, checking whether the logical structure and information flow of the file are clear, and evaluating the grammatical correctness and spelling accuracy of the file.

[0048] Optionally, performing a validation operation on the target file and obtaining the validation result may include: using the format-converted Markdown target file as input to a pre-configured deep learning model, and using the pre-configured deep learning model to perform deep analysis and validation on the target file. The model evaluates the text content based on pre-trained knowledge and rules.

[0049] Alternatively, by converting redundant JSONL files to a streamlined Markdown format, the amount of data required for review is significantly reduced, enabling large language models to efficiently review and score document content. The converted Markdown file contains only about 5% of the tokens of the original file, allowing even long documents to be processed smoothly by large language models and effectively avoiding token limitation issues.

[0050] The embodiments of this application can ensure the normal display and editing of rich text documents, and can also realize automated document quality review based on a large language model in the background, thus achieving a high-efficiency combination of user-friendliness and system efficiency.

[0051] Through steps S102 to S108, in response to a file verification command, the file to be verified as indicated by the file verification command is obtained; key text information is extracted from the file to be verified, wherein the key text information is used to characterize the text content in the file to be verified; using the key text information, the file to be verified is format-converted to obtain a target file, wherein the target file is in plain text format, which is different from the initial file format of the file to be verified; a verification operation is performed on the target file to obtain a verification result, wherein the verification result is used to characterize the completeness of the text content in the target file. In other words, in this embodiment, based on the key text information in the file to be verified, a format conversion is performed; the target file obtained after format conversion is verified to obtain a verification result, thereby achieving the technical effect of improving document proofreading efficiency and solving the technical problem of low document verification efficiency.

[0052] The embodiments of this application will be described in detail below with reference to the steps described above.

[0053] As an optional implementation method, extracting key text information from the file to be verified includes: removing redundant information from the file to be verified to obtain a simplified file; and extracting key text information from the simplified file.

[0054] In this embodiment, the aforementioned redundant information can be used to represent redundant data elements during document review, such as format styles, document creation time, modification time, and other metadata, as well as comments and annotations. The aforementioned simplified file can be used to represent the file version after redundant information has been removed, and may include, but is not limited to, the core structure and text content of the file to be verified.

[0055] Optionally, removing redundant information from the file to be verified to obtain a simplified file may include: identifying various data elements in the JSONL file, removing format-related data such as font, color, and alignment; filtering out metadata such as file creation time, modification time, author information, and version number; ignoring comments and annotations in the document; and retaining the basic structural information of the file, such as titles, lists, and references.

[0056] Optionally, extracting key text information from a simplified document may include: identifying headings and paragraphs in the simplified document and extracting the heading text and paragraph content; for structures such as lists and quotations, extracting the specific text of each item. If the simplified document contains code snippets, these code snippets can be isolated from the text and represented using specific Markdown syntax. For data tables and chart descriptions, they can be converted into equivalent Markdown representations.

[0057] Optionally, the file to be verified needs to be simplified. This simplification process can intelligently discard redundant formatting and style information in the file, retaining only the core structure and text content. Therefore, redundant information in the file to be verified can be removed to obtain a simplified file, from which key text information can be extracted, thereby improving the efficiency of information retrieval.

[0058] For example, a header in the file to be verified<h1style="..."> title< / h1> JSONL may contain complex JSON structures. These structures can be removed to obtain a simplified document, i.e., the converted #heading. Extracting the simplified document reveals the key text information: the heading.

[0059] In this embodiment, the above method can remove redundant information, simplify the size and complexity of the file to be verified, while preserving the key content and structural information (such as titles, lists, and references) of the file. The evaluation results will not be affected by missing information.

[0060] As an optional implementation, extracting key text information from a simplified document includes: extracting at least one object information from at least one line of string in the simplified document; and using a preset key, traversing at least one object information to obtain the key text information.

[0061] In this embodiment, the aforementioned string can be used to represent one or more lines of text content in Markdown format. The aforementioned object information can be used to represent specific text elements contained in the simplified document, such as headings, paragraph content, etc., and can be a JSON object. The aforementioned preset key can be used to represent specific characters or patterns used to identify and locate various text elements in the simplified document; for example, it can be a "type key" and a "content key".

[0062] Optionally, extracting at least one object information from at least one line of a simplified document may include: parsing the simplified Markdown document line by line, where each line may contain one or more object information, such as headings, paragraphs, list items, etc.; and using regular expressions or other pattern matching techniques to identify specific Markdown syntax patterns in each line of the document. In this embodiment, the object information may be a JSON object.

[0063] For example, you can convert each line of a simplified file into a JSON object. This can be done using a JSON parsing library, such as the JSON library in programming languages ​​(like Python) or the built-in JSON.parse() function in scripting languages ​​(like JavaScript).

[0064] Optionally, after converting each line of the Markdown string into a JSON object, each line will correspond to an entry in the JSON. A set of preset keys can be defined to locate and categorize different text elements within the JSON object. Each JSON object can be iterated through using these preset keys. During iteration, key text information can be further extracted and summarized based on the object's type and content.

[0065] For example, after parsing a JSON object, you can iterate through it to find the text content and other information you want to retain, obtaining key text information. You can extract the text content by accessing specific keys in the JSON object. For instance, for a heading element, you can look for "type":"heading" and "content":"heading text"; for a list item, you can look for "type":"listItem" and "content":"list item text", and so on.

[0066] In this embodiment, the above method can extract key text information from simplified documents, providing a precise and efficient data foundation for subsequent automated document review. By using pattern matching techniques such as regular expressions, different text elements in Markdown format are accurately identified, and the identified text elements are converted into JSON format. Preset keys (such as "type" and "content") are used to classify and store information, thus structuring and digitizing the document content.

[0067] As an optional implementation, performing a verification operation on the target file to obtain a verification result includes: inputting the target file into a language model for verification and obtaining a verification result, wherein the language model is used to verify the target file based on a preset verification standard.

[0068] In this embodiment, the language model described above can be used to represent a pre-trained large-scale language model (LLM). The verification criteria described above can be used to represent a set of predefined rules and indicators to guide and constrain the verification process of the large language model on the target file. These can be preset review criteria, such as document completeness, logical clarity, and professionalism.

[0069] Optionally, the target file can be input into a language model for validation. The validation result can include: sending a request to the LLM via a data interface to initiate the validation process. The LLM processes the target file internally and applies predefined validation criteria to evaluate it. After the model finishes processing, a validation report can be extracted from the model's returned results. The model's returned results can be a structured text describing whether various aspects of the file meet the validation criteria, including scores, issue identifiers, and suggested modifications.

[0070] Optionally, the language model is used to validate the target file based on preset validation criteria. This validation can include: the LLM first understanding all the text of the input file, including its syntax, context, and logical structure. The model then checks each aspect of the file according to the preset validation criteria. The LLM can detect parts of the document that do not conform to the validation criteria, such as missing key information, contradictory statements, and incorrect grammatical structures, and accurately mark the locations of these problems for subsequent repair and optimization.

[0071] For example, the LLM evaluates the Markdown text content, such as determining whether the Markdown file is structurally clear, logically complete, and free of typos, and assigns a score from 1 to 5. The review service returns the score (e.g., 4.5 points) and the LLM's suggestions to the backend and links them to the original document. Users can see the document's quality score and the LLM's improvement suggestions on the document's details page, enabling them to optimize the document. This method allows for efficient and automated control of document quality, providing continuous feedback and knowledge accumulation for the development team.

[0072] Optionally, during automated review, the raw JSONL file is not sent directly to the large language model. Instead, a crucial transformation step is performed: the core text content of the JSONL file is extracted and converted into a Markdown target file. The converted target file is then validated using the language model to obtain the validation result.

[0073] Optionally, after conversion, the document verification system takes the simplified Markdown file as input and sends it to the large language model for content quality review. Because the number of tokens in the Markdown file is significantly reduced (only about 5% of the number in the file to be verified), not only is the processing cost significantly lowered, but the issue of exceeding token limits is also avoided. The large language model can score and evaluate the Markdown file according to preset review criteria (such as document completeness, logical clarity, and professionalism) and return the results to the document verification system. The document verification system associates the review results with the file to be verified, providing a basis for subsequent quality management and optimization. This method, through a separate "storage-conversion-review" design, achieves efficient and automated quality review of documents using the large language model without sacrificing the front-end user experience.

[0074] In this embodiment, the above method enables a deep understanding and accurate evaluation of the target text content, improving the efficiency and quality of document verification. LLM (Literature Logic Model) can deeply understand the semantics of a document, including its syntax, context, and logical structure. By using predefined verification standards, LLM can automatically detect and locate problems in the document without requiring manual item-by-item checks, greatly saving time and manpower costs and improving verification efficiency.

[0075] As an optional implementation, the method further includes: obtaining an initial document generated by a text editor in the front-end module; and converting the initial document into a file to be verified according to a preset format.

[0076] In this embodiment, the aforementioned front-end module can represent an integrated user interface component that allows users to input, edit, and preview document content. The aforementioned Chinese text editor can represent a rich text editing tool that allows users to intuitively edit and format documents, including inserting images, tables, and links, setting font styles, sizes, and colors, and creating headings, lists, etc. The aforementioned initial document can represent the original document generated by the text editor, containing formatting information and content. The aforementioned preset format can represent a simplified, standardized document format, such as Markdown.

[0077] Optionally, the front-end module continuously monitors user actions in the text editor, including text input, formatting adjustments, and element additions. When the user makes changes in the editor, the front-end module updates the initial document's state to ensure its real-time performance and accuracy. The front-end module then sends the generated initial document to the back-end system via a network request for further processing and storage.

[0078] Optionally, the backend service may include: responsible for receiving and storing JSONL formatted documents submitted by the frontend.

[0079] Optionally, after receiving the initial JSONL document from the frontend, the backend uses a JSON parsing library or a custom parser to parse the document into an operable data structure containing the document's elements. It then iterates through the parsed data structure to extract the core text content, including headings, paragraphs, list items, code blocks, etc., ignoring or removing redundant formatting information and metadata. The extracted text content is then reorganized and formatted according to Markdown syntax; for example, headings are converted to corresponding Markdown heading tags, and list items are converted to Markdown list syntax. After mapping, the Markdown-formatted text content is saved as a file to be validated.

[0080] Alternatively, business users can create and edit documents using an online rich text editor (e.g., a Chinese text editor). This editor stores the document content as a plain text file in JSONL format, i.e., the initial document. While the JSONL format is convenient for storage and transmission, it contains a large amount of formatting information (e.g., fonts, colors, alignment), hierarchical structure, and metadata.

[0081] In this embodiment, the above method can, in response to a file verification command, obtain the file to be verified as indicated by the file verification command; extract key text information from the file to be verified, wherein the key text information is used to characterize the text content in the file to be verified; use the key text information to perform format conversion on the file to be verified to obtain a target file, wherein the target file is in plain text format, which is different from the initial file format of the file to be verified; perform a verification operation on the target file to obtain a verification result, wherein the verification result is used to characterize the completeness of the text content in the target file. In other words, in this embodiment, based on the key text information in the file to be verified, format conversion is performed; the target file obtained after format conversion is verified to obtain a verification result, thereby achieving the technical effect of improving document proofreading efficiency and solving the technical problem of low document verification efficiency.

[0082] The technical solutions of the embodiments of this application will be illustrated below with reference to preferred embodiments.

[0083] Currently, in modern software development processes, document systems are widely used for knowledge accumulation and experience sharing, aiming to accelerate the development process and improve product quality. However, existing technologies have significant shortcomings in automating document review and quality assessment. Traditional review methods typically rely on manual labor, which is inefficient and highly subjective, making large-scale implementation difficult. With the rapid development of large language models, automated document review has become possible. However, directly inputting raw documents (e.g., JSONL files generated by rich text editors) into large language models for review encounters two main problems: data redundancy and information overload. For example, raw JSONL files contain a large amount of formatting, style, and metadata information, which is redundant for understanding the document content and quality review; exceeding token limits, for example, redundant formatting information results in a very large number of tokens in the raw file, easily exceeding the token limit that large language models can handle, making review impossible.

[0084] The embodiments of this application aim to solve the problem that existing large language models cannot efficiently and automatically review rich text documents with excessive redundant information.

[0085] The beneficial effects of this application's embodiments include: significantly improved review efficiency, effective avoidance of token restrictions, and a balance between user experience and automated review. Specifically, the significantly improved review efficiency includes greatly reducing the amount of data required for review by converting redundant JSONL format files into a streamlined Markdown format, enabling large language models to efficiently review and score document content.

[0086] Optionally, effectively circumventing token restrictions includes: the number of tokens in the converted Markdown file is only about 5% of that in the original file, allowing even long documents to be processed smoothly by the large language model, effectively avoiding the token restriction problem. Balancing user experience and automated review includes: the embodiments of this application can both ensure that business personnel can normally view and edit the rendered rich text documents, and achieve automated, model-based document quality review in the background, achieving an effective combination of user-friendliness and system efficiency.

[0087] This application proposes a document storage review method and system. The core technical solution includes constructing an efficient and concise document representation for large language model review without affecting users' normal editing and viewing of rich text. The specific process includes: online creation and editing, automated review process, data simplification, large model review, and scoring and feedback.

[0088] Optionally, online creation and editing can include business users creating and editing documents using an online rich text editor. This editor stores the document content as a plain text file in JSONL format. This format is convenient for storage and transmission, but it contains a large amount of formatting information (such as fonts, colors, and alignment), hierarchical structure, and metadata.

[0089] Optionally, normal viewing and rendering may include: when business users view the document, the system reads data from the JSONL file and renders the data into a user-friendly rich text format to ensure a normal reading experience.

[0090] Optionally, the automated review process may include: during automated review, the system does not directly send the raw JSONL file to the large language model. Instead, it first performs a crucial transformation step: extracting the core text content from the JSONL file and converting it into Markdown format.

[0091] Optionally, data simplification may include intelligently discarding redundant formatting and style information in JSONL files, retaining only the core structure and text content of the document. For example, a heading<h1style="..."> The header in JSONL may contain a complex JSON structure, but after conversion it will only become a # header.

[0092] Optionally, the large model review may include: after conversion, the system sends the simplified Markdown file as input to the large language model for content quality review. Because the number of tokens in the Markdown file is significantly reduced (only about 5% of the original file), this not only significantly reduces processing costs but also avoids exceeding token limits.

[0093] Optionally, scoring and feedback may include: the large language model scoring and evaluating the Markdown file based on preset review criteria (such as document completeness, logical clarity, and professionalism), and returning the results to the system. The system then associates the review results with the original document, providing a basis for subsequent quality management and optimization.

[0094] In the embodiments of this application, the above method achieves efficient and automated quality review of documents using a large language model without sacrificing the front-end user experience through a separate design of "storage-conversion-review".

[0095] The specific implementation of the embodiments of this application can be achieved by combining a document system platform.

[0096] Optionally, the system architecture may include: a front-end module, a back-end service, a conversion service, and an LLM review service. The front-end module may include a UI interface based on a rich text editor, responsible for the creation, editing, and rendering of user documents.

[0097] Optionally, the backend service may include receiving and storing JSONL format documents submitted by the frontend. The conversion service may include a separate microservice or module specifically responsible for converting the stored JSONL files into Markdown format. The above services need to implement an efficient parsing and conversion algorithm capable of accurately extracting text content and mapping it to the corresponding Markdown syntax (e.g., headings, lists, code blocks, quotes, etc.).

[0098] Optionally, the LLM review service may include an API call that encapsulates a large language model, receives Markdown-formatted text, and returns review results (such as scores, modification suggestions, etc.) according to preset review rules.

[0099] In this embodiment, the above method automates, simplifies, and enhances the document review process. The document verification system automatically triggers the review process without manual intervention. The Large Language Model (LLM) automatically evaluates and scores content based on the converted Markdown file, significantly improving the automation level of the review. The efficient parsing and conversion algorithms used in the conversion service intelligently identify and extract key text information from the document, mapping it to Markdown syntax to ensure the integrity and accuracy of the document's core structure and content. By converting the original JSONL format document to the smaller Markdown format, the token restriction problem during large model processing is effectively avoided, reducing data processing costs and improving review efficiency.

[0100] In this embodiment, the rich text editing function of the front-end module provides users with an intuitive and convenient document editing experience, ensuring user-friendliness in document creation and editing. Simultaneously, feedback on review results helps users optimize document content and improve document quality, further enhancing the user experience. The large language model review not only provides document scores but also offers specific modification suggestions, helping users continuously optimize document content and structure, promoting continuous improvement in document quality, and facilitating knowledge accumulation and team learning. System architecture flexibility and scalability: The system architecture is flexible, easy to maintain and extend, adaptable to constantly changing document review needs, and also easy to integrate with other systems or services, supporting a wider range of application scenarios. Promoting standardized document formats and management: Markdown, as a universal plain text format, promotes document standardization, simplifies document management and review processes, and facilitates unified document management, retrieval, and sharing.

[0101] In this embodiment, the implementation process may include: a research and development engineer creates a design document in a document system and uses rich text features such as bold, lists, and code blocks. After the document is saved, the backend stores it in JSONL format in a database or file system. The document verification system triggers a review task periodically or after user submission. The review service reads the JSONL file from the database and calls the conversion service. The conversion service parses the JSONL file into concise Markdown text. For example, {"type":"paragraph","content":[{"type":"text","text":"core idea"}]} contained in the original JSONL will be converted to core idea. {"type":"bulletList","content":[...]} will be converted to -... The converted Markdown file is sent to the LLM review service.

[0102] Optionally, the LLM evaluates the Markdown document content, such as determining whether the content is clearly structured, logically complete, and free of typos, and assigns a score from 1 to 5. The review service returns the score (e.g., 4.5 points) and the LLM's suggestions to the backend and associates them with the original document. Users can view the document's quality score and the LLM's improvement suggestions on the document's details page, enabling them to optimize the document. In this way, this embodiment of the application can efficiently and automatically control document quality, providing continuous feedback for knowledge accumulation to the R&D team.

[0103] This application proposes a two-stage method for document review, which involves first converting rich text storage formats (such as JSONL) into a streamlined plain text format (such as Markdown) optimized for large language models, and then performing automated review.

[0104] This application proposes a core technology including a specific format conversion algorithm that can efficiently and losslessly extract and map the core text content and basic structure (headings, lists, code blocks, etc.) of rich text format to a simplified format.

[0105] The embodiments of this application greatly reduce the amount of data required for document review, effectively solve the problems of token overrun and inefficiency when large language models process rich text documents, while retaining the editing and display functions of rich text.

[0106] This application primarily addresses the issue of excessive data volume during large model review. Alternative solutions may exist, but each has its limitations, including directly chunking the raw JSONL data and then inputting each chunk into the large model for review. While this approach avoids token exceeding limits, it disrupts the overall document context. The large model cannot effectively assess the overall document structure and logical coherence, impacting review quality. Furthermore, it requires additional context management mechanisms to connect the chunked results, increasing complexity.

[0107] Optionally, a dedicated "small model" or "preprocessing model" can be trained to handle redundant JSONL formatting. This model can automatically filter redundant information and output concise text. However, training and maintaining such a model is costly, and its generalization ability may not be as good as the general format conversion algorithm in this solution. Furthermore, this approach may still face efficiency bottlenecks when processing long documents. In contrast, the conversion-review method of this application utilizes Markdown, a universal, lightweight, and highly readable format, as an intermediate medium. This solves the data volume problem while preserving the overall document structure, making it the best solution currently available that balances efficiency, cost, and review quality.

[0108] The embodiments of this application can significantly improve the efficiency and accuracy of document review through the above-described method, while ensuring that the front-end user experience is not affected. By efficiently converting documents from JSONL format to Markdown format, the document review process is automated and intelligent. The large language model can quickly process and evaluate the core content of the document without wasting computing resources on format and style information, thereby improving the efficiency and accuracy of review.

[0109] In this embodiment, the size of the converted Markdown file is significantly reduced, lowering the token requirements for processing large documents and reducing processing costs. This is particularly important for scenarios involving a large number of documents, as it can significantly reduce the overall operating cost of the review system.

[0110] In this embodiment, even if the document has undergone format conversion to adapt to the review of the large model, users can still view and edit the rendered rich text document on the front end, ensuring the user-friendliness of the document creation and editing process and not affecting the work efficiency and satisfaction of business personnel.

[0111] In this embodiment of the application, the review results are associated with the document. Users can optimize the document based on the score and suggestions of the large model, which promotes the improvement of the quality of knowledge accumulation, helps to build a more professional and accurate document library, and provides high-quality knowledge sharing and learning resources for the R&D team.

[0112] In the embodiments of this application, the above method is not only applicable to the current document system architecture, but also has good scalability and compatibility. It can be easily integrated into the existing software development process without large-scale changes to the front-end system, thus reducing the threshold and cost of technology implementation.

[0113] In this embodiment of the application, by reducing the amount of data processing, the load on the backend system is reduced, the response speed is accelerated, and the overall performance of the document system is improved, providing users with a smoother and faster document management experience.

[0114] In this application embodiment, the use of standardized, lightweight document formats such as Markdown is encouraged, which helps to unify document storage and review standards and facilitates long-term document management, retrieval, and sharing.

[0115] The methods of the embodiments of this application will be further illustrated below.

[0116] Figure 2 This is a schematic diagram of an optional document verification system according to an embodiment of this application. For example... Figure 2 As shown, the document verification system 200 includes: a front-end module 202, a back-end service module 204, a conversion service module 206, and a verification service module 208.

[0117] Front-end module 202 is used to generate an initial document in response to input commands.

[0118] In this embodiment, the aforementioned front-end module 202 may include an interactive interface based on a rich text editor, which can be responsible for creating, editing, and rendering user files, and can generate an initial document in response to user input commands. These input commands can be user actions on the editor interface via mouse or keyboard, including clicking buttons, using shortcut keys, or directly entering text in text boxes, and can be used to instruct the front-end module how to process user input to generate or modify files.

[0119] The backend service module 204 is used to convert the initial document into a file to be verified according to a preset format and store it.

[0120] In this embodiment, the backend service module 204 can provide backend services and can be used to receive and store JSONL format documents submitted by the frontend. The preset format can be JSONL format.

[0121] Optionally, the aforementioned backend service module can convert the initial document into a file to be verified according to a preset format and store it.

[0122] The conversion service module 206 is used to respond to a file verification instruction, obtain the file to be verified indicated by the file verification instruction; extract key text information from the file to be verified, wherein the key text information is used to characterize the text content in the file to be verified; and use the key text information to perform format conversion on the file to be verified to obtain a target file, wherein the target file is in plain text format, and the plain text format is different from the initial file format of the file to be verified.

[0123] In this embodiment, the conversion service module 206 can provide a conversion service, which can be a standalone microservice or module, used to convert stored JSONL files into Markdown format. This service needs to implement an efficient parsing and conversion algorithm that can accurately extract text content and map it to the corresponding Markdown syntax (such as headings, lists, code blocks, quotations, etc.).

[0124] The verification service module 208 is used to perform verification operations on the target file and obtain the verification result, which is used to characterize the integrity of the text content in the target file.

[0125] In this embodiment, the verification service module 208 can be used to provide LLM review services. It can be an LLM review service, or an API call encapsulating a large language model, receiving Markdown-formatted text and returning review results, such as scores and modification suggestions, according to preset review rules. These review results can be verification results, used to characterize the completeness of the text content in the target document.

[0126] For example, a development engineer creates a design document in the front-end module (which could be a document system) and uses rich text features such as bold, lists, and code blocks. This design document can be the initial document. After the document is saved, the back-end service module can store this initial document in JSONL format in a database or file system to obtain the file to be verified. The document system can issue a file verification command periodically or after user submission to trigger a review task. In response to the file verification command, the review service reads the JSONL file from the database and calls the conversion service. The conversion service module parses the JSONL file into concise Markdown text. For example, {"type":"paragraph","content":[{"type":"text","text":"core idea"}]} contained in the original JSONL will be converted to core idea; {"type":"bulletList","content":[...]} will be converted to -... The converted Markdown file, i.e., the target file, is sent to the LLM review service; the LLM evaluates the Markdown document content, judging whether the Markdown document is clearly structured, logically complete, and free of typos, and gives a score of 1-5. This rating serves as a validation result. The review service returns the rating (e.g., 4.5 points) and the LLM's recommendations to the backend, linking them to the original document. Users can then view the document's quality rating and the LLM's improvement suggestions on the document's details page, enabling them to optimize the document.

[0127] Through the above steps, this technical solution can efficiently and automatically control document quality.

[0128] In this embodiment, the document verification system described above allows the front-end module to generate an initial document in response to an input command. The back-end service module then converts the initial document into a file to be verified according to a preset format and stores it. Upon receiving a file verification command, the back-end service module retrieves the file to be verified as indicated by the command; extracts key text information from the file; and uses this key text information to perform format conversion on the file to obtain the target file. Finally, the conversion service module performs a verification operation on the target file to obtain the verification result.

[0129] Figure 3 This is a schematic diagram of an optional document verification device according to an embodiment of this application. Figure 3 As shown, the document verification device 300 includes: an acquisition unit 302, an extraction unit 304, a conversion unit 306, and a verification unit 308.

[0130] The acquisition unit 302 is used to acquire the file to be verified as indicated by the file verification command in response to the file verification command.

[0131] Extraction unit 304 is used to extract key text information from the file to be verified, wherein the key text information is used to characterize the text content in the file to be verified.

[0132] The conversion unit 306 is used to convert the format of the file to be verified using key text information to obtain a target file. The target file is in plain text format, which is different from the initial file format of the file to be verified.

[0133] The verification unit 308 is used to perform a verification operation on the target file and obtain a verification result, wherein the verification result is used to characterize the integrity of the text content in the target file.

[0134] In this embodiment, the document verification device described above can obtain the document to be verified by the document verification instruction in response to the document verification instruction by the acquisition unit; extract key text information from the document to be verified by the extraction unit; convert the document to be verified by the key text information by the conversion unit to obtain the target document; and perform a verification operation on the target document by the verification unit to obtain the verification result.

[0135] According to another aspect of the embodiments of this application, an electronic device is also provided, including: a memory storing an executable program; and a processor for running the program, wherein the program executes the method of any one of the above when it runs.

[0136] According to another aspect of the embodiments of this application, a computer-readable storage medium is also provided, the computer-readable storage medium including a stored executable program, wherein, when the executable program is running, it controls the device where the computer-readable storage medium is located to perform the methods of various embodiments of this application.

[0137] According to another aspect of the embodiments of this application, a computer program product is also provided, including a computer program that, when executed by a processor, implements the methods of various embodiments of this application.

[0138] According to another aspect of the embodiments of this application, a computer program product is also provided, including a non-volatile computer-readable storage medium storing a computer program, which, when executed by a processor, implements the methods in various embodiments of this application.

[0139] According to another aspect of the embodiments of this application, a computer program is also provided, which, when executed by a processor, implements the methods of the various embodiments of this application.

[0140] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0141] In the above embodiments of this application, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0142] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units can be a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual couplings, direct couplings, or communication connections may be through some interfaces; indirect couplings or communication connections between units or modules may be electrical or other forms.

[0143] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0144] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0145] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard drive, magnetic disk, or optical disk.

[0146] The above are merely preferred embodiments of this application. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of this application, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. A method for verifying a file, characterized in that, include: In response to a file verification command, obtain the file to be verified as indicated by the file verification command; Extract key text information from the file to be verified, wherein the key text information is used to characterize the text content in the file to be verified; Using the key text information, the format of the file to be verified is converted to obtain a target file, wherein the target file is in plain text format, and the plain text format is different from the initial file format of the file to be verified. A verification operation is performed on the target file to obtain a verification result, wherein the verification result is used to characterize the completeness of the text content in the target file.

2. The method according to claim 1, characterized in that, The extraction of key text information from the file to be verified includes: Redundant information is removed from the file to be verified to obtain a simplified file; Extract the key text information from the simplified document.

3. The method according to claim 2, characterized in that, The extraction of the key text information from the simplified document includes: Extract at least one object information from at least one line of string in the simplified file; Using a preset key, at least one of the object information is traversed to obtain the key text information.

4. The method according to claim 1, characterized in that, The step of performing a verification operation on the target file to obtain a verification result includes: The target file is input into a language model for verification to obtain the verification result. The language model is used to verify the target file based on a preset verification standard.

5. The method according to any one of claims 1 to 4, characterized in that, The method further includes: Get the initial document generated by the text editor in the front-end module; The initial document is converted into the file to be verified according to the preset format.

6. A file verification system, characterized in that, include: The front-end module is used to generate the initial document in response to input commands; The backend service module is used to convert the initial document into a file to be verified according to a preset format and store it. The conversion service module is used to obtain the file to be verified as indicated by the file verification instruction in response to receiving the file verification instruction; Extract key text information from the file to be verified, wherein the key text information is used to characterize the text content in the file to be verified; use the key text information to perform format conversion on the file to be verified to obtain a target file, wherein the target file is in plain text format, and the plain text format is different from the initial file format of the file to be verified; The verification service module is used to perform verification operations on the target file and obtain verification results, wherein the verification results are used to characterize the completeness of the text content in the target file.

7. A document verification device, characterized in that, include: The acquisition unit is configured to acquire the file to be verified as indicated by the file verification command in response to the file verification command; An extraction unit is used to extract key text information from the file to be verified, wherein the key text information is used to characterize the text content in the file to be verified. The conversion unit is used to convert the format of the file to be verified using the key text information to obtain a target file, wherein the target file is in plain text format and the plain text format is different from the initial file format of the file to be verified. The verification unit is used to perform a verification operation on the target file and obtain a verification result, wherein the verification result is used to characterize the completeness of the text content in the target file.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a stored program, wherein, when the program is executed, it controls the device on which the computer-readable storage medium is located to perform the method according to any one of claims 1 to 5.

9. A processor, characterized in that, The processor is used to run a program, wherein the program is executed by the processor to perform the method according to any one of claims 1 to 5.

10. A computer program product, characterized in that, Includes computer instructions that, when executed by a processor, implement the method described in any one of claims 1 to 5.

Images