A review method, apparatus, device, and medium

Abstract

Embodiments of the present application provide a review method, device, equipment and medium, the method comprises: obtaining an original file; layout detection is carried out on the original file to determine the content block in the original file, the type of content block and the layout sequence of content block;According to the type of content block, the corresponding identification model is called to generate the text corresponding to the content block;According to the layout sequence of content block, the text corresponding to each content block is spliced into a text file;Determine the project to be reviewed in the text file, and the review rule corresponding to the plurality of projects to be reviewed;Through the review rule, the plurality of projects to be reviewed are reviewed, and the review result is output.The embodiments of the present application convert the original file into a text file through layout detection and multi-model collaborative identification to determine the project to be reviewed in the text file, review the plurality of projects to be reviewed according to the review rule, and output the review result, which significantly improves the automation level and accuracy of the review work.

Description

Technical Field

[0001] This invention relates to the field of evaluation technology, and in particular to an evaluation method and an evaluation apparatus. Background Technology

[0002] In the context of increasingly complex and sophisticated review processes, traditional review methods face challenges such as low efficiency, heavy reliance on manual review, inconsistent review standards, and a high risk of missed submissions. Application documents are often submitted as scanned PDFs, with complex content and varying structures. Manual review is not only time-consuming and labor-intensive but also prone to errors or inconsistencies in expression, leading to a decline in review quality. Summary of the Invention

[0003] In view of the above problems, embodiments of the present invention are proposed to provide a review method, apparatus, device and medium that overcomes or at least partially solves the above problems.

[0004] According to a first aspect of the present invention, a review method is provided, the method comprising: Obtain the original file; The original file is subjected to layout detection to determine the content blocks in the original file, the type of the content blocks, and the layout order of the content blocks; Based on the type of the content block, the corresponding recognition model is invoked to generate the text corresponding to the content block; Based on the layout order of the content blocks, the text corresponding to each content block is concatenated into a text file; Identify multiple items to be reviewed in a text file, and the review rules corresponding to the multiple items to be reviewed; The multiple projects to be reviewed are reviewed according to the review rules corresponding to them, and the review results are output.

[0005] Optionally, the review of each project to be reviewed according to the review rules corresponding to each project to be reviewed includes: The projects to be reviewed are input into the review model, so that the review model reviews each project according to the review rules corresponding to each project.

[0006] Optionally, the review rules include review points and review parameters corresponding to the review points; the step of inputting the projects to be reviewed into the review model, so that the review model reviews each project to be reviewed according to the review rules corresponding to each project to be reviewed, includes: After inputting the project to be reviewed into the review model, the project parameters and project conclusions of the project to be reviewed are determined through the review model. The project parameters of the project to be reviewed are compared with the review parameters to obtain a first comparison result, and the project conclusions of the project to be reviewed are compared with the review points to obtain a second comparison result. Based on the first comparison result and the second comparison result, the review conclusions and review basis are output. The review basis is an analysis and explanation that infers the review conclusions based on the review points corresponding to the project and the parameters corresponding to the review points.

[0007] Optionally, the step of outputting review conclusions and review criteria based on the first comparison result and the second comparison result includes: If the first comparison result shows that the project parameters are consistent with the review parameters, and the second comparison result shows that the project conclusion is consistent with the review points, the output review conclusion is compliance with the rules, and the review basis corresponding to the project's compliance with the rules is also provided. If the first comparison result is that the project parameters are inconsistent with the review parameters, and / or the second comparison result is that the project conclusion is inconsistent with the review points, the output review conclusion is that the project does not comply with the rules, and the review basis corresponding to the project not complying with the rules is provided.

[0008] Optionally, the method further includes: Get search terms; Identify multiple target blocks in the content blocks of the original file that correspond to the search term; The target block is marked.

[0009] Optionally, determining the multiple target blocks in the content blocks of the original file that correspond to the search term includes: Determine the semantics of the text corresponding to the content blocks of the original file; Calculate the semantic similarity between the semantic term and the search term; Content blocks with semantic similarity greater than the similarity threshold are identified as target blocks.

[0010] Optionally, marking the target block includes: Determine the similarity level corresponding to the semantic similarity of the multiple target blocks; Target blocks with different similarity levels are marked with different colors.

[0011] According to a second aspect of the present invention, a review apparatus is provided, the apparatus comprising: The original file acquisition module is used to acquire the original file. The layout detection module is used to perform layout detection on the original file to determine the content blocks in the original file, the type of the content blocks, and the layout order of the content blocks; The text generation module is used to call the corresponding recognition model according to the type of the content block to generate the text corresponding to the content block; The splicing module is used to splice the text corresponding to each content block into a text file according to the layout order of the content blocks; The review project determination module is used to determine multiple projects to be reviewed in a text file, as well as the review rules corresponding to the multiple projects to be reviewed. The review module is used to review the multiple projects to be reviewed according to the review rules corresponding to the multiple projects to be reviewed, and output the review results.

[0012] Optionally, the review module includes: The review submodule is used to input the projects to be reviewed into the review model, so that the review model can review each project according to the review rules corresponding to each project.

[0013] Optionally, the review rules include review points and review parameters corresponding to the review points; the review submodule includes: The review unit is used to input the project to be reviewed into the review model, determine the project parameters and project conclusions of the project to be reviewed through the review model, compare the project parameters of the project to be reviewed with the review parameters to obtain a first comparison result, and compare the project conclusions of the project to be reviewed with the review points to obtain a second comparison result; output the review conclusions and review basis based on the first comparison results and the second comparison results; the review basis is an analysis and explanation of the review conclusions inferred from the review points corresponding to the project and the parameters corresponding to the review points.

[0014] Optionally, the review unit includes: The first review subunit is used to output the review conclusion that the project conforms to the rules and the review basis corresponding to the project conforming to the rules when the first comparison result is that the project parameters are consistent with the review parameters and the second comparison result is that the project conclusion is consistent with the review points. The second review subunit is used to output the review conclusion that the project does not comply with the rules, and the review basis corresponding to the project not complying with the rules, when the first comparison result is that the project parameters are inconsistent with the review parameters, and / or the second comparison result is that the project conclusion is inconsistent with the review points.

[0015] Optionally, the method further includes: The search term acquisition module is used to acquire search terms; The target block determination module is used to determine multiple target blocks in the content blocks of the original file that correspond to the search term; A marking module is used to mark the target block.

[0016] Optionally, the target block determination module includes: The semantic determination submodule is used to determine the semantics of the text corresponding to the content blocks of the original file; A similarity calculation module is used to calculate the semantic similarity between the semantic term and the search term; The target block determination submodule is used to determine content blocks with semantic similarity greater than a similarity threshold as target blocks.

[0017] Optionally, the marking module includes: The similarity level determination submodule is used to determine the similarity level corresponding to the semantic similarity of the multiple target blocks; The tagging submodule is used to tag target blocks with different similarity levels using different colors.

[0018] According to a third aspect of the present invention, an electronic device is provided, the electronic device comprising: a processor, a memory, and a computer program stored in the memory and capable of running on the processor, wherein the computer program, when executed by the processor, implements the steps of the review method as described in any of the preceding claims.

[0019] According to a fourth aspect of the present invention, a computer-readable storage medium is provided, on which a computer program is stored, which, when executed by a processor, implements the steps of the review method as described in any of the preceding claims.

[0020] The technical solutions provided by the embodiments of the present invention may include the following beneficial effects: This invention provides a review method that involves: acquiring an original file; performing layout detection on the original file to determine the content blocks, their types, and their layout order; generating text corresponding to each content block based on its type using a corresponding recognition model; concatenating the text of each content block into a text file according to their layout order; determining the items to be reviewed in the text file and the review rules for each item; reviewing the items according to the review rules and outputting the review results. This invention, through layout detection and multi-model collaborative recognition, converts the original file into a text file to determine the items to be reviewed, reviews the items according to the review rules, and outputs the review results, significantly improving the automation and accuracy of the review process. Attached Figure Description

[0021] Figure 1 This is a flowchart of the steps of a review method provided in an embodiment of the present invention; Figure 2 This is a flowchart of another review method provided in an embodiment of the present invention; Figure 3 This is a flowchart of another review method provided in an embodiment of the present invention; Figure 4 This is a flowchart of another review method provided in an embodiment of the present invention; Figure 5 This is a structural block diagram of a review device provided in an embodiment of the present invention. Detailed Implementation

[0022] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0023] One of the core concepts of this invention is that by using layout detection and multi-model collaborative recognition, the original file is converted into a text file to identify the items to be reviewed in the text file. Multiple items to be reviewed are then reviewed according to the review rules, and the review results are output, which significantly improves the automation level and accuracy of the review process.

[0024] Reference Figure 1 The diagram illustrates a flowchart of a review method provided by an embodiment of the present invention. The method may specifically include the following steps: Step 101, Obtain the original file; For example, the original document refers to a non-text electronic document submitted by a user whose internal structured content cannot be directly parsed by a computer program. Examples include PDF (Portable Document Format) files, especially scanned PDFs generated as images. While such files visually present text, tables, and images, their underlying data consists of image pixels or unannotated graphic objects, lacking a directly extractable text layer and logical structure. Therefore, deep analysis of the original PDF file is required using techniques such as layout analysis, optical character recognition (OCR), formula recognition, and multimodal understanding to transform it into a structured text representation usable by an automated review system, laying the foundation for subsequent rule matching and intelligent review.

[0025] Step 102: Perform layout detection on the original file to determine the content blocks in the original file, the type of the content blocks, and the layout order of the content blocks; For example, in review scenarios, application materials are often submitted as scanned PDFs, which are essentially images rather than editable text, making it impossible to directly extract structured information. Without layout detection, the system cannot distinguish between different semantic units such as titles, body text, tables, formulas, and images on a page, leading to confusion in subsequent recognition. For instance, mistaking a table for a regular paragraph or treating a figure caption as main text will severely disrupt the content logic. Layout detection can accurately locate the spatial position (coordinates) of each content block, identify its type (such as text, table, formula, image), and restore its reading order in the original document (i.e., layout order). This process not only provides a basis for regional division for subsequent differentiated recognition but also ensures the integrity of semantic units and contextual coherence. Especially for non-linear structures such as multi-column layouts, mixed text and images, or complex tables, the layout order facilitates semantic understanding of the logic after text concatenation.

[0026] Step 103: Based on the type of the content block, call the corresponding recognition model to generate the text corresponding to the content block; For example, different types of document elements have drastically different content characteristics and recognition requirements, making it difficult for general OCR models to achieve the required accuracy across all scenarios. For instance, ordinary text can be accurately recognized by standard OCR, but mathematical formulas contain numerous special symbols and subscript / superscript structures, requiring specialized formula recognition models (such as LaTeX converters) for correct parsing. Tables not only contain text but also structural information such as row and column relationships and cell merging, necessitating the use of table structure recognition models to reconstruct their logical layout. Images themselves contain no text but may contain crucial evidence (such as photomicrographs or chromatograms), requiring the generation of semantic descriptions using multimodal large-scale models. Using a single OCR engine to process all content blocks would lead to problems such as garbled formulas, tables collapsing into disordered text, and complete loss of image information, severely impacting the integrity and accuracy of the review criteria. Therefore, it is essential to dynamically allocate the most suitable recognition model based on the block type output by layout detection—using OCR for text blocks, formula recognizers for formula blocks, table parsers for table blocks, and visual language models for image blocks—to generate high-fidelity, structured text representations for each type of content, providing high-quality input for subsequent rule matching and intelligent judgment.

[0027] Step 104: According to the layout order of the content blocks, concatenate the text corresponding to each content block into a text file; For example, the semantics of a document depends not only on its textual content but also on its presentation order and spatial logic. For instance, a "conclusion" appearing before "methods" can lead to misunderstanding; tables broken up and inserted into irrelevant paragraphs will lose their data coherence. Ignoring layout order and simply piecing together text according to recognition sequence or coordinate order can easily disrupt the logical flow of the original text, causing subsequent misjudgments. For example, incorrectly placing page numbers in footers at the end of the main text or misplacing figure captions with other illustrations. By following the layout order of the original document (usually determined by the reading order or spatial topology output by the layout detection model), the generated text file can be ensured to semantically faithfully reproduce the structure of the original application materials—titles first, body text centered, tables following explanations, and formulas embedded in the derivation process. This structure-preserved text not only facilitates accurate understanding of the context by large models but also provides a reliable foundation for subsequent extraction of review elements by chapter and project.

[0028] Step 105: Determine multiple items to be reviewed in the text file, and the review rules corresponding to the multiple items to be reviewed; For example, by using natural language processing or keyword matching, specific items to be reviewed (such as "skin allergy test" or "lead content detection") are identified from structured text, enabling task focus. Secondly, each item is associated with pre-defined review rules (including judgment criteria, thresholds, and methodological requirements), forming a "item-rule" mapping relationship. This process shifts intelligent review from generalized reading to targeted verification, avoiding omissions of key inspection items. More importantly, the review rules, as authoritative evidence, ensure consistency and legality of judgments—different products, different times, and different reviewers all follow the same set of standards, eliminating subjective arbitrariness. Simultaneously, the rule-driven approach supports flexible expansion: adding new regulations only requires updating the rule base, without reconstructing the entire system. Ultimately, this step transforms massive amounts of text into a calculable, verifiable, and traceable review task list, serving as the logical hub for achieving efficient, accurate, and compliant intelligent review.

[0029] Step 106: Review the multiple projects to be reviewed according to the review rules corresponding to the multiple projects to be reviewed, and output the review results.

[0030] For example, traditional manual review relies heavily on expert experience and is susceptible to subjective judgment, fatigue, or knowledge gaps. By introducing a large language model as the review model and explicitly injecting review rules (such as regulatory clauses, technical thresholds, and methodological requirements) corresponding to the project being reviewed into its reasoning process, the model can understand the application content while strictly adhering to authoritative rules for logical comparison and compliance judgment. This review model can not only identify the fact that "lead content is 12.3 mg / kg," but also automatically deduce the conclusion of "non-compliance" by combining it with the rule that "the limit is 10 mg / kg," and generate traceable evidence such as "the detected value exceeds the limit in Article 4.3 of the Cosmetic Safety Technical Specifications." In addition, the model can handle synonyms, omissions, or indirect descriptions in natural language, improving its robust understanding of unstructured text. More importantly, by dynamically injecting rules as contextual prompts, the system can quickly adapt to new regulations or new review points without retraining the model, significantly enhancing flexibility and maintainability.

[0031] This invention provides a review method that involves: acquiring an original file; performing layout detection on the original file to determine the content blocks, their types, and their layout order; generating text corresponding to each content block based on its type using a corresponding recognition model; concatenating the text of each content block into a text file according to their layout order; determining the items to be reviewed in the text file and the review rules for each item; reviewing the items according to the review rules and outputting the review results. This invention, through layout detection and multi-model collaborative recognition, converts the original file into a text file to determine the items to be reviewed, reviews the items according to the review rules, and outputs the review results, significantly improving the automation and accuracy of the review process.

[0032] Reference Figure 2 The diagram illustrates a flowchart of a review method provided by an embodiment of the present invention. The method may specifically include the following steps: Step 201, Obtain the original file; For example, the original document refers to a non-text electronic document submitted by a user whose internal structured content cannot be directly parsed by a computer program. Examples include PDF (Portable Document Format) files, especially scanned PDFs generated as images. While such files visually present text, tables, and images, their underlying data consists of image pixels or unannotated graphic objects, lacking a directly extractable text layer and logical structure. Therefore, deep analysis of the original PDF file is required using techniques such as layout analysis, optical character recognition (OCR), formula recognition, and multimodal understanding to transform it into a structured text representation usable by an automated review system, laying the foundation for subsequent rule matching and intelligent review.

[0033] Step 202: Perform layout detection on the original file to determine the content blocks in the original file, the type of the content blocks, and the layout order of the content blocks; For example, in review scenarios, application materials are often submitted as scanned PDFs, which are essentially images rather than editable text, making it impossible to directly extract structured information. Without layout detection, the system cannot distinguish between different semantic units such as titles, body text, tables, formulas, and images on a page, leading to confusion in subsequent recognition. For instance, mistaking a table for a regular paragraph or treating a figure caption as main text will severely disrupt the content logic. Layout detection can accurately locate the spatial position (coordinates) of each content block, identify its type (such as text, table, formula, image), and restore its reading order in the original document (i.e., layout order). This process not only provides a basis for regional division for subsequent differentiated recognition but also ensures the integrity of semantic units and contextual coherence. Especially for non-linear structures such as multi-column layouts, mixed text and images, or complex tables, the layout order facilitates semantic understanding of the logic after text concatenation.

[0034] Step 203: Based on the type of the content block, call the corresponding recognition model to generate the text corresponding to the content block; For example, different types of document elements have drastically different content characteristics and recognition requirements, making it difficult for general OCR models to achieve the required accuracy across all scenarios. For instance, ordinary text can be accurately recognized by standard OCR, but mathematical formulas contain numerous special symbols and subscript / superscript structures, requiring specialized formula recognition models (such as LaTeX converters) for correct parsing. Tables not only contain text but also structural information such as row and column relationships and cell merging, necessitating the use of table structure recognition models to reconstruct their logical layout. Images themselves contain no text but may contain crucial evidence (such as photomicrographs or chromatograms), requiring the generation of semantic descriptions using multimodal large-scale models. Using a single OCR engine to process all content blocks would lead to problems such as garbled formulas, tables collapsing into disordered text, and complete loss of image information, severely impacting the integrity and accuracy of the review criteria. Therefore, it is essential to dynamically allocate the most suitable recognition model based on the block type output by layout detection—using OCR for text blocks, formula recognizers for formula blocks, table parsers for table blocks, and visual language models for image blocks—to generate high-fidelity, structured text representations for each type of content, providing high-quality input for subsequent rule matching and intelligent judgment.

[0035] Step 204: According to the layout order of the content blocks, concatenate the text corresponding to each content block into a text file; For example, the semantics of a document depends not only on its textual content but also on its presentation order and spatial logic. For instance, a "conclusion" appearing before "methods" can lead to misunderstanding; tables broken up and inserted into irrelevant paragraphs will lose their data coherence. Ignoring layout order and simply piecing together text according to recognition sequence or coordinate order can easily disrupt the logical flow of the original text, causing subsequent misjudgments. For example, incorrectly placing page numbers in footers at the end of the main text or misplacing figure captions with other illustrations. By following the layout order of the original document (usually determined by the reading order or spatial topology output by the layout detection model), the generated text file can be ensured to semantically faithfully reproduce the structure of the original application materials—titles first, body text centered, tables following explanations, and formulas embedded in the derivation process. This structure-preserved text not only facilitates accurate understanding of the context by large models but also provides a reliable foundation for subsequent extraction of review elements by chapter and project.

[0036] Step 205: Determine multiple items to be reviewed in the text file, and the review rules corresponding to the multiple items to be reviewed; For example, by using natural language processing or keyword matching, specific items to be reviewed (such as "skin allergy test" or "lead content detection") are identified from structured text, enabling task focus. Secondly, each item is associated with pre-defined review rules (including judgment criteria, thresholds, and methodological requirements), forming a "item-rule" mapping relationship. This process shifts intelligent review from generalized reading to targeted verification, avoiding omissions of key inspection items. More importantly, the review rules, as authoritative evidence, ensure consistency and legality of judgments—different products, different times, and different reviewers all follow the same set of standards, eliminating subjective arbitrariness. Simultaneously, the rule-driven approach supports flexible expansion: adding new regulations only requires updating the rule base, without reconstructing the entire system. Ultimately, this step transforms massive amounts of text into a calculable, verifiable, and traceable review task list, serving as the logical hub for achieving efficient, accurate, and compliant intelligent review.

[0037] Step 206: Input the projects to be reviewed into the review model so that the review model can review each project according to the review rules corresponding to each project.

[0038] For example, traditional manual review relies heavily on expert experience and is susceptible to subjective judgment, fatigue, or knowledge gaps. By introducing a large language model as the review model and explicitly injecting review rules (such as regulatory clauses, technical thresholds, and methodological requirements) corresponding to the project being reviewed into its reasoning process, the model can understand the application content while strictly adhering to authoritative rules for logical comparison and compliance judgment. This review model can not only identify the fact that "lead content is 12.3 mg / kg," but also automatically deduce the conclusion of "non-compliance" by combining it with the rule that "the limit is 10 mg / kg," and generate traceable evidence such as "the detected value exceeds the limit in Article 4.3 of the Cosmetic Safety Technical Specifications." In addition, the model can handle synonyms, omissions, or indirect descriptions in natural language, improving its robust understanding of unstructured text. More importantly, by dynamically injecting rules as contextual prompts, the system can quickly adapt to new regulations or new review points without retraining the model, significantly enhancing flexibility and maintainability.

[0039] Reference Figure 3This diagram illustrates a flowchart of another review method provided by an embodiment of the present invention. The overall process can be implemented by three modules: The document preprocessing module identifies text, tables, images, formulas, and other content blocks in the page of the document to be reviewed (i.e., the original file) using a layout detection model. It then extracts each type of content using specialized models such as OCR, table recognition, and formula recognition, generating Markdown text (i.e., a structured text file) that retains the original logical order, achieving efficient conversion from unstructured image documents to structured text. Subsequently, the single-item review module extracts information from the items to be reviewed in the Markdown text using a large information extraction model based on the extraction elements corresponding to each review rule under each item. The extracted information may include item parameters and item conclusions. The data is then input into the review model, which combines the extracted relevant content with each review rule of the item to be reviewed, and outputs review conclusions and review basis (i.e., the review results and thought process shown in the diagram). Finally, the summary opinion generation module slices, vectorizes, and stores industry knowledge such as laws and regulations, historical cases, and work standards into a vector library to form an industry knowledge base. Based on the review results of each project and relevant information in the knowledge base, the summary opinion generation model is input to generate the summary opinion.

[0040] In one embodiment, the review rules include review points and review parameters corresponding to the review points; step 206 includes the following sub-steps: Sub-step S11: After inputting the project to be reviewed into the review model, the project parameters and project conclusions of the project to be reviewed are determined through the review model. The project parameters of the project to be reviewed are compared with the review parameters to obtain a first comparison result. The project conclusions of the project to be reviewed are compared with the review points to obtain a second comparison result. The review conclusions and review basis are output based on the first comparison result and the second comparison result. The review basis is an analysis and explanation that infers the review conclusions based on the review points corresponding to the project and the parameters corresponding to the review points.

[0041] For example, the review guidelines describe the core requirements for a certain test in regulations or technical specifications (e.g., "at least 60% of animals in the positive control group should show skin allergic reactions"), while the review parameters are the structured judgment indicators that can be extracted from the guidelines (e.g., "positive rate threshold: 60%"). First, the item to be reviewed is input into the review model, which extracts the actual item parameters (e.g., "the sensitization rate of the positive control is 15%) and the implicit item conclusions (e.g., "the test results indicate that the positive control is effective") from the original text. Then, the system performs a double comparison: on the one hand, it compares the item parameters with the review parameters numerically or logically to obtain the first comparison result (e.g., "15% < 60%, not up to standard"); on the other hand, it performs a consistency analysis between the item conclusions and the semantic requirements of the review guidelines to obtain the second comparison result (e.g., "the conclusion claims effectiveness, but the data does not support it"). Ultimately, the review model integrates the two comparison results to output a clear review conclusion (such as "non-compliant") and detailed review basis—that is, an analysis and explanation derived from the review points and parameters. For example, "In the skin allergy test, the sensitization rate of the positive control group was only 15%, which is lower than the 60% threshold required by the review rules. It cannot prove the effectiveness of the test system, so it is judged as non-compliant." This dual-dimensional verification mechanism not only improves the accuracy of the judgment but also enhances the interpretability and compliance credibility of the intelligent review. It avoids misjudgments that may be caused by relying solely on keyword matching or single numerical comparison, ensuring that the review process is both in line with technical specifications and has logical rigor and legal basis.

[0042] Furthermore, the review conclusion is not limited to "compliant" or "non-compliant," but can also include "not applicable" and "recommended for manual review" to address complex and diverse application scenarios. When the experimental methods, technical approaches, or product types used in the project under review are inconsistent with the scope of application explicitly defined by the review rules (e.g., the rules require intradermal injection, while the application materials use a topical application method), the rule is deemed inapplicable to the current project, resulting in a "not applicable" conclusion to avoid misjudgment due to forced application. Conversely, when the document content is formally complete but contains information that may imply medical efficacy, uses vague medical terminology, poses a risk of exaggerated claims, or involves emerging technologies without clear judgment standards (e.g., claims of "inhibiting tyrosinase and preventing melanin production"), the system, due to insufficient rule coverage or semantic uncertainty, will output a "recommended for manual review" conclusion with a risk warning, for final expert assessment. The introduction of these two conclusions significantly enhances the system's adaptability and security, preventing rule abuse while retaining necessary manual intervention channels for boundary cases, achieving an organic balance between automated efficiency and review rigor.

[0043] In one embodiment, sub-step S11 includes the following sub-steps: Sub-step S111: If the first comparison result is that the project parameters are consistent with the review parameters, and the second comparison result is that the project conclusion is consistent with the review points, the output review conclusion is compliance with the rules, and the review basis corresponding to the project compliance with the rules. For example, the review criteria describe the qualitative requirements in regulations or technical specifications (such as "positive controls should be effective"), reflecting compliance standards at the logical and semantic levels; the review parameters extract quantifiable indicators from these criteria (such as "sensitization rate ≥ 60%)," providing objective and calculable criteria for judgment. The review model first extracts project parameters (actual detection values) and project conclusions (explanations or claims of results) from the application text, and then compares them with the review parameters and review criteria respectively, forming a first comparison result (whether the data meets the standards) and a second comparison result (whether the conclusion is reasonable). This dual-track verification method can effectively identify complex situations such as "data is qualified but the conclusion is wrong" or "the conclusion seems reasonable but the data does not support it." For example, a report claims that "positive controls are effective," but the sensitization rate is only 15%. In this case, the project conclusion is superficially consistent with the review criteria, but the project parameters deviate significantly from the review parameters, and the system can still determine that it is "non-compliant." Ultimately, the review basis is the logical analysis and explanation generated based on these two comparisons, ensuring that the conclusion is supported by data and has semantic reasonableness. This design significantly improves the rigor, robustness, and interpretability of intelligent review, avoiding misreviews or omissions due to biased judgments.

[0044] Sub-step S112: If the first comparison result is that the project parameters are inconsistent with the review parameters, and / or the second comparison result is that the project conclusion is inconsistent with the review points, the output review conclusion is that the project does not comply with the rules, and the review basis corresponding to the project not complying with the rules.

[0045] For example, when the first comparison result is inconsistent (i.e., the project parameters do not meet the review parameter requirements, such as lead content of 12.3 mg / kg > limit of 10 mg / kg), it indicates that the objective data has violated the technical standards, and regardless of its textual description, it constitutes a substantial non-compliance. When the second comparison result is inconsistent (i.e., the project conclusion contradicts the review points, such as claiming "the test system is effective" but not using the prescribed method or lacking key controls), it indicates that the applicant's understanding or expression of the results is biased or misleading. Even if the data happens to meet the standards, its scientific validity and reliability are still questionable. More seriously, if both are inconsistent (e.g., the data exceeds the standard and the conclusion is incorrect), the problem is even more prominent. Therefore, adopting an "AND / OR" logic—that is, if either dimension is inconsistent, it is judged as "non-compliant"—is a necessary requirement for the rigor of the review. This mechanism can effectively prevent circumvention behaviors such as "using correct conclusions to cover up erroneous data" or "using compliant data to package invalid methods." At the same time, targeted review criteria can be generated based on this, such as: "The test value exceeds the limit, and the declaration conclusion does not mention the risk of exceeding the standard, which does not meet the requirements of Article X of the 'Regulations'." This two-dimensional rejection strategy not only conforms to the "data-logic" dual verification principle of regulatory science, but also ensures the authority, impartiality and legal admissibility of the intelligent review results.

[0046] In one embodiment, the method further includes: acquiring a search term; determining multiple target blocks in the content blocks of the original file that correspond to the search term; and marking the target blocks.

[0047] For example, the original document can be the original application materials. During the intelligent review process, reviewers may need to verify and validate the conclusions output by the model. However, the original application materials are mostly PDF scans, making it impossible to directly locate the original text through string search. Therefore, a semantic-based source tracing highlighting method is introduced: first, the user-input search terms (such as "sensitization rate" or "lead content") are obtained; then, semantically relevant fragments are retrieved from the structured original document content blocks to identify multiple target blocks, which are then visually marked. This design allows reviewers to jump to the original evidence location on which the intelligent judgment is based with one click, achieving rapid linkage between "conclusion—basis—original text." This not only significantly improves the efficiency of manual review but also enhances the transparency and credibility of intelligent review.

[0048] For example, traditional editable documents (such as Word, editable PDFs, or plain text .txt files) support character-level tracing and highlighting through precise string matching due to their embedded native text layer and structured metadata. This type of method relies on the text content and precise location coordinates provided by the document itself, making it simple and efficient. However, it only works with plain text content and cannot handle non-text elements such as images, tables, or formulas. Furthermore, it is sensitive to differences in expression (such as synonyms or word order changes) and lacks semantic understanding capabilities. However, in actual business scenarios such as government affairs, drug supervision, and cosmetics review, user-submitted application materials are mostly PDF scans—image-type PDFs generated from scanned paper documents. These lack a true text layer, containing only pixel information, rendering traditional string matching completely ineffective and preventing any text selection, searching, or highlighting operations.

[0049] In this embodiment of the invention, the original scanned document is first subjected to layout detection to identify different types of semantic regions, such as text, tables, images, and formulas. Then, an OCR engine, a table structure recognition model, a formula recognition model, and a multimodal large model are invoked to generate structured text descriptions of each region (such as LaTeX formulas, HTML tables, image semantic descriptions, etc.), and their precise coordinates in the original page are recorded simultaneously. Based on this, the search term and each content block text are segmented using jieba, and the BM25 algorithm is used to calculate semantic relevance scores based on word frequency and inverse document frequency. Compared to simple string matching, BM25 can effectively capture semantically similar but literally different expressions such as "sensitization rate" and "positive reaction ratio," significantly improving the quality of source tracing. Finally, the Top-N target blocks are selected based on similarity ranking, and their coordinates are used to achieve block-level visualization and highlighting on the original PDF. This solution not only overcomes the limitation of scanned documents lacking a text layer but also achieves unified source tracing and highlighting of multimodal content such as text, tables, images, and formulas, providing reliable and intuitive technical support for the manual review of intelligent evaluation results.

[0050] In one embodiment, determining multiple target blocks in the content blocks of the original file that correspond to the search term includes: determining the semantics of the text corresponding to the content blocks of the original file; calculating the semantic similarity between the semantics and the search term; and determining the content blocks with a semantic similarity greater than a similarity threshold as target blocks.

[0051] For example, the semantic vector of each content block is extracted (encoded via an embedding model or a large model), and then its semantic similarity (e.g., cosine similarity) with the search term is calculated. By setting a reasonable threshold, blocks with high semantic relevance are selected as target blocks. This method can effectively identify synonyms, hyponyms, or contextual implicit information (e.g., "animals develop red spots" implies "skin irritation"), significantly improving recall and localization accuracy. Compared to traditional word frequency statistics methods, semantic similarity is better able to capture deep semantic connections, ensuring that key evidence is not missed and providing technical support for accurate source tracing.

[0052] In one embodiment, marking the target blocks includes: determining the similarity levels corresponding to the semantic similarity of the plurality of target blocks; and marking target blocks with different similarity levels with different colors.

[0053] For example, to help reviewers quickly distinguish the relevance of evidence, target blocks can be divided into multiple levels of semantic similarity, such as high, medium, and low, and assigned different colors (e.g., red for high relevance, yellow for medium relevance, and blue for weak relevance). This hierarchical visual labeling has significant practical value: on the one hand, reviewers can prioritize high-similarity areas, improving review efficiency; on the other hand, when multiple candidate segments exist, color hints help determine which is more likely to be the true basis for the intelligent conclusion. Furthermore, color coding can also expose potential problems—for example, if the intelligent conclusion is based on a low-similarity (blue) block, it may indicate a model misjudgment or retrieval failure, requiring focused verification. This design transforms abstract semantic scores into intuitive visual signals, improving both human-computer interaction efficiency and the interpretability and reliability of the review process, making it a key auxiliary means for achieving efficient and reliable human review.

[0054] Reference Figure 4 This paper illustrates the steps of another review method provided by an embodiment of the present invention. First, a text file is generated using a recognition model corresponding to different types of text blocks. The text is then semantically encoded to generate an embedding vector. Next, the query terms and document content are segmented using jieba word segmentation, and the keyword matching score is calculated and ranked using the BM25 algorithm. Finally, the Top N similar fragment blocks are obtained based on the ranking results, and the coordinates of the corresponding fragment blocks are obtained and displayed in color.

[0055] For example, the similarity score can be determined using the following formula.

[0056]

[0057] in, This represents the final relevance score between the original document D and the search term Q. This represents the i-th word in the search term Q. Indicator In the original file D, the term frequency is represented by DL, which indicates the length (number of words) of the text description in the original file. This indicates the average length (number of words) of the text description in each text block. Indicator Inverse document frequency, 'b' is a parameter that adjusts the contribution of word frequency to the similarity score (controlling the word frequency saturation effect), and 'b' is a parameter that adjusts the degree of score correction by document length (balancing the fairness of scoring between long and short documents), which can be 1.2 and 0.75.

[0058] This invention provides a review method that involves: acquiring an original file; performing layout detection on the original file to determine the content blocks, their types, and their layout order; generating text corresponding to each content block based on its type using a corresponding recognition model; concatenating the text of each content block into a text file according to their layout order; determining the items to be reviewed in the text file and the review rules for each item; reviewing the items according to the review rules and outputting the review results. This invention, through layout detection and multi-model collaborative recognition, converts the original file into a text file to determine the items to be reviewed, reviews the items according to the review rules, and outputs the review results, significantly improving the automation and accuracy of the review process.

[0059] It should be noted that, for the sake of simplicity, the method embodiments are all described as a series of actions. However, those skilled in the art should understand that the embodiments of the present invention are not limited to the described order of actions, because according to the embodiments of the present invention, some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions involved are not necessarily essential to the embodiments of the present invention.

[0060] Reference Figure 5 The diagram shows a structural block diagram of a review device provided by an embodiment of the present invention, which may specifically include the following modules: The original file acquisition module 301 is used to acquire the original file; The layout detection module 302 is used to perform layout detection on the original file to determine the content blocks in the original file, the type of the content blocks, and the layout order of the content blocks; The text generation module 303 is used to call the corresponding recognition model according to the type of the content block to generate the text corresponding to the content block; The splicing module 304 is used to splice the text corresponding to each content block into a text file according to the layout order of the content blocks; The review item determination module 305 is used to determine multiple items to be reviewed in a text file, as well as the review rules corresponding to the multiple items to be reviewed. The review module 306 is used to review the multiple projects to be reviewed according to the review rules corresponding to the multiple projects to be reviewed, and output the review results.

[0061] In one embodiment, the review module includes: The review submodule is used to input the projects to be reviewed into the review model, so that the review model can review each project according to the review rules corresponding to each project.

[0062] In one embodiment, the review rules include review points and review parameters corresponding to the review points; the review submodule includes: The review unit is used to input the project to be reviewed into the review model, determine the project parameters and project conclusions of the project to be reviewed through the review model, compare the project parameters of the project to be reviewed with the review parameters to obtain a first comparison result, and compare the project conclusions of the project to be reviewed with the review points to obtain a second comparison result; output the review conclusions and review basis based on the first comparison results and the second comparison results; the review basis is an analysis and explanation of the review conclusions inferred from the review points corresponding to the project and the parameters corresponding to the review points.

[0063] In one embodiment, the review unit includes: The first review subunit is used to output the review conclusion that the project conforms to the rules and the review basis corresponding to the project conforming to the rules when the first comparison result is that the project parameters are consistent with the review parameters and the second comparison result is that the project conclusion is consistent with the review points. The second review subunit is used to output the review conclusion that the project does not comply with the rules, and the review basis corresponding to the project not complying with the rules, when the first comparison result is that the project parameters are inconsistent with the review parameters, and / or the second comparison result is that the project conclusion is inconsistent with the review points.

[0064] In one embodiment, the method further includes: The search term acquisition module is used to acquire search terms; The target block determination module is used to determine multiple target blocks in the content blocks of the original file that correspond to the search term; A marking module is used to mark the target block.

[0065] In one embodiment, the target block determination module includes: The semantic determination submodule is used to determine the semantics of the text corresponding to the content blocks of the original file; A similarity calculation module is used to calculate the semantic similarity between the semantic term and the search term; The target block determination submodule is used to determine content blocks with semantic similarity greater than a similarity threshold as target blocks.

[0066] In one embodiment, the marking module includes: The similarity level determination submodule is used to determine the similarity level corresponding to the semantic similarity of the multiple target blocks; The tagging submodule is used to tag target blocks with different similarity levels using different colors.

[0067] This invention provides a review method that involves: acquiring an original file; performing layout detection on the original file to determine the content blocks, their types, and their layout order; generating text corresponding to each content block based on its type using a corresponding recognition model; concatenating the text of each content block into a text file according to their layout order; determining the items to be reviewed in the text file and the review rules for each item; reviewing the items according to the review rules and outputting the review results. This invention, through layout detection and multi-model collaborative recognition, converts the original file into a text file to determine the items to be reviewed, reviews the items according to the review rules, and outputs the review results, significantly improving the automation and accuracy of the review process.

[0068] As the device embodiment is basically similar to the method embodiment, the description is relatively simple, and relevant parts can be found in the description of the method embodiment.

[0069] This invention also provides an electronic device, comprising: It includes a processor, a memory, and a computer program stored in the memory and capable of running on the processor. When the computer program is executed by the processor, it implements the various processes of the above-described review method embodiments and achieves the same technical effect. To avoid repetition, it will not be described again here.

[0070] This invention also provides a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, it implements the various processes of the above-described review method embodiments and achieves the same technical effect. To avoid repetition, it will not be described again here.

[0071] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.

[0072] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, apparatus, or computer program products. Therefore, embodiments of the present invention can take the form of entirely hardware embodiments, entirely software embodiments, or embodiments combining software and hardware aspects. Furthermore, embodiments of the present invention can take the form of computer program products implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0073] Embodiments of the present invention are described with reference to flowchart illustrations and / or block diagrams of methods, terminal devices (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing terminal device to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing terminal device, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0074] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing terminal device to operate in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0075] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal equipment, causing a series of operational steps to be performed on the computer or other programmable terminal equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable terminal equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0076] Although preferred embodiments of the present invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of the embodiments of the present invention.

[0077] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or terminal device that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or terminal device. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or terminal device that includes said element.

[0078] The above provides a detailed description of the evaluation method and evaluation device provided by the present invention. Specific examples have been used to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only for the purpose of helping to understand the method and core ideas of the present invention. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of the present invention. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A review method, characterized in that, The method includes: Obtain the original file; The original file is subjected to layout detection to determine the content blocks in the original file, the type of the content blocks, and the layout order of the content blocks; Based on the type of the content block, the corresponding recognition model is invoked to generate the text corresponding to the content block; Based on the layout order of the content blocks, the text corresponding to each content block is concatenated into a text file; Identify multiple items to be reviewed in a text file, and the review rules corresponding to the multiple items to be reviewed; The multiple projects to be reviewed are reviewed according to the review rules corresponding to them, and the review results are output.

2. The review method according to claim 1, characterized in that, The review process, which involves evaluating each project according to its corresponding review rules, includes: The projects to be reviewed are input into the review model, so that the review model reviews each project according to the review rules corresponding to each project.

3. The review method according to claim 1, characterized in that, The review rules include review points and review parameters corresponding to the review points; the step of inputting the projects to be reviewed into the review model, so that the review model reviews each project according to the review rules corresponding to each project, includes: After inputting the project to be reviewed into the review model, the project parameters and project conclusions of the project to be reviewed are determined through the review model. The project parameters of the project to be reviewed are compared with the review parameters to obtain a first comparison result, and the project conclusions of the project to be reviewed are compared with the review points to obtain a second comparison result. Based on the first comparison result and the second comparison result, the review conclusions and review basis are output. The review basis is an analysis and explanation that infers the review conclusions based on the review points corresponding to the project and the parameters corresponding to the review points.

4. The review method according to claim 3, characterized in that, The step of outputting review conclusions and review criteria based on the first comparison result and the second comparison result includes: If the first comparison result shows that the project parameters are consistent with the review parameters, and the second comparison result shows that the project conclusion is consistent with the review points, the output review conclusion is compliance with the rules, and the review basis corresponding to the project's compliance with the rules is also provided. If the first comparison result is that the project parameters are inconsistent with the review parameters, and / or the second comparison result is that the project conclusion is inconsistent with the review points, the output review conclusion is that the project does not comply with the rules, and the review basis corresponding to the project not complying with the rules is provided.

5. The review method according to claim 1, characterized in that, The method further includes: Get search terms; Identify multiple target blocks in the content blocks of the original file that correspond to the search term; The target block is marked.

6. The review method according to claim 5, characterized in that, The step of determining multiple target blocks in the content blocks of the original file that correspond to the search term includes: Determine the semantics of the text corresponding to the content blocks of the original file; Calculate the semantic similarity between the semantic term and the search term; Content blocks with semantic similarity greater than the similarity threshold are identified as target blocks.

7. The review method according to claim 6, characterized in that, The marking of the target block includes: Determine the similarity level corresponding to the semantic similarity of the multiple target blocks; Target blocks with different similarity levels are marked with different colors.

8. A review device, characterized in that, The device includes: The original file acquisition module is used to acquire the original file. The layout detection module is used to perform layout detection on the original file to determine the content blocks in the original file, the type of the content blocks, and the layout order of the content blocks; The text generation module is used to call the corresponding recognition model according to the type of the content block to generate the text corresponding to the content block; The splicing module is used to splice the text corresponding to each content block into a text file according to the layout order of the content blocks; The review project determination module is used to determine multiple projects to be reviewed in a text file, as well as the review rules corresponding to the multiple projects to be reviewed. The review module is used to review the multiple projects to be reviewed according to the review rules corresponding to the multiple projects to be reviewed, and output the review results.

9. An electronic device, characterized in that, include: A processor, a memory, and a computer program stored in the memory and capable of running on the processor, wherein the computer program, when executed by the processor, implements the steps of the review method as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, A computer program is stored on the computer-readable storage medium, which, when executed by a processor, implements the steps of the review method as described in any one of claims 1-7.

Images