Data verification method, data verification device and storage medium

Abstract

The invention relates to a data verification method, a data verification device and a storage medium. The method comprises the following steps: firstly, through semantic analysis, establishing a mapping relationship between a first position corresponding to first text data in an editable format in a to-be-verified first document and a second position of second text data in a target format in a second document, so that an original to-be-verified text and a to-be-verified text after format conversion form a position corresponding relationship; then, verifying the second document through semantic analysis, and taking second text data which does not pass verification in the second document as annotation data; and finally, based on the mapping relationship, determining the labeling position of the labeling data in the first document, and generating an editable error prompt at the labeling position. Therefore, the time for manually searching the error position of the original document is shortened, and the verification and correction efficiency is improved, so that the data verification efficiency is improved.

Description

Technical Field

[0001] This application relates to the field of data processing technology, specifically to a data verification method, a data verification device, and a storage medium. Background Technology

[0002] Data validation refers to the process of verifying the accuracy, completeness, consistency, validity, and compliance of data during data processing to ensure that the data meets preset standards. This directly affects the reliability of the data and the effectiveness of its subsequent applications. Traditional data validation typically employs deep learning information extraction combined with probabilistic model comparison for analysis. However, since the same type of indicator may have different descriptions, involving numerous semantic understanding aspects, traditional models struggle to handle the flexible and varied scenarios. Furthermore, traditional data validation, such as consistency checks, requires extensive human verification after extracting and comparing indicator values. Traditional models usually provide probabilistic conclusions, lacking analytical mechanisms and reasons, necessitating human intervention for checking and verification. Therefore, traditional data validation is time-consuming and prone to omissions, resulting in low efficiency. Summary of the Invention

[0003] The purpose of this application is to provide a data verification method, a data verification device, and a storage medium to solve the problem of low efficiency in traditional data verification.

[0004] To achieve the above objectives, the first aspect of this application provides a data verification method, comprising: Retrieve the editable first text data from the first document to be verified; The first text data is converted into second text data in a target format to generate a second document, wherein the target format is a structured format that meets the conditions for semantic analysis. Based on the semantic relationship between the first document and the second document, a mapping relationship is established between the first position of the first text data in the first document and the second position of the second text data in the second document; The second document is validated through semantic analysis, and the second text data that fails the validation in the second document is used as labeled data. Based on the mapping relationship, the annotation position of the annotation data in the first document is determined, and an editable error message is generated at the annotation position.

[0005] A second aspect of this application provides a data verification device, comprising: The acquisition module is used to acquire editable first text data from the first document to be verified. A conversion module is used to convert the first text data into second text data in a target format to generate a second document, wherein the target format is a structured format that meets the semantic analysis conditions; The mapping module is used to establish a mapping relationship between the first text data at a first position in the first document and the second text data at a second position in the second document based on the semantic relationship between the first document and the second document; The annotation module is used to validate the second document through semantic analysis and to use the second text data in the second document that fails the validation as annotation data. The prompting module is used to determine the annotation position of the annotation data in the first document based on the mapping relationship, and generate an editable error prompt at the annotation position.

[0006] A third aspect of this application provides a computer-readable storage medium storing a program that can be loaded by a processor and executed using the data verification method described above.

[0007] The beneficial effects of this application are: This application first uses semantic analysis to establish a mapping relationship between the first position of editable first text data in the first document to be verified and the second position of target-format second text data in the second document, thus creating a positional correspondence between the original text to be verified and the converted text to be verified. Then, semantic analysis is used to verify the second document, and the second text data that fails verification is used as annotation data. Finally, based on the mapping relationship, the annotation position of the annotation data in the first document is determined, and an editable error message is generated at the annotation position. This automates the entire process of format conversion, relationship mapping, semantic verification judgment, and error annotation, reducing manual annotation and location steps, thereby reducing false positives and false negatives caused by human error. Furthermore, based on the mapping relationship between the first and second positions, an editable error message is directly generated in the first document according to the verification result of the second document. Users can directly modify the data based on the editable error message at the annotation position, reducing the time spent manually searching for error positions in the original document and improving verification and correction efficiency. This improves the efficiency of data verification.

[0008] Other features and advantages of this application will be described in detail in the following detailed description section. Attached Figure Description

[0009] Figure 1 This is a flowchart illustrating a data verification method provided in an embodiment of this application; Figure 2 This is a flowchart illustrating a data verification method provided in a specific embodiment of this application; Figure 3 This is a schematic diagram of the structure of a data verification device provided in the embodiments of this application; Figure 4 This is a schematic diagram of the structure of a data verification system provided in an embodiment of this application. Detailed Implementation

[0010] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0011] In the description of this application, it should be understood that the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the stated features. In the description of this application, "a plurality of" means two or more, unless otherwise explicitly specified. In this application, the term "exemplary" is used to mean "used as an example, illustration, or description." Any embodiment described as "exemplary" in this application is not necessarily to be construed as being more preferred or advantageous than other embodiments. The following description is provided to enable any person skilled in the art to implement and use this application. In the following description, details are set forth for illustrative purposes. It should be understood that those skilled in the art will recognize that this application can be implemented without using these specific details. In other instances, well-known structures and processes will not be described in detail to avoid unnecessary detail that would obscure the description of this application. Therefore, this application is not intended to be limited to the embodiments shown, but is consistent with the broadest scope of the principles and features disclosed in this application.

[0012] Figure 1 This is a flowchart illustrating a data verification method provided in an embodiment of this application. Figure 1 As shown, this data verification method may include steps 101-105, which will be described in detail below.

[0013] Step 101: Obtain the first text data in editable format from the first document to be verified.

[0014] In this embodiment, the first document is the original document to be verified, and the first document may include one or more documents related to business scenarios. The first text data is the data in the first document. An editable format refers to a format that allows users to modify the data. Therefore, the first text data is stored in a format that allows users to directly modify text, tables, images, etc. As an example, the first document may include, but is not limited to, Word documents, Excel spreadsheets, and Portable Document Format (PDF). Accordingly, the first text data can be extracted from the first document using methods such as an OpenXML-based Word parser or the Apache POI library.

[0015] In addition, during the extraction of the first text data, the physical location of the first text data in the first document, i.e., the first position, can be recorded simultaneously, such as the XML node of a paragraph in a Word document or the cell coordinates of an Excel table. In this way, the binding of the first text data to the original position can be ensured.

[0016] Step 102: Convert the first text data into second text data in the target format to generate the second document.

[0017] In this embodiment, the second text data is text data converted from editable first text data to a target format, and the document composed of the second text data is the second document. The target format is a structured format that meets the conditions for semantic analysis. The target format is characterized by a clear data structure and machine parsing capability, allowing for subsequent verification of the second document based on semantic analysis. As an example, the target format can be JSON or Markdown. By parsing the nested structure of JSON, data locations can be found using "key paths." Markdown formats typically consist of blocks such as headings, paragraphs, and tables. Therefore, data locations can be found using line numbers and block types.

[0018] Editable documents, designed for ease of user creation, modification, formatting, and presentation, typically do not adhere to uniform machine-readable rules—that is, they lack automated machine parsing logic. Therefore, it is necessary to convert the initial text data, which lacks standardized rules, into a structured format that meets the requirements of semantic analysis. Converting the initial text data to a unified target format provides consistent input for subsequent semantic analysis of the second document, reducing validation omissions caused by data type differences.

[0019] In addition, while constructing the second document, a unique second position can be generated simultaneously for each second document's second text data. The second position is a logical position identifier based on structured grammar, used to uniquely determine the location of the second text data in the second document. For example, when converting a paragraph in a Word document to a JSON field, the path of that field in the JSON can be recorded as the second position; when converting an Excel cell to a Markdown table, the row and column indices can be recorded as the second position.

[0020] Step 103: Based on the semantic relationship between the first document and the second document, establish a mapping relationship between the first text data in the first position of the first document and the second text data in the second position of the second document.

[0021] This application's embodiments enable cross-document positional association through semantic analysis, establishing a mapping relationship between the positions of text data in a first and second document. As an example, a pre-trained language model (such as Sentence-BERT) can be used to encode the first and second text data, obtaining high-dimensional semantic vectors to quantify the semantic information of the text. Then, based on the semantic information, the similarity between the encoded first and second text data is calculated. Text pairs consisting of highly similar first and second text data are considered as text data with a mapping relationship. A mapping relationship is then established between the first position corresponding to the first text data and the second position corresponding to the second text data in this text pair. Establishing the positional mapping between the original text data and the converted text data based on semantics rather than simple format tags reduces the difficulty of associating different expressions of the same content and reduces erroneous associations caused by semantic ambiguity, providing reliable data for subsequent localization.

[0022] Step 104: Verify the second document through semantic analysis, and use the second text data in the second document that fails the verification as labeled data.

[0023] In this embodiment, the second text data in the second document is data in a target format, i.e., a structured format that satisfies the semantic analysis conditions. Therefore, the second document can be validated through semantic analysis to obtain the validation result, and the second text data that fails the validation in the result is used as annotation data. For example, the annotation data can be second text data that does not conform to the validation rules or contains contradictions.

[0024] As an example, we can obtain the metrics to be validated. Metrics are the text data in the second set of text data that needs to be validated, such as a specific field. The metric description is the explanation, description, and supplementary information for the metric. Validation is performed through semantic understanding rather than simple string matching, enabling the handling of flexibly expressed text data. For example, the hydrogen ion concentration index can be called "pH value" or "acidity / alkalinity," and can serve as the same metric description. "Greater than or equal to 5" and "≥5" can also serve as the same metric description.

[0025] As an example, the validation of second documents includes, but is not limited to, various complex scenarios. For instance, the validation of second documents may include consistency checks on the metric descriptions corresponding to the same metric, or validation of the compliance of the metric descriptions. In this way, validation can meet the needs of different complex scenarios, reducing false positives and false negatives.

[0026] Step 105: Based on the mapping relationship, determine the annotation position of the annotation data in the first document, and generate an editable error message at the annotation position.

[0027] In this embodiment, the second location of the annotation data can be found in the second document first. Then, based on the pre-built mapping relationship between the first and second locations, the first location of the annotation data in the original first document is determined. The first location of the annotation data in the first document is the annotation location. Simultaneously, an editable error message can be generated at the annotation location. An editable error message is a prompt message that allows users to perform editing operations such as modification, deletion, and addition. For example, in a Word document, an error message can be displayed in revision mode, allowing users to edit the annotation data at the annotation location based on the error message. As an example, modifications to the first document can be automatically synchronized to the second document, thus achieving synchronous updates between documents of different formats. By associating the error message with the original document location, users do not need to switch between the two documents, reducing the time spent locating errors. Furthermore, inheriting corrections and references through editable error messages reduces the cost of manual judgment, improves correction efficiency, and reduces secondary errors caused by incomplete information.

[0028] This application addresses the core pain points of traditional data validation—such as difficulty in cross-format correlation, slow error location, weak multimodal processing, and low correction efficiency—through a chain of semantic mapping, semantic verification, and editable error prompts. It achieves end-to-end automation from data extraction to error correction. Compared to traditional solutions, data validation efficiency is significantly improved, error location accuracy is increased, and manual costs are reduced. Users can directly modify errors based on the marked error prompts, reducing the time spent manually searching for error locations in the original document and improving validation and correction efficiency. This enhances data validation efficiency and is applicable to fields with extremely high data standardization requirements, such as healthcare, pharmaceuticals, and finance, demonstrating strong practical value and industry adaptability.

[0029] In this embodiment, the first text data may include various types of data, such as text data, table data, or image data. Different extraction methods can be used for different types of first text data. Therefore, in step 102, the first text data can first be parsed to separate the first text data, the first table data, and the first image data. The first text data, the first table data, and the first image data are respectively the text-formatted data, table-formatted data, and image-formatted data separated from the first text data.

[0030] As an example, the first text data can be formatted using a document parsing engine (such as Apache POI, Tesseract OCR, etc.), splitting the content according to data type characteristics, for example, based on XML tags or typesetting rules. For the first text data, plain text paragraphs, headings, and comments can be extracted; for the first table data, the table's row, column, and cell structure can be identified, and the correspondence between table headers and data can be preserved; for the first image data, the image file path or binary data can be located, and its position within the first document can be recorded. By recording the position information of each data point, data support can be provided for subsequent mapping relationships. Then, different types of data can be processed using different processing methods.

[0031] The first text data and the first table data can be structured to obtain the first structured data in the target format. The first structured data is the structured data obtained through structured processing. In one example, for the first text data, long texts with the same semantic theme can be split into independent units according to semantic segmentation, and key information, i.e., indicators, can be labeled using named entity recognition technology. Then, logical relationships between entities can be established, for example, "drug name" corresponds to "specification parameters". In another example, the header data of the first table data can be associated to obtain the first table data in "field-value" format, and then the text values ​​can be converted into structured data such as numbers and dates. Through structured processing, the first text data and the first table data can be converted into a format that can be processed by the semantic analysis model, improving the efficiency of automatic verification. Furthermore, unifying the expression format of text and tables, for example, converting them all to JSON objects, facilitates subsequent verification and comparison operations.

[0032] For the first image data, the first image data and the first cue word can be input into a Multimodal Large Language Model (MLLM) for recognition. Then, the second text data is extracted from the first image data and processed into structured data to obtain the second structured data in the target format. The first cue word is a text instruction that prompts the MLLM to perform recognition, guiding it to generate the required output; it can be a question, description, or task instruction. The second structured data is the structured data obtained through MLLM processing.

[0033] Taking the image data of a user's handwritten record as an example, handwritten record image data is usually pre-processed using an Optical Character Recognition (OCR) model. However, handwritten data is diverse in style, complex in environment, and arbitrary in layout, while OCR is mainly trained on printed data, which has insufficient coverage of handwritten data. Therefore, the recognition accuracy of handwritten record data processed by the OCR model cannot be guaranteed. Based on this, the embodiments of this application can achieve a deep understanding of handwritten record data by fusing visual and linguistic information through MLLM. MLLM is a Large Language Model (LLM) that integrates multiple modal information processing capabilities. On the basis of the traditional text understanding and generation capabilities of LLM, it further expands the data processing capabilities of non-text modalities such as images, audio, and video, and can realize cross-modal information fusion, understanding, and reasoning.

[0034] Specifically, the first image data and the first prompt word can be input into the MLLM. The MLLM first extracts visual features, for example, by using a CNN to identify the image features of the text in the first image data, generating visual vectors. Then, the visual vectors are fused with the textual knowledge of a pre-trained language model (such as BERT), injecting linguistic knowledge. Next, the recognition results are validated through multiple rounds of inference using generative capabilities, outputting the validation results and corresponding confidence scores. If the recognition results contradict other information, a second validation by the MLLM can be triggered. Then, the second text data is output, and the extracted second text data undergoes a structured transformation.

[0035] MLLM can not only recognize text, but also understand the semantic relationships in images, such as the relationship between label text and graphics, thereby improving the recognition accuracy of image data. It also solves the problem that traditional verification tools struggle to handle image content, making it particularly suitable for documents containing a large number of images, such as medical reports and engineering drawings.

[0036] Finally, the first and second structured data are merged to obtain the second text data in the target format. This integrates different types of first text data into a unified target format, facilitating subsequent semantic analysis model validation. As an example, related fields can be merged based on entity associations to achieve data alignment. When data inconsistencies occur, conflicts can be flagged while preserving the original data source. Then, the final second text data is obtained according to the syntax rules of the target format.

[0037] In step 103, firstly, the first position corresponding to the first text data is extracted in the first document, and the second position corresponding to the second text data is extracted in the second document. As an example, the physical position of the first text data, i.e., the first position, can be located in the first document. This can be achieved by using an XML node path to represent the physical position of the first text data, by using row and column coordinates to represent the physical position of the first table data, or by using anchor points or bounding box coordinates to locate the physical position of the first image data. As another example, the logical position of the second text data, i.e., the second position, can be located in the second document. This can be achieved by using a JSONPath expression or an XPath path to represent the logical position of the second text data.

[0038] Then, the first text data is encoded to obtain a first semantic vector, and the second text data is encoded to obtain a second semantic vector. As an example, a pre-trained language model can be used to convert text into high-dimensional vectors, thus enabling the text data to be converted into a computable distance representation.

[0039] Next, the similarity between the first and second semantic vectors is calculated to generate matching pairs. Each matching pair can include a first semantic vector and a second semantic vector that have a mapping relationship. As an example, cosine similarity can be used to calculate the similarity between different semantic vectors, with the result ranging from -1 to 1, where a higher value indicates greater semantic similarity. Then, based on a set similarity threshold, semantic vectors with similarity values ​​higher than the threshold are retained as matching pairs, generating a set of matching pairs. Each matching pair can include a first semantic vector and a second semantic vector.

[0040] Finally, the positions corresponding to the semantic vectors in the matching pairs are mapped. Specifically, the first position corresponding to the first semantic vector in each matching pair is mapped to the second position corresponding to the second semantic vector, establishing a mapping relationship between the first position of the first text data in the first document and the second position of the second text data in the second document. This establishes a positional mapping relationship between the original document and the structured document, enabling bidirectional tracing. If a validation failure occurs during subsequent validation of the second document, the mapping table can quickly locate the specific position in the original first document, improving the efficiency of validation location and correction. Furthermore, semantic-driven mapping, compared to simple format matching, overcomes the limitations caused by fixed templates in traditional methods, supports multiple document structures, and can adapt to different validation scenarios.

[0041] In text data similarity assessment, there may be issues where semantically similar data exists but in different contexts. Taking medical records as an example, in medical records, diagnoses and medication recommendations may contain the same names, but format features can distinguish that diagnoses and recommendations belong to different medical processes. Therefore, in step 103, a first structural feature corresponding to the first text data can be extracted from the first document, and a second structural feature corresponding to the second text data can be extracted from the second document. Structural features reflect the formal representation of text data in the document's arrangement or hierarchy, and together with the semantic content of the text data, constitute the complete information of the text. For example, structural features may include at least one of paragraph hierarchy, table row and column numbers, and image boundary coordinates.

[0042] Then, the first structural feature is encoded and fused with the first semantic vector to obtain a first fused vector, and the second structural feature is encoded and fused with the second semantic vector to obtain a second fused vector. Next, the similarity between the first and second fused vectors is calculated to obtain the initial similarity of multiple candidate matching pairs. A candidate matching pair refers to the first and second fused vectors whose semantic content similarity is greater than a set similarity threshold. However, there may be fused vectors with similar semantic content but mismatched structural features. Therefore, it is necessary to adjust the initial similarity according to the matching rules of the first and second structural features, and select candidate matching pairs with matching structural features as matching pairs. In this way, combining semantic content and structural features to determine matching pairs can improve the detection accuracy of matching pairs and reduce false detections.

[0043] Since different descriptions may exist for the same indicator—meaning that the same indicator description may express the same meaning but be presented in different written forms—semantic understanding is involved in many aspects, even considering context and formula calculations. Therefore, this application proposes a method for consistency checking of the indicator descriptions corresponding to the indicator to be verified based on semantic relationships.

[0044] In step 104, initial indicators to be verified can be searched in the second document based on semantic analysis, and identical indicators among the initial indicators can be identified. Initial indicators refer to pre-set indicators that need to be verified. For example, assuming the input indicators to be verified are A, B, and C, and in the second document there exist A1, A2, B1, B2, and C1, which have semantic relationships with A, B, and C respectively, then A1, A2, B1, B2, and C1 can all be used as initial indicators. Based on semantic relationships, A can be identified as the same indicator as A1 and A2, B as the same indicator as B1 and B2, and C as the same indicator as C1.

[0045] Then, for the same metric, semantic analysis can be used to determine metric descriptions that have a semantic relationship with the metric. For metric descriptions corresponding to the same metric, mismatched metric descriptions are identified as second text data that failed validation. Here, mismatch refers to metric descriptions having different meanings or containing contradictions. This can be determined through semantic analysis, and mismatches may include inconsistent units, numerical values ​​exceeding thresholds, or semantic conflicts in description.

[0046] For example, suppose the indicator to be validated is "glucose injection specifications". Through semantic reasoning, the prompt can be set to "Please compare the above indicators, find the inconsistent values ​​and mark them". Assuming "10%" and "20%" can be found in the second document, comparing "10%" and "20%" can determine that these two indicator descriptions are inconsistent, and the validation fails. As an example, the indicator to be validated and the prompt can be input into an LLM with semantic analysis capabilities to obtain the descriptions of the indicators that failed validation. By determining whether the descriptions of the same indicator are synonyms or near-synonyms through semantic association, more implicitly related indicator descriptions can be covered without predefining a thesaurus, thus improving the accuracy of validation. Furthermore, the same indicator may appear in different locations in a document or in a batch of different documents. Through semantic search, all corresponding indicator descriptions can be associated, reducing the possibility of inconsistent descriptions of the same indicator in different places and improving consistency validation across paragraphs, fields, or documents.

[0047] In one example, a target model with semantic analysis capabilities can be used to search for the metric description corresponding to the metric to be validated. For example, the target model could be an LLM (Liquid Language Model), which can generate natural language text and understand the meaning of language text.

[0048] Specifically, the second document is first segmented to obtain segmented data. Each segment contains continuous second text data with a consistent semantic topic. In other words, the second document is divided into continuous segments according to semantic topics. As an example, a sliding window can be used for detection. For instance, Sentence-BERT can be used to calculate the semantic similarity between adjacent sliding windows. When the similarity is less than a set similarity threshold, it can be used as the position to segment the text, thus obtaining multiple continuous segments with a consistent semantic topic.

[0049] Then, a label is added to each data segment, including a unique identifier and a second position of the segment. The segmented data can be used as input to the target model with structured instructions. As an example, each data segment can be uniquely identified using document hash and segment sequence number. In this way, segmentation can reduce the performance degradation of long text in LLM and lower the computational cost of LLM. Furthermore, adding a unique identifier and a second position facilitates the location of subsequent validation results.

[0050] Next, the segmented data is added to the target model's contextual structured text instructions, and second prompt words are constructed based on the metrics to be validated. The second prompt word is a text instruction that prompts the LLM to perform recognition, guiding it to generate the required output. It can be a question, description, or task instruction. The second structured data is structured data obtained through LLM processing. For example, the second prompt word could be "Find the numerical description of 'glucose injection specifications' in the following text." Here, "glucose injection specifications" is a dynamically populated metric. By dynamically populating the second prompt word based on the segmented data and the metrics to be validated, only one template is needed for multiple populations to generate the corresponding second prompt words. This eliminates the need to develop code based on new metrics, improving the prediction efficiency of the target model.

[0051] Finally, semantic analysis is performed on the segmented data based on the second cue word to obtain the indicator descriptions that have semantic relationships with the indicator. LLM can capture all indicator descriptions that have semantic relationships with the indicator in a single traversal based on the input second cue word, without the need for multiple regular expressions, reducing the workload of manual review. In addition, the second position carried can be easily mapped back to the first document, achieving efficient positioning.

[0052] For the same indicator, different users may input different description formats. For example, suppose the search results for semantic descriptions based on "glucose injection specifications" include {"glucose injection specifications": "250ml:25g"} and {"glucose injection specifications": "20%"}. Therefore, it is necessary to unify the semantic descriptions to the same format for easier subsequent comparisons. Based on this, embodiments of this application can set preset format rules for the description format of indicator descriptions. For example, for glucose injection specifications, it is necessary to present them as percentages. Therefore, embodiments of this application can introduce a conversion tool to unify the description format of indicator descriptions to the preset format rules. As an example, since preset format rules may include rules for unified numerical units, rules for significant digits, or rules for symbol standardization, the conversion tool may include at least one of a unit conversion module and a numerical normalization module.

[0053] Specifically, in step 104, the description format in the indicator description can be obtained first. Then, it is determined whether the description format conforms to the preset format rules. If the description format does not conform to the preset format rules, a conversion tool is called to convert the description format to a format that conforms to the preset format rules. For example, to convert the unit of the indicator description, a unit conversion tool can be called to perform the conversion, maintaining the consistency of unit measurement in the unified parameter specification.

[0054] Traditional indicator verification typically focuses on improving accuracy by optimizing the model itself, making it difficult to incorporate common-sense information. If the documents being reviewed contain common-sense errors that don't fit the business context, they are difficult to detect. For example, in the pharmaceutical industry, if the text data to be verified is "glucose," but the text data to be validated contains "bubble gum," while "bubble gum" conforms to grammatical rules, it doesn't fit the current business context. Traditional indicator verification would struggle to recognize "bubble gum" as a failed verification.

[0055] Therefore, embodiments of this application can construct a knowledge base corresponding to a business scenario. In step 104, a pre-constructed knowledge base can be obtained first. This knowledge base can include the mapping relationship between sample indicators, description rules, and reference documents. For example, for the pharmaceutical industry, pharmaceutical expertise can be stored in the knowledge base. For instance, pharmacopoeias and other standards can be parsed using XML and converted into Markdown format files, and then vectorized using an embedding model through code slicing. Then, the vectorized content is stored in a vector database to obtain the knowledge base. If the reference documents in the knowledge base have version updates, the upgraded reference documents can be parsed using XML, and then the increment can be written into the rule table, automatically upgrading the version number.

[0056] Then, based on the indicator, the system searches the knowledge base for the corresponding description rules and determines whether the indicator description meets the rules. As an example, Retrieval-Augmented Generation (RAG) technology can be used to check if the indicator description conforms to the rules. For instance, the description rule can be converted into a query vector. Then, vector comparison is performed in the knowledge base to find the matching paragraphs from the most similar reference documents. Next, using LLM as another example, the indicator description can be extracted, the indicator text vectorized using Sentence-BERT, rules searched in the knowledge base, and the rule's identifier number returned. This identifier number corresponds to a canonical value for the indicator description. Then, the canonical value corresponding to the rule's identifier number is matched against the indicator description. If the indicator description does not match the canonical value, it is determined that the indicator description does not meet the description rule. Finally, the indicator descriptions that do not meet the description rules are identified as the second text data that failed validation, and a reference document corresponding to the description rule is generated at the marked position in the indicator description. In this way, the reason for the validation error and the corresponding reference document can be directly presented in the original document, facilitating the correction of text data based on the reference document.

[0057] In step 105, the second position of the annotated data in the second document can be determined first. That is, the position of the second text data that failed verification in the second document. Then, based on the mapping relationship between the first and second positions, the first position of the annotated data in the first document is used as the annotation position. In this way, the verification result can be accurately presented in the corresponding position of the first document. Finally, an error message is embedded in the annotation position in the form of an editable annotation. This error message may include, but is not limited to, error type, error description, reference source, and correction suggestions. Thus, users can check the correctness of the data content based on the editable errors displayed in the first document and can directly perform correction operations in the original first document based on the error messages. This embodiment of the application, after data verification, can directly provide error reminders and correction suggestions in the original document, reducing the time and cost of manual confirmation, reducing human error and false positives, and improving the efficiency of data verification and correction.

[0058] Figure 2 This is a flowchart illustrating a data verification method provided in a specific embodiment of this application. Figure 2 As shown, taking data validation in the pharmaceutical industry as an example, with the first document being a Word document and the second document being in Markdown format, this data validation method includes steps 201-208.

[0059] Step 201: Construct a knowledge base for the pharmaceutical industry. For example, knowledge such as pharmacopoeias, operating procedures, and Good Manufacturing Practices (GMP) for pharmaceutical products is segmented into a vector library using embeddingchuck and used as the documents to be searched.

[0060] Step 202: Extract the first text data of the original Word document using OpenXML. Select the Word document to be checked, and use OpenXML technology to extract all the text content of the existing Word document.

[0061] Step 203: Convert the first text data into second text data in Markdown format. Convert the extracted text content into Markdown format that LLM can understand, and map the text paragraphs and tables in Markdown to the positions of paragraphs and tables in XML, and mark the source of the file.

[0062] Step 204: Content Paragraph Segmentation and Semantic Extraction of Metrics. The segmented Markdown paragraphs and tables are further segmented. Using LLM semantic understanding capabilities, preliminary metric lookup and value extraction are performed based on the given metrics. If the original file is a scanned document or an image, the MLLM model is used directly for lookup and parsing.

[0063] Step 205: Perform common-sense standardization checks on the extracted metrics using Retrieval-Augmented Generation (RAG) technology. Based on the extracted metric names and values, use RAG semantic augmentation vector retrieval to search the knowledge base content to determine if the metrics conform to industry common-sense standards. If the retrieved knowledge related to the metric is confirmed to be non-compliant, render it back to Word using file and line number mappings to provide a prompt, and cite the knowledge base source for annotation.

[0064] Step 206: Perform unit calculations and unit conversions on the extracted indicator descriptions. Continue to analyze the indicator semantics and check whether the units and descriptions of the same indicator are consistent (e.g., units: mg and ml). If the indicator units are inconsistent, the large model calls a unit conversion tool or other indicator calculation tool through the Model Context Protocol (MCP) to perform unit conversion, ensuring that the indicator units are at the same granularity.

[0065] Step 207: Perform a consistency check using LLM and annotate the original Word document. After standardizing the unit descriptions of the indicators, construct prompt words and use LLM's semantic understanding capabilities to analyze and compare the consistency of the same indicator sequentially. If there are discrepancies or differences in the descriptions, the annotations are manipulated through file and line number mapping positions and rendered back into Word to prompt the user that there are inconsistencies in the indicators. The unit conversion process and the reasons for the inconsistencies are also annotated for the user's reference.

[0066] Step 208: The user confirms the correctness of the indicator verification and makes calibration modifications. If the user confirms any inconsistencies in the indicators, they manually correct and modify them in the annotation.

[0067] The following example illustrates this. Suppose we need to check the accuracy of all procedural documents for a drug, including: pharmacopoeia, GMP specifications, process information sheets, job operation manuals, process procedures, and drug quality standards and testing operation procedures. The detailed implementation method is as follows.

[0068] First, the pharmaceutical knowledge base is stored in the database. The pharmacopoeia and GPM specifications are parsed using XML and converted into Markdown format files. Then, the code is sliced ​​and the slices are vectorized using the embedding model. The vectorized content is then stored in the pgvector vector database. For example, the paragraph in the pharmacopoeia: "Specifications of glucose injection: (1) 10ml: 1g; (2) 10ml: 2g; (3) 10ml: 5g; (4) 20ml: 5g; (5) 20ml: 10g; (6) 50ml: 2.5g; (7) 50ml: 5g; (8) 100ml: 5g; (9) 100ml: 10g; (10) 100ml: 50g; (11) 200ml: 10g; (12) 250ml: 12.5g; (13) 250ml: 25g; (14) 250ml: 50g; (15) 250ml: 62.5g; (16) 250ml: 100g; (17) 250ml: 125g; (18) 300ml: 15g; (19) 500ml: 25g; (20) 500ml: 50g; (21) 500ml: 125g; (22) 1000ml: 50g; (23) 1000ml: 100g; (24) 1000ml: 250g” After vectorization, it is put into the database.

[0069] Next, the content of the original Word documents was extracted. The Word documents containing process information tables, job operation manuals, process procedures, and drug quality standards and inspection operation procedures were extracted using OpenXML technology, outputting JSON structures of text and tables, and then converted to Markdown, with positional markers added to the Markdown content.

[0070] Next, the content is segmented into paragraphs. The extracted Markdown content is input into the LLM (Local Modeling Language) for semantic recognition. Paragraphs with the same content are segmented for LLM recognition. The purpose of this step is mainly because the contextual semantic length is limited in large modeling techniques, so the segmentation method is used for batch recognition. For indicator semantic extraction, the segmented content can be added to the context prompt of the large model. The user outputs the indicator to be checked, such as "glucose injection specifications". The large model is then used to construct a second prompt, such as "If the user wants to query <glucose injection specifications>, please find the matching content in the following paragraphs and provide the corresponding value...".

[0071] After LLM inference, the resulting indicators are assumed to be {"Glucose Injection Specification": "250ml:25g"} and {"Glucose Injection Specification": "20%"}. The prompt message "Determine if the units of measurement for the following indicators are consistent; if not, call the 'Unit Conversion Tool' for conversion" is used. At this point, the unit 250ml:25g is converted using the MCP server's tool via the large model, resulting in {"Glucose Injection Specification": "10%"} and {"Glucose Injection Specification": "20%"}, maintaining consistency in unit measurement within the same parameter specification. The MCP toolset can be established by writing a unit conversion tool in code, taking volume and weight as input parameters and concentration as output parameters, and then establishing the MCP server.

[0072] It is known that the pharmacopoeia knowledge base content has already been added in the first step. By using embedding vector retrieval to calculate "glucose injection specifications": "250ml:25g", the original text of the knowledge base is found as follows: "glucose injection specifications: (1) 10ml:1g; (2) 10ml:2g; (3) 10ml:5g; (4) 20ml:5g; (5) 20ml:10g; (6) 50ml:2.5g; (7) 50ml:5g; (8) 100ml:5g; (9) 100ml:10g; (10) 100ml:50g; (11) 200ml:10g; (12) 250ml:12.5g; (13) 250ml:25g; (14) 250ml:50g; (15) 250ml:62.5g; (16)" 250ml: 100g; (17) 250ml: 125g; (18) 300ml: 15g; (19) 500ml: 25g; (20) 500ml: 50g; (21) 500ml: 125g; (22) 1000ml: 50g; (23) 1000ml: 100g; (24) 1000ml: 250g. At this time, the LLM semantic understanding ability is used to determine whether the user indicator meets the standard mentioned in the context by calling the MCP concentration calculation tool to obtain the concentration range of glucose injection between 5% and 50%, and only including the above specifications, using the large model prompt words. Only "yes" and "no" are needed to get the large model to answer "yes", and the indicator common sense problem is checked and passed. If the check fails, the line number of the failed indicator markdown and the corresponding line number of the corresponding Word XML are located, and the XML is edited for annotation. The Word document contains the annotation "Does not conform to the specifications of glucose injection in the Pharmacopoeia".

[0073] Next, a consistency check is performed. After the previous check passes, the indicators and corresponding values ​​are obtained: {"Glucose Injection Specification": "10%"}, {"Glucose Injection Specification": "20%"}. The second prompt in the LLM is "Please compare the above indicators, find the inconsistent values ​​and mark them." At this point, the LLM performs semantic reasoning and finds the different indicators {"Glucose Injection Specification": ["10%", "20%"]}. Then, it starts tracing the source. Through the mapping relationship of file name-line number-markdown line number established in the above steps, it directly traces the source to the XML relationship line number established in the original Word document. The XML position in the Word document is found and annotated with "Indicator comparison results are inconsistent:\nIndicator name:\n《xxx process specification》Glucose sodium injection specification 250ml: 25g\n《xxx job operation procedure》Glucose injection specification 10%".

[0074] Finally, users can check the annotation results. By viewing the Word annotations, they can check the accuracy of the content and manually change "20%" to "10%" in the "Glucose Injection Specifications" section to complete the document consistency check and correction operation.

[0075] This application addresses the problem of low efficiency in verifying multiple numerical indicators in pharmaceutical industry documents. A typical pharmaceutical process document includes process information sheets, job operation procedures, drug process specifications, production batch records, etc., with different documents describing the same indicator in the drug manufacturing process. Examples include pH value, ingredient weight, and sterilization temperature in drug production. After the initial document writing, verifying the accuracy and consistency of the numerical values ​​becomes essential. This application improves the accuracy and consistency verification of indicators in pharmaceutical industry documents through the aforementioned steps.

[0076] It should be noted that although the task involves consistency checks of pharmaceutical industry documents, the technical solutions of this application can be used in high-frequency scenarios involving numerical calculations and contract consistency assessments. For example, this application can also be applied to scenarios such as financial data verification, payroll calculation, personnel performance evaluation, and contract consistency assessments.

[0077] Figure 3 This is a schematic diagram of the structure of a data verification device 300 provided in an embodiment of this application. Figure 3As shown, the data verification device 300 may include an acquisition module 301, a conversion module 302, a mapping module 303, an annotation module 304, and a prompting module 305. The acquisition module 301 acquires editable first text data from a first document to be verified. The conversion module 302 converts the first text data into second text data in a target format to generate a second document; the target format is a structured format that meets semantic analysis conditions. The mapping module 303 establishes a mapping relationship between the first text data at a first position in the first document and the second text data at a second position in the second document, based on the semantic relationship between the first and second documents. The annotation module 304 verifies the second document through semantic analysis, using the second text data that fails verification as annotation data. The prompting module 305 determines the annotation position of the annotation data in the first document based on the mapping relationship and generates an editable error prompt at the annotation position.

[0078] In this embodiment, the conversion module 302 may include a parsing unit, a first structuring unit, a second structuring unit, and a merging unit. The parsing unit parses the first text data, separating it into first text data, first table data, and first image data. The first structuring unit performs structuring processing on the first text data and the first table data to obtain first structured data in the target format. The second structuring unit inputs the first image data and the first prompt word into a multimodal large language model for recognition, extracts the second text data from the first image data, and performs structuring processing on the second text data to obtain second structured data in the target format. The merging unit merges the first structured data and the second structured data to obtain second text data in the target format.

[0079] The mapping module 303 may include a first extraction unit, an encoding unit, a calculation unit, and a matching unit. The first extraction unit extracts a first position corresponding to the first text data in a first document and extracts a second position corresponding to the second text data in a second document. The encoding unit encodes the first text data to obtain a first semantic vector and encodes the second text data to obtain a second semantic vector. The calculation unit calculates the similarity between the first and second semantic vectors, generating matching pairs, each matching pair including a first semantic vector and a second semantic vector with a mapping relationship. The matching unit maps the first position corresponding to the first semantic vector in each matching pair to the second position corresponding to the second semantic vector, establishing a mapping relationship between the first position of the first text data in the first document and the second position of the second text data in the second document.

[0080] The mapping module 303 may further include a first extraction unit and a fusion unit. The first extraction unit is used to extract a first structural feature corresponding to the first text data in the first document, and to extract a second structural feature corresponding to the second text data in the second document. The fusion unit is used to encode the first structural feature and fuse it with a first semantic vector to obtain a first fusion vector, and to encode the second structural feature and fuse it with a second semantic vector to obtain a second fusion vector. The calculation unit is also used to calculate the similarity between the first fusion vector and the second fusion vector to obtain an initial similarity of multiple candidate matching pairs; according to the matching rules of the first and second structural features, the initial similarity is weighted and adjusted, and candidate matching pairs with matching structural features are selected as matching pairs.

[0081] The annotation module 304 includes a format acquisition unit, a judgment unit, a calling unit, a first search unit, a first determination unit, and a second determination unit. The format acquisition unit acquires the description format from the indicator description. The judgment unit determines whether the description format conforms to preset format rules. The calling unit, if the description format does not conform to the preset format rules, calls a conversion tool to convert the description format to conform to the preset format rules. The first search unit searches for the initial indicator to be verified in the second document based on semantic analysis and identifies identical indicators among the initial indicators. The first determination unit, for identical indicators, identifies indicator descriptions that have a semantic relationship with the indicator. The second determination unit, for indicator descriptions corresponding to identical indicators, identifies mismatched indicator descriptions as second text data that failed verification.

[0082] The first determining unit is also used to segment the second document to obtain segmented data, each segment containing continuous second text data with a consistent semantic theme; add a label to each segment, the label including the unique identifier of the segment and the second position of the segment; add the segmented data to the contextual structured text instructions of the target model, and construct second prompt words based on the indicators, the target model being a model with semantic analysis capabilities; perform semantic analysis on the segmented data based on the second prompt words to obtain indicator descriptions that have a semantic relationship with the indicators.

[0083] The annotation module 304 further includes a knowledge base acquisition unit, a second search unit, and a third determination unit. The knowledge base acquisition unit acquires a pre-built knowledge base, which includes a mapping relationship between sample indicators, description rules, and reference files. The second search unit searches the knowledge base for description rules corresponding to the indicators and determines whether the corresponding indicator descriptions satisfy the description rules. The third determination unit identifies indicator descriptions that do not satisfy the description rules as second text data that has failed verification and generates a reference file corresponding to the description rule at the annotation location of the indicator description.

[0084] The prompt module 305 may include a fourth determining unit, a fifth determining unit, and an embedding unit. The fourth determining unit is used to determine the second position of the annotation data in the second document. The fifth determining unit is used to determine the first position of the annotation data in the first document as the annotation position based on the mapping relationship between the first and second positions. The embedding unit is used to embed an error prompt at the annotation position in the form of an editable annotation. The error prompt includes the error type, error description, source of reference documents, and correction suggestions.

[0085] Figure 4 This is a schematic diagram of the structure of a data verification system provided in an embodiment of this application. Figure 3 As shown, the data verification system 400 may include a memory 401 and a processor 402. The memory 401 is configured to store instructions. The processor 402 is configured to retrieve instructions from the memory 401 and, when executing the instructions, to implement the aforementioned data verification method.

[0086] This application also provides a computer-readable storage medium storing a program that can be loaded by a processor and executed by any of the data verification methods described in this application.

[0087] Since the instructions stored in the data verification device, data verification system, and computer-readable storage medium can execute the steps in any of the data verification methods provided in the embodiments of this application, the beneficial effects that any of the data verification methods provided in the embodiments of this application can achieve can be realized. For details, please refer to the previous embodiments, which will not be repeated here.

[0088] Those skilled in the art will understand that all or part of the functions of the various methods in the above embodiments can be implemented by hardware or by computer programs. When all or part of the functions in the above embodiments are implemented by computer programs, the program can be stored in a computer-readable storage medium, which may include: read-only memory, random access memory, disk, optical disk, hard disk, etc., and the program is executed by a computer to achieve the above functions. For example, the program can be stored in the memory of a device, and when the program in the memory is executed by the processor, all or part of the above functions can be achieved. In addition, when all or part of the functions in the above embodiments are implemented by computer programs, the program can also be stored in a server, another computer, disk, optical disk, flash drive, or external hard drive, etc., and can be downloaded or copied to the memory of a local device, or the system of the local device can be updated. When the program in the memory is executed by the processor, all or part of the functions in the above embodiments can be achieved.

[0089] The above examples illustrate this application only to aid understanding and are not intended to limit its scope. Those skilled in the art to which this application pertains can make various simple deductions, modifications, or substitutions based on the ideas presented.

Claims

1. A data verification method, characterized in that, include: Retrieve the editable first text data from the first document to be verified; The first text data is converted into second text data in a target format to generate a second document, wherein the target format is a structured format that meets the conditions for semantic analysis. Based on the semantic relationship between the first document and the second document, a mapping relationship is established between the first position of the first text data in the first document and the second position of the second text data in the second document; The second document is validated through semantic analysis, and the second text data that fails the validation in the second document is used as labeled data. Based on the mapping relationship, the annotation position of the annotation data in the first document is determined, and an editable error message is generated at the annotation position.

2. The data verification method according to claim 1, characterized in that, The step of converting the first text data into second text data in the target format includes: Parse the first text data to separate the first text data, the first table data, and the first image data; The first text data and the first table data are subjected to structuring processing to obtain the first structured data in the target format; The first image data and the first prompt word are input into a multimodal large language model for recognition, the second text data in the first image data is extracted, and the second text data is processed in a structured manner to obtain the second structured data in the target format; The first structured data and the second structured data are merged to obtain the second text data in the target format.

3. The data verification method according to claim 1, characterized in that, The step of establishing a mapping relationship between the first text data at a first position in the first document and the second text data at a second position in the second document based on the semantic relationship between the first document and the second document includes: Extract the first position corresponding to the first text data in the first document, and extract the second position corresponding to the second text data in the second document; The first text data is encoded to obtain a first semantic vector, and the second text data is encoded to obtain a second semantic vector; Calculate the similarity between the first semantic vector and the second semantic vector to generate matching pairs, each of the matching pairs including the first semantic vector and the second semantic vector having a mapping relationship; In each matching pair, the first position corresponding to the first semantic vector and the second position corresponding to the second semantic vector are mapped to establish a mapping relationship between the first position of the first text data in the first document and the second position of the second text data in the second document.

4. The data verification method according to claim 3, characterized in that, The step of establishing a mapping relationship between the first text data at a first position in the first document and the second text data at a second position in the second document based on the semantic relationship between the first document and the second document further includes: Extract a first structural feature corresponding to the first text data from the first document, and extract a second structural feature corresponding to the second text data from the second document; The first structural feature is encoded and fused with the first semantic vector to obtain a first fusion vector, and the second structural feature is encoded and fused with the second semantic vector to obtain a second fusion vector; The step of calculating the similarity between the first semantic vector and the second semantic vector to generate matching pairs includes: Calculate the similarity between the first fusion vector and the second fusion vector to obtain the initial similarity of multiple candidate matching pairs; Based on the matching rules of the first structural feature and the second structural feature, the initial similarity is weighted and adjusted, and the candidate matching pairs with matching structural features are selected as the matching pairs.

5. The data verification method according to claim 1, characterized in that, The step of verifying the second document through semantic analysis includes: Based on semantic analysis, search for the initial metrics to be verified in the second document, and identify the same metrics among the initial metrics; For the same metric, determine the metric description that has a semantic relationship with the metric; Obtain the description format from the indicator description; Determine whether the description format conforms to preset format rules; If the description format does not conform to the preset format rules, then the conversion tool is invoked to convert the description format into a description format that conforms to the preset format rules; For the same indicator corresponding to the indicator description, the mismatched indicator description is determined as the second text data that failed the verification.

6. The data verification method according to claim 5, characterized in that, For the same metric, determining the metric description that has a semantic relationship with the metric includes: The second document is split into fragments to obtain fragment data, and each fragment data contains continuous second text data with a consistent semantic topic. A label is added to each of the data segments, the label including a unique identifier for the data segment and a second position of the data segment; The fragmented data is added to the contextual structured text instructions of the target model, and a second prompt word is constructed based on the indicators. The target model is a model with semantic analysis capabilities. Based on the second prompt word, semantic analysis is performed on the segmented data to obtain the indicator description that has a semantic relationship with the indicator.

7. The data verification method according to claim 5, characterized in that, The step of verifying the second document through semantic analysis also includes: Obtain a pre-built knowledge base, which includes the mapping relationship between sample indicators, description rules and reference documents; Based on the indicator, search the knowledge base for the description rule corresponding to the indicator, and determine whether the indicator description corresponding to the indicator satisfies the description rule; The indicator descriptions that do not meet the description rules are identified as the second text data that failed the verification, and a reference file corresponding to the description rules is generated at the marked position of the indicator description.

8. The data verification method according to claim 1, characterized in that, The step of determining the annotation position of the annotation data in the first document based on the mapping relationship, and generating an editable error message at the annotation position, includes: Determine the location of the labeled data in the second position of the second document; Based on the mapping relationship between the first position and the second position, the first position of the labeled data in the first document is taken as the labeled position; Error messages are embedded in the marked locations in the form of editable comments. The error messages include the error type, error description, source of referenced documents, and suggested corrections.

9. A data verification device, characterized in that, include: The acquisition module is used to acquire editable first text data from the first document to be verified. A conversion module is used to convert the first text data into second text data in a target format to generate a second document, wherein the target format is a structured format that meets the semantic analysis conditions; The mapping module is used to establish a mapping relationship between the first text data at a first position in the first document and the second text data at a second position in the second document based on the semantic relationship between the first document and the second document; The annotation module is used to validate the second document through semantic analysis and to use the second text data in the second document that fails the validation as annotation data. The prompting module is used to determine the annotation position of the annotation data in the first document based on the mapping relationship, and generate an editable error prompt at the annotation position.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a program that can be loaded by a processor and executed as described in any one of claims 1 to 8.

Images