Data verification method and related device
By constructing a bimodal text extraction architecture and data association discrimination model for invoices and medical records, the problem of identifying the correlation between invoices and medical records was solved, enabling accurate identification of abnormal situations, improving the accuracy and efficiency of the automated claims system, and reducing the risk for insurance companies.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-12-23
- Publication Date
- 2026-03-13
AI Technical Summary
Existing technologies cannot effectively identify the correlation between invoices and medical records in outpatient claims, leading to mismatches in automated claims systems or increased manual review costs. Furthermore, traditional optical character recognition technology cannot determine the medical semantic logical correlation between treatment items and disease diagnosis conclusions, increasing the operational risks for insurance companies.
A dual-modal text extraction architecture is constructed, consisting of an invoice information extraction model and a medical record diagnosis extraction model. The data association and discrimination model is used to process the invoice details text and medical record diagnosis text, quantify their logical correlation, and accurately identify abnormal situations where the treatment items and diagnosis conclusions do not match.
It enables the verification of the correlation between invoices and medical records, accurately identifies abnormal situations, reduces the risk of interruption in the automated claims process, reduces manual review costs, improves claims processing efficiency and user experience, and reduces the operational risks of insurance companies.
Smart Images
Figure CN121661661A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of data processing technology, and in particular to a data verification method and related apparatus. Background Technology
[0002] Outpatient claims processing is a crucial link in the operation of the medical security system, and the effectiveness of its risk identification and control directly affects the security of the medical insurance fund. In actual claims review work, some violations manifest as obvious medical logical contradictions between the treatment items and the medical record diagnosis results. For example, the medical record diagnoses a mild illness such as an upper respiratory tract infection, but the corresponding invoice details include unnecessary large-scale examinations such as CT scans and X-rays. Such situations often point to acts such as falsifying medical records, which not only leads to the unreasonable consumption of medical insurance funds but also disrupts the normal order of medical services.
[0003] Therefore, how to verify the correlation between invoices and medical records, and thus accurately identify the aforementioned violations, has become a pressing technical problem to be solved in the field of outpatient claims risk control. Summary of the Invention
[0004] To address the aforementioned issues, this application provides a data verification method for verifying the correlation between invoices and medical records, and for accurately identifying discrepancies between treatment items and diagnostic conclusions.
[0005] The embodiments of this application disclose the following technical solutions:
[0006] The first aspect of this application provides a data verification method, including:
[0007] Acquire the target outpatient invoice image and the target medical record diagnosis image;
[0008] The target outpatient invoice image is processed using an invoice information extraction model to obtain the invoice details text; and the target medical record diagnosis image is processed using a medical record diagnosis extraction model to obtain the medical record diagnosis text.
[0009] The invoice details text and the medical record diagnosis text are processed by a data association discrimination model to obtain a first probability and a second probability; the first probability represents the probability that the target outpatient invoice image and the target medical record diagnosis image match; the second probability represents the probability that the target outpatient invoice image and the target medical record diagnosis image do not match.
[0010] In one optional implementation, the step of processing the invoice detail text and the medical record diagnosis text through a data association discriminant model to obtain a first probability and a second probability includes:
[0011] The invoice details text is encoded using a Sentence-Transformer to obtain a first feature vector; the first feature vector is a 768-dimensional semantic feature vector.
[0012] The medical record diagnosis text is encoded using the Sentence-Transformer to obtain a second feature vector; the second feature vector is a 768-dimensional semantic feature vector.
[0013] Based on the first feature vector and the second feature vector, a target feature vector is generated;
[0014] The target feature vector is processed by a binary classifier to obtain the first probability and the second probability.
[0015] In one optional implementation, generating the target feature vector based on the first feature vector and the second feature vector includes:
[0016] Perform a dot product operation on the first feature vector and the second feature vector to obtain the third feature vector;
[0017] Subtract the first feature vector from the second feature vector to obtain the fourth feature vector;
[0018] The first feature vector, the second feature vector, the third feature vector, and the fourth feature vector are concatenated to obtain the target feature vector.
[0019] In one optional implementation, the steps for obtaining the invoice information extraction model include:
[0020] Obtain multiple sample outpatient invoice images;
[0021] The detailed data in each of the sample outpatient invoice images is annotated to obtain the first annotated data corresponding to each sample outpatient invoice image;
[0022] The sample outpatient invoice image and the corresponding first labeled data are used as the first sample training data; and the LayoutLMv2 is trained using multiple sets of the first sample training data to obtain the invoice information extraction model.
[0023] In one optional implementation, the step of obtaining the medical record diagnosis extraction model includes:
[0024] Acquire diagnostic images from multiple sample medical records;
[0025] The diagnostic conclusions in each of the sample medical record diagnostic images are labeled to obtain the second labeling data corresponding to each of the sample medical record diagnostic images;
[0026] The sample medical record diagnostic images and corresponding second labeled data are used as second sample training data; and the LayoutLMv2 is trained using multiple sets of second sample training data to obtain the medical record diagnostic extraction model.
[0027] In one optional implementation, the steps for obtaining the positive and negative sample pairs corresponding to the data association discrimination model include:
[0028] Multiple sample cases are obtained; the sample cases include sample outpatient invoice images and corresponding sample medical record diagnosis images.
[0029] The sample outpatient invoice image is processed using the invoice information extraction model to generate sample invoice detail text; and the sample medical record diagnosis image is processed using the medical record diagnosis extraction model to generate sample medical record diagnosis text.
[0030] The sample invoice detail text is filtered to obtain the filtered sample invoice text; the filtered sample invoice text does not include the invoice detail items in the fuzzy invoice detail set.
[0031] Based on the three-level disease name and the filtered sample invoice text and corresponding sample medical record diagnosis text in the sample cases, the positive sample pairs and the negative sample pairs are constructed.
[0032] In one optional implementation, constructing the positive sample pairs and the negative sample pairs based on the third-level disease name and the filtered sample invoice text and corresponding sample medical record diagnosis text in the sample cases includes:
[0033] If the disease names corresponding to the N sample medical record diagnostic texts all belong to the same disease category in the custom three-level disease classification system after disease classification mapping, then the N sample cases corresponding to the N sample medical record diagnostic texts will be used as target sample cases.
[0034] The filtered sample invoice text and the corresponding sample medical record diagnosis text in the target sample case are taken as the positive sample pair;
[0035] The filtered sample invoice text and another sample medical record diagnosis text from the target sample case are used as the negative sample pair.
[0036] A second aspect of this application provides a data verification device, comprising:
[0037] The target image acquisition module is used to acquire the target outpatient invoice image and the target medical record diagnosis image;
[0038] The initial text data acquisition module is used to process the target outpatient invoice image through an invoice information extraction model to obtain invoice detail text; and to process the target medical record diagnosis image through a medical record diagnosis extraction model to obtain medical record diagnosis text.
[0039] The association result determination module is used to process the invoice detail text and the medical record diagnosis text through a data association discrimination model to obtain a first probability and a second probability; the first probability represents the probability that the target outpatient invoice image and the target medical record diagnosis image match; the second probability represents the probability that the target outpatient invoice image and the target medical record diagnosis image do not match.
[0040] A third aspect of this application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method described in any implementation of the first aspect.
[0041] A fourth aspect of this application provides an electronic device, comprising:
[0042] A memory on which computer programs are stored;
[0043] A processor for executing the computer program in the memory to implement the steps of the method described in any implementation of the first aspect.
[0044] Compared with the prior art, this application has the following beneficial effects:
[0045] This application provides a data verification method, including: acquiring a target outpatient invoice image and a target medical record diagnosis image; processing the target outpatient invoice image using an invoice information extraction model to obtain invoice detail text; processing the target medical record diagnosis image using a medical record diagnosis extraction model to obtain medical record diagnosis text; and processing the invoice detail text and medical record diagnosis text using a data association discrimination model to obtain a first probability and a second probability; wherein, the first probability represents the probability that the target outpatient invoice image and the target medical record diagnosis image match; and the second probability represents the probability that the target outpatient invoice image and the target medical record diagnosis image do not match. This application constructs a dual-modal text extraction architecture of an invoice information extraction model and a medical record diagnosis extraction model to achieve structured text conversion between outpatient invoices and medical record diagnoses, and relies on a data association discrimination model to quantitatively discriminate the diagnostic logic correlation between the two types of text, thereby realizing the correlation verification between invoices and medical records and accurately identifying abnormal situations where treatment items and diagnostic conclusions do not match. Attached Figure Description
[0046] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0047] Figure 1 A flowchart illustrating a data verification method provided in this application embodiment;
[0048] Figure 2 A flowchart illustrating a method for obtaining positive and negative sample pairs corresponding to a data association discrimination model, provided in an embodiment of this application;
[0049] Figure 3 This is a schematic diagram of the structure of a data verification device provided in an embodiment of this application. Detailed Implementation
[0050] In the automated claims process for outpatient health insurance, medical invoices and medical records are the core documents supporting the compliant progress of the claims process, and the two must meet the requirement of one-to-one correspondence. However, in actual claims scenarios, there are situations where users visit different departments multiple times on the same calendar day. When submitting a claims application, they usually upload all medical invoices and medical records generated on that day in one package, without distinguishing and labeling the documents corresponding to different medical events.
[0051] In the traditional manual review model, reviewers can rely on their professional experience to extract key information such as the department visited, treatment time, and symptom description from medical records and manually compare it with the charging items and billing departments in invoices, thereby achieving accurate association of documents corresponding to the same medical event.
[0052] However, in automated claims processing, due to the lack of a unified and clear association identifier between invoices and medical records for different medical events, existing automated verification systems cannot independently identify the attribution relationship between documents, making it difficult to automatically match invoices and medical records for the same medical event. If this situation of multiple sets of documents being uploaded together is not specifically addressed, the automated claims system is highly prone to triggering verification anomalies due to document mismatch when executing association verification rules, leading to the interruption of the automated claims process. If such claims are directly returned to manual review or users are required to re-categorize and upload the documents, it will significantly increase manual review costs, reduce claims processing efficiency, and severely impact the user's claims service experience.
[0053] Meanwhile, insurance risk control is a core aspect that insurance companies focus on during their operations and management. Analysis of past insurance fraud cases reveals that fraudsters often commit insurance fraud by fabricating or altering medical invoices or records to seek illegal profits. For example, they can use image editing tools to tamper with key information on invoices, such as treatment items, charges, and departments, to create fake invoices that meet the claim requirements; they can also issue invoices for large-scale examinations and treatments that do not match the patient's current medical needs, and then split the illegal gains through hospital refund channels after the claim is received.
[0054] In response to the aforementioned fraudulent activities, traditional optical character recognition technology can only extract and recognize textual information on documents, but cannot determine the medical semantic logical relationship between treatment items and disease diagnosis conclusions, which brings significant operational risks to insurance companies.
[0055] Based on this, this application provides a data verification method, including: acquiring a target outpatient invoice image and a target medical record diagnosis image; processing the target outpatient invoice image using an invoice information extraction model to obtain invoice detail text; processing the target medical record diagnosis image using a medical record diagnosis extraction model to obtain medical record diagnosis text; and processing the invoice detail text and medical record diagnosis text using a data association discrimination model to obtain a first probability and a second probability; wherein, the first probability represents the probability that the target outpatient invoice image and the target medical record diagnosis image match; and the second probability represents the probability that the target outpatient invoice image and the target medical record diagnosis image do not match. This application constructs a dual-modal text extraction architecture of an invoice information extraction model and a medical record diagnosis extraction model to achieve structured text conversion between outpatient invoices and medical record diagnoses, and relies on a data association discrimination model to quantitatively discriminate the diagnostic logic correlation between the two types of text, thereby realizing the correlation verification between invoices and medical records and accurately identifying abnormal situations where treatment items and diagnostic conclusions do not match.
[0056] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present application.
[0057] Figure 1 This is a flowchart illustrating a data verification method provided in an embodiment of this application. (In conjunction with...) Figure 1 As shown, the data verification method disclosed in this application includes:
[0058] S101, Obtain the target outpatient invoice image and the target medical record diagnosis image.
[0059] The target outpatient invoice image and the target medical record diagnosis image in this application are outpatient invoice images and medical record diagnosis images that are to be determined whether there is a logical relationship between them in diagnosis and treatment.
[0060] Understandably, the target outpatient invoice image includes key information such as the patient's basic information, the time of visit, the department visited, the name and code of the treatment items, the amount charged, the invoice issuing agency, and the invoice number.
[0061] Understandably, the target medical record diagnostic image includes core medical record information such as the patient's basic information, consultation time, department visited, chief complaint, physical examination results, disease diagnosis conclusion, medical advice, and the attending physician's signature.
[0062] S102, the target outpatient invoice image is processed using an invoice information extraction model to obtain invoice detail text; and the target medical record diagnosis image is processed using a medical record diagnosis extraction model to obtain medical record diagnosis text.
[0063] The invoice information extraction model in this application is a pre-trained model used to extract detailed items from outpatient invoice images; the medical record diagnosis extraction model in this application is a pre-trained model used to extract diagnostic results from medical record diagnosis images.
[0064] After obtaining the target outpatient invoice image, the image is input into the invoice information extraction model, which outputs the invoice details text. This invoice details text includes items such as the name and code of the medical service and the charge amount.
[0065] After obtaining the target medical record diagnostic image, the image is input into the medical record diagnostic extraction model, which outputs the medical record diagnostic text. This text includes the disease diagnosis conclusion and other relevant information.
[0066] In one optional implementation, the steps for obtaining the invoice information extraction model include:
[0067] A1, obtain multiple sample outpatient invoice images.
[0068] The sample outpatient invoice images in this application are a sample set of outpatient invoice images containing complete medical information and clear text features, selected from the historical document database of health insurance claims. They cover invoice data from different medical institutions, different departments, and different types of medical services.
[0069] A2, annotate the detailed data in each of the sample outpatient invoice images to obtain the first annotation data corresponding to each sample outpatient invoice image.
[0070] Specifically, the details in each sample outpatient invoice image are labeled to obtain the first labeled data corresponding to each sample outpatient invoice image.
[0071] The first set of labeled data includes: patient's name, date of visit, department visited, name and code of treatment items, unit price of items, total price, invoice number and name of the issuing hospital, etc.
[0072] A3, the sample outpatient invoice image and the corresponding first labeled data are used as the first sample training data; and the LayoutLMv2 is trained by multiple sets of the first sample training data to obtain the invoice information extraction model.
[0073] LayoutLMv2 is a multimodal Transformer pre-trained model specifically designed for visually rich document understanding tasks. It adds deep fusion of image modalities, which greatly improves performance in tasks such as document information extraction and image classification.
[0074] In this application, the sample outpatient invoice image and the corresponding first annotation data are used as the first sample training data, so that multiple sets of first sample training data can be obtained; then, the LayoutLMv2 is trained using multiple sets of first sample training data to obtain the invoice information extraction model.
[0075] Since training LayoutLMv2 based on multiple sets of first sample training data to obtain the invoice information extraction model is well known to those skilled in the art, this application will not elaborate further on this content.
[0076] In one optional implementation, the step of obtaining the medical record diagnosis extraction model includes:
[0077] B1, acquire diagnostic images from multiple sample medical records.
[0078] The sample medical record diagnostic images in this application are medical record diagnostic images selected from the historical document database of health insurance claims business. They contain patient identity information, time and space information of medical treatment (such as date of medical treatment, department of medical treatment, time of consultation, etc.), clinical symptom information, core diagnostic information (such as disease diagnosis conclusion, diagnosis code, severity of illness, etc.), and treatment guidance information (such as medical advice, medication plan, follow-up requirements, etc.).
[0079] B2, the diagnostic conclusions in each of the sample medical record diagnostic images are labeled to obtain the second labeling data corresponding to each of the sample medical record diagnostic images.
[0080] Specifically, the diagnostic conclusions in the diagnostic images of each sample medical record are labeled to obtain the second labeling data corresponding to each sample medical record diagnostic image.
[0081] B3, the sample medical record diagnosis image and the corresponding second annotation data are used as the second sample training data; and the LayoutLMv2 is trained by multiple sets of the second sample training data to obtain the medical record diagnosis extraction model.
[0082] The sample medical record diagnosis image and the corresponding second annotation data are used as the second sample training data to obtain multiple sets of second sample training data; then, the LayoutLMv2 is trained using multiple sets of first sample training data to obtain the invoice information extraction model.
[0083] Since training LayoutLMv2 based on multiple sets of second sample training data to obtain a medical record diagnosis extraction model is well known to those skilled in the art, this application will not elaborate further on this content.
[0084] S103, the invoice details text and the medical record diagnosis text are processed by the data association discrimination model to obtain the first probability and the second probability.
[0085] Wherein, the first probability represents the probability that the target outpatient invoice image and the target medical record diagnosis image match; the second probability represents the probability that the target outpatient invoice image and the target medical record diagnosis image do not match.
[0086] When the first probability is greater than the preset first value, such as 0.7, it is determined that the target outpatient invoice image and the target medical record diagnosis image match; when the second probability is greater than the preset first value, such as 0.7, it is determined that the target outpatient invoice image and the target medical record diagnosis image do not match.
[0087] The data association discrimination model in this application is a pre-trained model used to output the matching probability and non-matching probability of the target outpatient invoice image and the target medical record diagnosis image.
[0088] In one optional implementation, the invoice details text and the medical record diagnosis text are processed using a data association discriminant model to obtain a first probability and a second probability, including:
[0089] C1, the invoice details text is encoded using a Sentence-Transformer to obtain a first feature vector; simultaneously, the medical record diagnosis text is encoded using the Sentence-Transformer to obtain a second feature vector.
[0090] Wherein, the first feature vector is a 768-dimensional semantic feature vector; the second feature vector is a 768-dimensional semantic feature vector.
[0091] Sentence-Transformer is a sentence embedding model based on the Transformer architecture. It is a Python library that can be used for sentence, text, and image embedding. It can transform the input sentence into a vector of fixed length 768.
[0092] In one alternative implementation, the invoice detail text is encoded using a Sentence-Transformer to obtain the first feature vector, which specifically refers to:
[0093] The first step is to filter the invoice detail text using a preset fuzzy invoice detail item set to obtain the filtered invoice detail text.
[0094] Because outpatient invoices for different claims contain a large number of identical or highly disparate expense items lacking specific medical indication, such as registration fees, outpatient consultation fees, laboratory fees, and fees for traditional Chinese medicine and Western medicine, these items only reflect basic fee categories and are not linked to specific medical treatments or symptoms. Therefore, they lack effective reference value for determining the correlation between invoices and medical record diagnostic images. Retaining them would introduce invalid noise data, interfering with the accuracy of subsequent correlation determinations. Thus, these items must be removed.
[0095] Based on this, this application conducts batch analysis using historical document image data from health insurance claims, and summarizes the aforementioned non-specific expense details through manual screening to construct a set of fuzzy invoice details.
[0096] The extracted invoice details text is matched and validated against the fuzzy details item set. All details that match the set are directly removed, and only high-discrimination details containing disease-indicating features such as specific treatment items, drug names, and examination contents are retained to obtain the filtered invoice text details.
[0097] After obtaining the filtered invoice text details, the filtered invoice text details are input into Sentence-Transformer. Since the filtered invoice text details include P fields, Sentence-Transformer converts the P fields into a P×768-dimensional feature vector. Then, the average operation of the P dimensions is performed to transform the P×768-dimensional feature vector into a 1×768-dimensional first feature vector.
[0098] Similarly, the medical record diagnosis text can be encoded using a Sentence-Transformer to obtain the second feature vector.
[0099] C2, based on the first feature vector and the second feature vector, generates the target feature vector.
[0100] After obtaining the first and second feature vectors, a dot product operation can be performed on the first and second feature vectors to obtain the third feature vector; the subtraction operation can be performed on the first and second feature vectors to obtain the fourth feature vector; the first, second, third, and fourth feature vectors are then concatenated to obtain the target feature vector. The target feature vector is a 1×3072-dimensional vector that retains the semantic information of both the invoice details text and the medical record diagnosis text, reflecting the interactive differences and similarities between the invoice details text and the medical record diagnosis text. The expression for the target feature vector is as follows:
[0101] in, It is the target feature vector; and The first eigenvector and the second eigenvector, respectively; It is the third eigenvector; It is the fourth eigenvector.
[0102] C3, the target feature vector is processed by a binary classifier to obtain the first probability and the second probability.
[0103] Specifically, the target feature vector is input into a pre-defined fully connected network for dimensionality compression and feature mapping. First, the target feature vector of dimension 3072 is mapped to an intermediate feature vector of dimension 768 through the first fully connected layer. Then, the second fully connected layer outputs a 2-dimensional raw score to be discriminated. Subsequently, the above 2-dimensional raw score to be discriminated is normalized by the softmax function, and a probability vector of shape 1×2 is output (corresponding to the probabilities of the two categories in the binary classification task, and the sum of the probabilities is 1).
[0104] In this context, the first dimension index position of the probability vector corresponds to the first probability, representing the probability that the target outpatient invoice image matches the target medical record diagnosis image; the second dimension index position of the probability vector corresponds to the second probability, representing the probability that the target outpatient invoice image does not match the target medical record diagnosis image.
[0105] Figure 2 This is a flowchart illustrating a method for obtaining positive and negative sample pairs corresponding to a data association discriminant model, as provided in an embodiment of this application. (Combined with...) Figure 2 As shown, the methods for obtaining positive and negative sample pairs include:
[0106] S201, obtain multiple sample cases.
[0107] The sample cases include images of sample outpatient invoices and corresponding sample medical record diagnostic images.
[0108] S202, the sample outpatient invoice image is processed by the invoice information extraction model to generate sample invoice detail text; and the sample medical record diagnosis image is processed by the medical record diagnosis extraction model to generate sample medical record diagnosis text.
[0109] S203, the sample invoice detail text is filtered to obtain the filtered sample invoice text; the filtered sample invoice text does not include the invoice detail items in the invoice fuzzy detail set.
[0110] This process is described in the relevant section of C1 and will not be repeated here.
[0111] S204, Based on the third-level disease name and the filtered sample invoice text and corresponding sample medical record diagnosis text in the sample case, construct the positive sample pair and the negative sample pair.
[0112] The first step is to determine the target sample cases if the disease names corresponding to the N sample medical record diagnosis texts all belong to the same disease category in the custom three-level disease classification system after disease classification mapping.
[0113] Specifically, in clinical practice, there are many cases where diseases have different names but highly similar clinical manifestations and treatment plans, such as enteritis, chronic enteritis, and appendicitis, whose corresponding examination items and medication regimens are significantly consistent. If negative sample pairs of invoices and medical records are constructed solely based on the literal differences in disease names, it will fail to accurately reflect the correlation characteristics of clinical diagnosis and treatment, leading to insufficient effectiveness in subsequent model training.
[0114] Based on this, this application pre-constructs a custom three-level disease classification table, which is based on the International Classification of Diseases (ICD-10) and optimized and adjusted in combination with the actual needs of health insurance claims business.
[0115] In the sample preprocessing stage, the specific disease names in the diagnostic texts of each sample's medical records are matched and mapped with a custom three-level disease classification table to obtain the corresponding three-level disease category names. For example, the aforementioned enteritis, chronic enteritis, and appendicitis are uniformly classified into the three-level disease name "non-infectious enteritis and colitis" after mapping. Through this classification mapping method, the normalization of clinically similar diseases is achieved, ensuring that the selection of target sample cases conforms to the actual diagnosis and treatment logic, and resulting in multiple target sample cases.
[0116] The second step is to use the filtered sample invoice text and the corresponding sample medical record diagnosis text from the target sample case as the positive sample pair.
[0117] The third step is to use the filtered sample invoice text and another sample medical record diagnosis text from the target sample case as the negative sample pair.
[0118] It is understandable that after obtaining multiple sets of positive sample pairs and multiple sets of negative sample pairs, the target model to be trained (including Sentence-Transformer, fully connected layers, and binary classifiers) can be trained to obtain a data association discrimination model. Since this process is well known to those skilled in the art, it will not be described in detail in this application.
[0119] In summary, this application provides a data verification method, comprising: acquiring a target outpatient invoice image and a target medical record diagnosis image; processing the target outpatient invoice image using an invoice information extraction model to obtain invoice detail text; processing the target medical record diagnosis image using a medical record diagnosis extraction model to obtain medical record diagnosis text; and processing the invoice detail text and medical record diagnosis text using a data association discrimination model to obtain a first probability and a second probability; wherein, the first probability represents the probability that the target outpatient invoice image and the target medical record diagnosis image match; and the second probability represents the probability that the target outpatient invoice image and the target medical record diagnosis image do not match. This application constructs a dual-modal text extraction architecture of an invoice information extraction model and a medical record diagnosis extraction model, realizing the structured text conversion of outpatient invoices and medical record diagnoses, and relying on a data association discrimination model to quantitatively discriminate the diagnostic logic correlation between the two types of text, thereby realizing the correlation verification between invoices and medical records and accurately identifying abnormal situations where treatment items and diagnostic conclusions do not match.
[0120] Based on the same inventive concept, this application provides a data verification device. Figure 3 This is a schematic diagram of a data verification device provided in an embodiment of this application. Figure 3 As shown, the data verification device 300 includes:
[0121] The target image acquisition module 301 is used to acquire the target outpatient invoice image and the target medical record diagnosis image;
[0122] The initial text data acquisition module 302 is used to process the target outpatient invoice image through an invoice information extraction model to obtain invoice detail text; and to process the target medical record diagnosis image through a medical record diagnosis extraction model to obtain medical record diagnosis text.
[0123] The association result determination module 303 is used to process the invoice detail text and the medical record diagnosis text through a data association discrimination model to obtain a first probability and a second probability; the first probability represents the probability that the target outpatient invoice image and the target medical record diagnosis image match; the second probability represents the probability that the target outpatient invoice image and the target medical record diagnosis image do not match.
[0124] In one alternative implementation, the association result determination module 303 includes:
[0125] The first feature vector acquisition unit is used to encode the invoice detail text using a Sentence-Transformer to obtain a first feature vector; the first feature vector is a 768-dimensional semantic feature vector.
[0126] The second feature vector acquisition unit is used to encode the medical record diagnosis text through the Sentence-Transformer to obtain a second feature vector; the second feature vector is a 768-dimensional semantic feature vector.
[0127] The target feature vector acquisition unit is used to generate a target feature vector based on the first feature vector and the second feature vector;
[0128] The association result determination unit is used to process the target feature vector through a binary classifier to obtain the first probability and the second probability.
[0129] In one alternative implementation, the target feature vector acquisition unit includes:
[0130] The third feature vector acquisition subunit is used to perform a dot product operation on the first feature vector and the second feature vector to obtain the third feature vector;
[0131] The fourth feature vector acquisition subunit is used to subtract the first feature vector and the second feature vector to obtain the fourth feature vector.
[0132] The target feature vector determination subunit is used to concatenate the first feature vector, the second feature vector, the third feature vector, and the fourth feature vector to obtain the target feature vector.
[0133] In one alternative implementation, the data verification device 300 further includes:
[0134] The first sample image acquisition module is used to acquire multiple sample outpatient invoice images;
[0135] The first annotation data acquisition module is used to annotate the detailed data in each of the sample outpatient invoice images to obtain the first annotation data corresponding to each sample outpatient invoice image;
[0136] The first model determination module is used to use the sample outpatient invoice image and the corresponding first annotation data as the first sample training data; and to train LayoutLMv2 with multiple sets of the first sample training data to obtain the invoice information extraction model.
[0137] In one alternative implementation, the data verification device 300 further includes:
[0138] The second image acquisition module is used to acquire diagnostic images of multiple sample medical records;
[0139] The second annotation data acquisition module is used to annotate the diagnostic conclusions in each of the sample medical record diagnostic images to obtain the second annotation data corresponding to each of the sample medical record diagnostic images.
[0140] The second model determination module is used to use the sample medical record diagnosis image and the corresponding second annotation data as the second sample training data; and to train LayoutLMv2 with multiple sets of the second sample training data to obtain the medical record diagnosis extraction model.
[0141] In one alternative implementation, the data verification device 300 further includes...
[0142] The sample case acquisition module is used to acquire multiple sample cases; the sample cases include sample outpatient invoice images and corresponding sample medical record diagnosis images;
[0143] The sample case data acquisition module is used to process the sample outpatient invoice image through the invoice information extraction model to generate sample invoice detail text; and to process the sample medical record diagnosis image through the medical record diagnosis extraction model to generate sample medical record diagnosis text.
[0144] The sample invoice text filtering module is used to filter the sample invoice detail text to obtain the filtered sample invoice text; the filtered sample invoice text does not include the invoice detail items in the invoice fuzzy detail set;
[0145] The sample pair construction module is used to construct the positive sample pair and the negative sample pair based on the third-level disease name and the filtered sample invoice text and corresponding sample medical record diagnosis text in the sample case.
[0146] In one alternative implementation, the sample pair building module includes:
[0147] The target sample case acquisition module is used to take the N sample cases corresponding to the N sample medical record diagnosis texts as target sample cases if the disease names corresponding to the N sample medical record diagnosis texts are all classified into the same disease category in the custom three-level disease classification system after disease classification mapping.
[0148] The positive sample pair construction module is used to take the filtered sample invoice text and the corresponding sample medical record diagnosis text in the target sample case as the positive sample pair;
[0149] The negative sample pair construction module is used to take the filtered sample invoice text and another sample medical record diagnosis text from the target sample case as the negative sample pair.
[0150] Based on the data verification method and apparatus provided in the foregoing embodiments, this application also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements some or all of the steps in the data verification method mentioned above.
[0151] Based on the data verification method and apparatus provided in the foregoing embodiments, this application also provides an electronic device, including:
[0152] A memory on which computer programs are stored;
[0153] A processor is configured to execute the computer program in the memory to implement some or all of the steps in the data verification method provided in the foregoing embodiments.
[0154] It should be noted that the various embodiments in this specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, for the device embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the description of the method embodiments. The device embodiments described above are merely illustrative, and the units described as separate components may or may not be physically separate. The components indicated as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment solution according to actual needs. Those skilled in the art can understand and implement this without creative effort.
[0155] The above description is merely one specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.
Claims
1. A data verification method, characterized in that, The method includes: Acquire the target outpatient invoice image and the target medical record diagnosis image; The target outpatient invoice image is processed using an invoice information extraction model to obtain the invoice details text; and the target medical record diagnosis image is processed using a medical record diagnosis extraction model to obtain the medical record diagnosis text. The invoice details text and the medical record diagnosis text are processed by a data association discrimination model to obtain a first probability and a second probability; the first probability represents the probability that the target outpatient invoice image and the target medical record diagnosis image match; the second probability represents the probability that the target outpatient invoice image and the target medical record diagnosis image do not match.
2. The method according to claim 1, characterized in that, The step of processing the invoice details text and the medical record diagnosis text through a data association discrimination model to obtain a first probability and a second probability includes: The invoice details text is encoded using a Sentence-Transformer to obtain a first feature vector; the first feature vector is a 768-dimensional semantic feature vector. The medical record diagnosis text is encoded using the Sentence-Transformer to obtain a second feature vector; the second feature vector is a 768-dimensional semantic feature vector. Based on the first feature vector and the second feature vector, a target feature vector is generated; The target feature vector is processed by a binary classifier to obtain the first probability and the second probability.
3. The method according to claim 2, characterized in that, The step of generating a target feature vector based on the first feature vector and the second feature vector includes: Perform a dot product operation on the first feature vector and the second feature vector to obtain the third feature vector; Subtract the first feature vector from the second feature vector to obtain the fourth feature vector; The first feature vector, the second feature vector, the third feature vector, and the fourth feature vector are concatenated to obtain the target feature vector.
4. The method according to claim 1, characterized in that, The steps for obtaining the invoice information extraction model include: Obtain multiple sample outpatient invoice images; The detailed data in each of the sample outpatient invoice images is annotated to obtain the first annotated data corresponding to each sample outpatient invoice image; The sample outpatient invoice image and the corresponding first labeled data are used as the first sample training data; and the LayoutLMv2 is trained using multiple sets of the first sample training data to obtain the invoice information extraction model.
5. The method according to claim 1, characterized in that, The steps for obtaining the medical record diagnosis extraction model include: Acquire diagnostic images from multiple sample medical records; The diagnostic conclusions in each of the sample medical record diagnostic images are labeled to obtain the second labeling data corresponding to each of the sample medical record diagnostic images; The sample medical record diagnostic images and corresponding second labeled data are used as second sample training data; and the LayoutLMv2 is trained using multiple sets of second sample training data to obtain the medical record diagnostic extraction model.
6. The method according to claim 1, characterized in that, The steps for obtaining the positive and negative sample pairs corresponding to the data association discrimination model include: Multiple sample cases are obtained; the sample cases include sample outpatient invoice images and corresponding sample medical record diagnosis images. The sample outpatient invoice image is processed using the invoice information extraction model to generate sample invoice detail text; and the sample medical record diagnosis image is processed using the medical record diagnosis extraction model to generate sample medical record diagnosis text. The sample invoice detail text is filtered to obtain the filtered sample invoice text; the filtered sample invoice text does not include the invoice detail items in the fuzzy invoice detail set. Based on the three-level disease name and the filtered sample invoice text and corresponding sample medical record diagnosis text in the sample cases, the positive sample pairs and the negative sample pairs are constructed.
7. The method according to claim 6, characterized in that, The construction of the positive sample pairs and the negative sample pairs based on the third-level disease name and the filtered sample invoice text and corresponding sample medical record diagnosis text in the sample cases includes: If the disease names corresponding to the N sample medical record diagnostic texts all belong to the same disease category in the custom three-level disease classification system after disease classification mapping, then the N sample cases corresponding to the N sample medical record diagnostic texts will be used as target sample cases. The filtered sample invoice text and the corresponding sample medical record diagnosis text in the target sample case are taken as the positive sample pair; The filtered sample invoice text and another sample medical record diagnosis text from the target sample case are used as the negative sample pair.
8. A data verification device, characterized in that, The device includes: The target image acquisition module is used to acquire the target outpatient invoice image and the target medical record diagnosis image; The initial text data acquisition module is used to process the target outpatient invoice image through an invoice information extraction model to obtain invoice detail text; and to process the target medical record diagnosis image through a medical record diagnosis extraction model to obtain medical record diagnosis text. The association result determination module is used to process the invoice detail text and the medical record diagnosis text through a data association discrimination model to obtain a first probability and a second probability; the first probability represents the probability that the target outpatient invoice image and the target medical record diagnosis image match; the second probability represents the probability that the target outpatient invoice image and the target medical record diagnosis image do not match.
9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the program implements the steps of the method described in any one of claims 1-7.
10. An electronic device, characterized in that, include: A memory on which computer programs are stored; A processor for executing the computer program in the memory to implement the steps of the method according to any one of claims 1-7.