An ai-generated field evaluation method and system based on shadow table isolation

Abstract

The present application relates to the technical field of medical data processing, and more particularly to an AI generated field evaluation method and system based on shadow table isolation, comprising: obtaining AI generated candidate field values and their original text positions in the source document, and writing into a shadow table physically isolated from the formal data table; according to manual confirmation or rewriting, extracting baseline information from the source document version and writing into the shadow table; after the source document is updated, based on the baseline information verification, obtaining a first determination result of whether the original record is invalid; identifying the transfer relationship of multiple source data of the candidate field value to merge into an independent source, judging whether the values are consistent, and obtaining a second determination result; when the original record is invalid or the values of the independent source are inconsistent, receiving a review result and writing into the shadow table; and sequentially writing the baseline information, the determination result and the review result into a log with a hash value linking the records before and after. The present application provides a verifiable trace basis for the evaluation process and review process of the candidate field values, and improves the accuracy of data tracing.

Description

Technical Field

[0001] This invention relates to the field of medical data processing technology, and in particular to an AI-generated field evaluation method and system based on shadow table isolation. Background Technology

[0002] In scenarios such as filling out electronic case report forms for clinical trials, entering insurance claim elements, and filling out financial customer information, the values ​​extracted by machines (large language models or rule engines) from source documents need to be entered into a regulated business database. Some existing solutions have implemented an isolated confirmation process where machine output is first written to an authenticated copy library, then manually verified, and finally mirrored to the formal business system, supporting traceability by recording the person, time, and content of the modifications; other solutions calculate a digest hash of the data when it is submitted as a whole to support source data verification. However, these solutions have limited ability to handle situations where source documents are revised again after manual confirmation: they either rely on reviewers to manually discover changes periodically, or they can only detect whether the document as a whole has changed, and cannot automatically determine at the field level whether a specific value that has been manually confirmed is still equal to the current original record of that data point.

[0003] Further analysis reveals that even with the introduction of character-level evidence pointers and comparison with neighboring text hashes, several technical obstacles unique to this field and not covered by general text processing methods still exist: First, revisions to the source document can cause the entire text adjacent to the value to shift. If the character offset before the revision is directly used to retrieve the character in the new version, irrelevant text will be retrieved. However, general anchor relocation techniques often treat the text adjacent to the value as the usable anchor point, ignoring the fact that the adjacent text is precisely the location most likely to be subject to audit revisions, leading to such revisions being incorrectly identified as environmental noise and missed. Second, a single overall hash comparison cannot distinguish the value itself. The three completely different types of text changes—namely, alteration of the text itself, alteration of the qualifier in the same sentence but not the value, and irrelevant modification of the distant text outside the sentence containing the value—cannot identify the substantial overwriting of the original value by later records within the same time window. Thirdly, when there are multiple source records for the same field, simply counting the number of sources with the same value as evidence of consistency will both misinterpret records that copy the same result from each other as independent evidence, thus artificially inflating the credibility, and allow the model's own high confidence to average out the conflict when there is a real independent source conflict, thus allowing high-risk fields that should be blocked to pass.

[0004] Patent publication number CN116703337A discloses a project document review system and method based on artificial intelligence technology. This solution allows for flexible configuration of project data sources, document types, and review rules through a configuration module. It utilizes a document information extraction module to extract key information from documents and combines this with a large language model module and an automatic review module to automatically review the extracted key information for compliance, content completeness, etc., according to preset review rules, ultimately generating a review report. The solution also supports configuring logical relationships such as equality and inclusion between fields through a visual rule orchestration engine to improve the automation and accuracy of document review. However, this type of solution targets a one-time compliance review at the time of document submission. In scenarios requiring long-term retention and continuous auditing, such as clinical trials, source documents often undergo multiple revisions due to disease progression and other reasons. If the review process only performs a comparison at the time of document submission, once subsequent versions change, the field values ​​that have previously passed review or been manually confirmed may become outdated. This discrepancy is often difficult to trace and detect during compliance audits or clinical audits, posing a real challenge to maintaining the credibility of document data in a regulatory environment. Summary of the Invention

[0005] In view of this, the present invention provides an AI-generated field evaluation method and system based on shadow table isolation to solve the technical problem in the prior art that in scenarios where the source document is subsequently revised, the document review in highly regulated fields such as clinical trials lacks continuous traceability and it is difficult to detect inconsistencies between historical review results and current source document records in a timely manner.

[0006] The technical solution of this invention is implemented as follows: On one hand, this invention provides an AI-generated field evaluation method based on shadow table isolation, including the following steps: S1. Obtain the candidate field values ​​generated by AI and the original text position of the candidate field values ​​in the source document. Write the candidate field values ​​into a shadow table. The shadow table is physically isolated from the formal data table. The formal data table is the data table into which the candidate field values ​​are written after confirmation or review. S2. In response to manual confirmation or rewriting operations for candidate field values, extract the original content of the candidate field values ​​from the source document version corresponding to the original document version when the operation is performed, based on the original text position, as the original record, and extract baseline information, and write the original record and baseline information into the shadow table. S3. When the source document is updated or a new source document is received, the content corresponding to the original record is relocated in the current source document based on the baseline information. The relocated content and its context are compared with the original record. It is also detected whether there is data in the current source document whose source time is later than the original record and whose value is different, which covers the field corresponding to the candidate field value. Based on whether the relocation is successful, whether the comparison is consistent, and whether there is any overwriting, a first determination result of whether the original record is invalid is obtained. S4. Obtain the source data of the candidate field value corresponding to the field in multiple source documents. The source data includes source time, source role and original text fragment. Based on the time sequence, role relationship and original text fragment association between the source data, identify the copying relationship between the source data, merge the source data with copying relationship into the same independent source, determine whether the values ​​of each independent source are consistent after merging, and obtain the second determination result. S5. When the first judgment result is that the original record is invalid, or the second judgment result is that the values ​​of independent sources are inconsistent, a review task for the candidate field values ​​is generated, relevant information for review and adjudication is presented, the review result for the review task is received, and the review result is written to the shadow table. S6. Write the baseline information, the first judgment result, the second judgment result, and the review result into the log in sequence. The records in the log are linked together by hash value.

[0007] Based on the above technical solutions, preferably, step S2 specifically includes: S21. Based on the original text location, extract the neighboring text containing the original record from the source document version corresponding to the manual confirmation or rewriting operation, and divide the neighboring text into a value core area and a limiting clause area according to the clause boundaries. The value core area corresponds to the original record, and the limiting clause area corresponds to the context of the original record. S22. Extract landmarks within the neighborhood after excluding the core area and calculate the relative displacement of the landmarks relative to the starting point of the core area. S23. Extract fingerprint information from the value kernel area and the qualifier area respectively, and extract the qualifiers belonging to the preset qualifier set from the qualifier area; S24. Extract field labels within the vicinity of the value kernel area and record the source time and source role of the candidate field values; S25. Write the original record into the shadow table, and write the landmark, relative displacement, value core area fingerprint information, qualifier area fingerprint information, qualifier, field label, source time and source role into the shadow table as baseline information.

[0008] Based on the above technical solution, preferably, in step S22, the neighborhood search window is determined according to the length of the value core region. Within the neighborhood search window, excluding the value core region, the shortest substring that satisfies the uniqueness condition is selected as the landmark. The relative displacement of the landmark is calculated using the following formula: ; in This is relative displacement. To determine the starting character position of the value core area, For the selected road signs, This indicates the starting character position of the substring in the source document version corresponding to the manual confirmation or rewrite operation.

[0009] Based on the above technical solution, preferably, in step S23, the fingerprint information is calculated after normalization processing is performed on the value kernel region and the limiting term region respectively: ; ; in To obtain the fingerprint information of the core area, This refers to the fingerprint information of the restricted text area, where D represents the source document version corresponding to the manual confirmation or rewrite operation. For the value core region, For the qualifier section, This indicates normalization processing. This represents a hash operation; after distinguishing the qualifier words, the intersection with the preset qualifier set is taken to obtain the qualifier.

[0010] Based on the above technical solutions, preferably, step S3 specifically includes: S31. Based on landmarks and relative displacement, relocate the value core area and the qualifier area in the current source document to obtain the location result; S32. Extract fingerprint information from the repositioned value core area and compare it with the value core area fingerprint information in the baseline information to obtain the value core area comparison result; S33. Extract fingerprint information and limiting terms from the repositioned limiting term region, and compare them with the limiting term region fingerprint information and limiting terms in the baseline information to obtain the limiting term comparison result. S34. Based on field labels, source time, and source role, retrieve later data in the current source document whose source time is later than the original record and whose values ​​are different, and obtain the later coverage detection results. S35. When the positioning result is that repositioning is unsuccessful, or the value core area comparison result or the qualifying statement comparison result is inconsistent, or the subsequent coverage detection result is that coverage exists, the original record corresponding to the candidate field value is determined to be invalid, and the first determination result is obtained.

[0011] Based on the above technical solutions, preferably, step S31 specifically includes: S311. Find the starting character position of the landmark in the current source document, add it to the relative displacement to obtain the candidate positioning position, and perform matching in the vicinity of the candidate positioning position to obtain the repositioned value core area position; when the landmark is not unique in the current source document or no repositioned value core area position is matched, in the neighborhood range after excluding the value core area, select the next unselected substring in the order of increasing length of the substring that meets the uniqueness condition to replace the landmark, recalculate the relative displacement and repeat this step until the repositioned value core area position is obtained or no unselected substring exists; S312. When there is no unselected substring and the value core area position after repositioning has not been obtained, a matching method different from character matching is used to match the neighboring text to obtain the value core area position after repositioning or the positioning result is that the repositioning is unsuccessful. S313. Based on the positional relationship between the repositioned value core area and the qualifier area, the repositioned qualifier area is obtained, and the positioning result is obtained.

[0012] Based on the above technical solution, preferably, in step S34, data matching the field label and belonging to the same source time window is searched in the current source document to obtain the value, source time, and source role of the matching data; when the matching data simultaneously meets the following conditions: the source role is the preset original source role, the source time is later than the source time in the baseline information, and the value does not meet the equivalence criterion with the candidate field value, it is determined that there is coverage; the coverage detection result is then obtained by the following formula: ; in For subsequent coverage detection results, S=0 indicates the presence of coverage, and S=1 indicates the absence of coverage. To match the source role of the data, As the default original source character, To match the source time of the data, The source time is the baseline information. To match the values ​​of the data, For candidate field values, This serves as an equivalence criterion.

[0013] Based on the above technical solutions, preferably, in step S4, the source role includes a preset original source role and a relaying source role; the identification of the transcription relationship between source data specifically includes: For any first source data and second source data, provided that the source time of the first source data is no later than the source time of the second source data, and the values ​​of the first source data and the second source data satisfy the equivalence criterion, the second source data is determined to be copied from the first source data if any of the following conditions are met: ① When the source role of the first source data is the original source role, and the source role of the second source data is the relay source role; ②The original text fragments of the second source data contain the original text fragments of the first source data; ③ The similarity between the original text fragments of the first source data and the second source data is not lower than the preset threshold.

[0014] Based on the above technical solutions, preferably, in step S4, source data that is not copied from other source data is identified as independent source; source data with copying relationships are merged into the independent source corresponding to the starting point of their copying chain; when the number of independent sources is not less than two, and the values ​​of at least two of the independent sources do not meet the equivalence criterion, the independent source values ​​are determined to be inconsistent, and a second determination result is obtained.

[0015] On the other hand, the present invention also provides an AI-generated field evaluation system based on shadow table isolation, comprising: The shadow table module is physically isolated from the formal data table, which is the data table written after the candidate field values ​​are confirmed or reviewed. It obtains and stores the candidate field values ​​generated by AI and their original positions in the source document, as well as the original records, baseline information, first judgment result, second judgment result, and review result. When the first judgment result is that the original record is invalid or the second judgment result is that the values ​​from independent sources are inconsistent, a review task for the candidate field value is generated, relevant information for review and adjudication is presented, and the review result is received and stored. The document verification module responds to manual confirmation or rewriting operations for candidate field values. Based on the original text position, it extracts the original text content of the candidate field value from the source document version corresponding to the operation as the original record and extracts baseline information. When the source document is updated or a new source document is received, it relocates the content corresponding to the original record in the current source document based on the baseline information. It compares the relocated content and its context with the original record and detects whether there is data in the current source document with a later source time and different value that covers the field corresponding to the candidate field value. Based on whether the relocation is successful, whether the comparison is consistent, and whether there is overwriting, a first determination result is obtained as to whether the original record is invalid. The source analysis module obtains the source data of the candidate field value corresponding to the field in multiple source documents. The source data includes source time, source role, and original text fragment. Based on the time sequence, role relationship, and original text fragment association between the source data, the module identifies the transcription relationship between the source data, merges the source data with transcription relationship into the same independent source, and determines whether the values ​​of each independent source are consistent after merging to obtain a second determination result. The log module writes the baseline information, the first judgment result, the second judgment result, and the review result into the log in sequence. The records in the log are linked by hash values.

[0016] The present invention has the following advantages over the prior art: (1) By writing candidate field values ​​into a shadow table that is physically isolated from the official data table, the AI-generated fields do not enter the query, statistics and export path of the official dataset before the completion of manual confirmation or review and adjudication. This avoids unverified values ​​from polluting the downstream official data. At the same time, the manual confirmation or rewriting operation itself will trigger the extraction and retention of baseline information, so that the data verification process and the reading and writing path of the official dataset are physically separated, taking into account both the field extraction efficiency and the credibility of the official data.

[0017] (2) This invention achieves the repositioning of the value core area and the qualifier area between versions of the source document by combining road signs with relative displacement. The positioning result, the value core area comparison result, the qualifier comparison result, and the subsequent coverage detection result are used as four independent criteria that do not compensate for each other. Different types of mismatches that may occur after the source document is revised, such as changes in the value itself, changes in the nature of the qualifier, missing evidence paragraphs, and subsequent evidence coverage, can be identified separately. Compared with the verification method that only relies on a single overall comparison method, it reduces the possibility of omission of the original record invalidation judgment due to the inability of a single criterion to cover a certain type of mismatch.

[0018] (3) This invention identifies the copying relationship between source data and merges source data with copying relationship into the same independent source. Then, it judges the consistency of values ​​between the merged independent sources. This avoids the original information being repeatedly included in the consistency evidence after being relayed multiple times, thus obscuring the real value conflict between independent sources. This makes the conclusion of the multi-source consistency judgment closer to the actual independence of the information source.

[0019] (4) This invention writes the baseline information, the first judgment result, the second judgment result and the review result into a log linked by hash value. If any historical record in the log is tampered with, the hash value of the record and its subsequent records will be inconsistent with the recalculated result. This provides a verifiable traceability basis for the evaluation and review process of candidate field values ​​and meets the audit requirements for data traceability in scenarios such as clinical trials. Attached Figure Description

[0020] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0021] Figure 1 This is a flowchart of the AI-generated field evaluation method of the present invention; Figure 2 This is a schematic diagram illustrating baseline information location and cross-version relocation in this invention; Figure 3 This is a schematic diagram illustrating the multi-source copying merging and consistency judgment of the present invention; Figure 4 This is a schematic diagram of the hash chain log structure of the present invention; Figure 5 This is a schematic diagram illustrating the system architecture and physical isolation of the shadow table in this invention. Detailed Implementation

[0022] The technical solutions of the present invention will be clearly and completely described below with reference to the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.

[0023] like Figure 1 As shown, this invention provides an AI-generated field evaluation method based on shadow table isolation, comprising the following steps: S1. Obtain the candidate field values ​​generated by AI and the original text position of the candidate field values ​​in the source document. Write the candidate field values ​​into a shadow table. The shadow table is physically isolated from the formal data table. The formal data table is the data table into which the candidate field values ​​are written after confirmation or review. S2. In response to manual confirmation or rewriting operations for candidate field values, extract the original content of the candidate field values ​​from the source document version corresponding to the original document version when the operation is performed, based on the original text position, as the original record, and extract baseline information, and write the original record and baseline information into the shadow table. S3. When the source document is updated or a new source document is received, the content corresponding to the original record is relocated in the current source document based on the baseline information. The relocated content and its context are compared with the original record. It is also detected whether there is data in the current source document whose source time is later than the original record and whose value is different, which covers the field corresponding to the candidate field value. Based on whether the relocation is successful, whether the comparison is consistent, and whether there is any overwriting, a first determination result of whether the original record is invalid is obtained. S4. Obtain the source data of the candidate field value corresponding to the field in multiple source documents. The source data includes source time, source role and original text fragment. Based on the time sequence, role relationship and original text fragment association between the source data, identify the copying relationship between the source data, merge the source data with copying relationship into the same independent source, determine whether the values ​​of each independent source are consistent after merging, and obtain the second determination result. S5. When the first judgment result is that the original record is invalid, or the second judgment result is that the values ​​of independent sources are inconsistent, a review task for the candidate field values ​​is generated, relevant information for review and adjudication is presented, the review result for the review task is received, and the review result is written to the shadow table. S6. Write the baseline information, the first judgment result, the second judgment result, and the review result into the log in sequence. The records in the log are linked together by hash value.

[0024] In one embodiment of the present invention, in step S1, the candidate field value is generated by extracting information from the source document using a large language model or a rule engine. The large language model can be a general-purpose large language model or a model fine-tuned for a specific document type. The rule engine can employ a deterministic extraction procedure based on regular expressions and dictionary matching. The candidate field value carries its original position in the source document, represented as an interval between the start and end character offsets. Before being written to the shadow table, the candidate object undergoes interface verification, which includes whether the candidate field value carries the complete original position and the corresponding original text fragment. Candidate objects lacking original positions or fragments are not accepted and are transferred to the manual entry branch, where business personnel directly enter the value. The verified candidate field values ​​are written to the shadow table in a pending confirmation state, along with the model identifier that generated the candidate field value and the prompt word template version number. Records in the pending confirmation state are not visible to downstream statistical analysis and do not participate in the export of the formal dataset or database locking. The shadow table is physically independent and deployed in a different database instance or database schema than the official data table. No foreign key constraints or view mappings are established between the two; logical relationships are only established through the business primary key corresponding to the candidate field values ​​and dimensions such as subjects and visits. Candidate field values ​​are not included in the query, statistics, and export paths of the official dataset until manually confirmed or reviewed. The official data table is the table where candidate field values ​​are written after confirmation or review.

[0025] In one embodiment of the present invention, such as Figure 2 As shown, in step S2, the manual confirmation or rewrite operation refers to the confirmation or rewrite action performed by business personnel on the verification interface for the candidate field value. A confirmation action indicates that the business personnel acknowledge that the candidate field value is consistent with the source document record, while a rewrite action indicates that the business personnel have corrected the candidate field value. After responding to this operation, within the same database transaction, based on the original text position carried by the candidate field value, the original text content of the candidate field value is extracted from the source document version corresponding to the manual confirmation or rewrite operation as the original record, and baseline information is extracted. If a new original text position is manually specified during the rewrite operation, the original record and baseline information are extracted according to the newly specified original text position. The original record and baseline information are then written to the shadow table.

[0026] like Figure 2 As shown, step S2 specifically includes: S21. Based on the original text location, extract the neighboring text containing the original record from the source document version corresponding to the manual confirmation or rewriting operation, and divide the neighboring text into a value core area and a limiting clause area according to the clause boundaries. The value core area corresponds to the original record, and the limiting clause area corresponds to the context of the original record.

[0027] Extract neighboring text based on the original text position. The value range is denoted as... The neighborhood interval is determined by the following formula: ; ; in , These represent the starting and ending character positions of the value core area, respectively. The neighborhood half-window width is determined by the larger of a fixed lower limit of 128 characters and twice the length of the value kernel area. Neighborhood interval. Simultaneously serving as a neighborhood search window, it is only used for signpost searches and does not constitute the outer boundary of the qualifying text area. The neighborhood text is divided into a value core area and a qualifying text area based on clause boundaries. The qualifying text area determines its boundary by searching for the nearest delimiter positions to the left and right sides of the value core area according to a preset set of clause delimiters. The set of clause delimiters includes periods, semicolons, line breaks, and table cell boundaries. The left and right extension lengths are calculated using the following formulas: ; ; in , These represent the actual extension lengths of the limiting text area to the left and right, respectively. This is the character position of the nearest separator to the left of the start of the core area. This is the character position of the nearest delimiter to the right of the end of the core area. This is the maximum length for the area, with a default value of 80 characters. If there is no separator on one side, the extended length on that side is [value missing]. The qualifier region is defined independently by the clause boundaries and may extend beyond the adjacent region. If the limit is exceeded, the text will still be truncated by clause boundaries, without falling back to the boundaries of neighboring regions. If the source document does not have available clause punctuation, the limited text area degenerates into a fixed-length window truncated by the upper limit of length.

[0028] S22. Extract landmarks within the neighborhood after excluding the core region, and calculate the relative displacement of the landmarks relative to the starting point of the core region. The neighborhood search window is the neighborhood interval. The candidate range for a roadmark is the area within the neighborhood search window excluding the value kernel region. A minimum length limit is set for the roadmark, with a default of 4 characters. Within the candidate range, the search proceeds incrementally from this minimum length limit, selecting the shortest length that produces at least one substring uniquely appearing throughout the entire source document. The unique substring within this length that is closest to the value kernel region is then chosen as the roadmark, denoted as […]. When distances are equal, the landmark to the left of the value core area is prioritized. Simultaneously, following the same rule, several unique substrings that have not yet been selected are saved in ascending order of length; two are saved by default as unselected substrings for positioning. The relative displacement of the landmark to the starting point of the value core area is calculated using the following formula: ; in This is relative displacement. This indicates the starting character position of the substring in the source document version corresponding to the manual confirmation or rewrite operation.

[0029] S23. Extract fingerprint information from the value kernel area and the qualifier area respectively, and extract the qualifiers belonging to the preset qualifier set from the qualifier area.

[0030] Fingerprint information is calculated after normalization processing is performed on the value kernel region and the qualifying region respectively. Normalization processing includes NFKC normalization and whitespace character compression of the text. The fingerprint information is calculated using the following formula: ; ; in To obtain the fingerprint information of the core area, This refers to the fingerprint information of the restricted text area, where D represents the source document version corresponding to the manual confirmation or rewrite operation. For the value core region, For the qualifier section, This indicates normalization processing. This represents a hash operation, specifically using the SHA-256 algorithm. After segmenting the text in the qualifier region, the intersection of this segment with the preset qualifier set corresponding to the value type of the candidate field is taken to obtain the qualifier. The preset qualifier set is pre-compiled into closed sets according to the value type, containing common limiting or modifying words under that value type, such as fasting and post-meal, left and right sides, sitting and lying positions, invalid and subject to retesting. The closed sets are taken from the existing value type definitions and common qualifiers of the field, without introducing additional domain-specific words unrelated to the field.

[0031] S24. Extract field labels from the vicinity of the value kernel area and record the source time and source role of the candidate field values. Search for the nearest project name or field name text as the field label within the left vicinity of the value kernel area; if no valid field label is found within the left vicinity, search within the right vicinity of the value kernel area. Simultaneously record the source time of the candidate field values, i.e., the source time of the original text upon which the baseline information is based, and the source role. The source role is taken from a preset classification set to distinguish whether the document recording the candidate field values ​​is a first-hand record in terms of business operations. For example, test reports and instrument reports are classified into one category, while indirect documents such as medical records and discharge summaries are classified into another.

[0032] S25. Write the original record into the shadow table, and write the landmark, relative displacement, value core area fingerprint information, qualifier area fingerprint information, qualifier, field label, source time and source role as baseline information into the shadow table within the same database transaction as the manual operation.

[0033] In one embodiment of the present invention, step S3 specifically includes: S31. Based on landmarks and relative displacement, relocate the value core area and the qualifying term area in the current source document to obtain the location result, including the following steps: S311. Search for landmarks in the current source document. If multiple landmarks are found, the landmark is determined to be non-unique, and the process proceeds to the next substring replacement step. If a landmark is uniquely found, the candidate location is obtained by adding its starting character position to its relative displacement, calculated using the following formula: ; in Candidate location positions, This indicates the starting character position of the substring in the current source document. For landmarks (i.e., unique location substrings selected during manual confirmation, located outside the value core area). This represents the relative displacement between the starting character position of the road sign and the starting character position of the value core area. Then, using the original text fragments of 32 characters to the left and right of the value core area at the time of confirmation as the matching pattern, matching is performed within the vicinity of the candidate positioning position; if multiple matching positions are obtained, the matching point closest to the candidate positioning position is taken as the repositioned value core area position. ; in This refers to the repositioned core region location. This refers to the set of all hit locations obtained by matching the matching pattern within the vicinity of the candidate location. For the set of hit locations The elements in the array. If no matching position is found, it is determined that the repositioned value core area position has not been matched. When the landmark is not unique or the repositioned value core area position is not matched, within the neighborhood range excluding the value core area, the next unused substring is selected in ascending order of length among the substrings that meet the uniqueness condition to replace the landmark. Priority is given to using the saved unused substrings. The relative displacement is recalculated and this step is repeated until the repositioned value core area position is obtained, or no unused substring exists.

[0034] S312. When no unselected substring exists and the repositioned value kernel location is not yet obtained, a different matching method than character matching is used to match the neighboring text. Specifically, two methods are executed in parallel: approximate matching based on edit distance and semantic retrieval based on text embedding vectors. The text embedding vector can use a general Chinese text embedding model, such as the BGE series or M3E series sentence vector model. The neighboring text of the value kernel and the candidate segments in the current source document are encoded into vectors, and the cosine similarity is calculated. The segment with the highest similarity is taken as the candidate matching position. When the candidate matching positions given by the edit distance approximate matching and semantic retrieval methods are consistent, or the difference between the two positions does not exceed the preset tolerance, the repositioned value kernel location is obtained. When the positions given by the two methods are inconsistent and the difference exceeds the preset tolerance, the positioning result is determined to be unsuccessful, and the reason for failure is that the position is questionable. When neither method can give a valid match, the positioning result is determined to be unsuccessful, and the reason for failure is that the evidence paragraph no longer exists.

[0035] S313. Based on the positional relationship between the repositioned value core region and the limiting region, and following the original division method where the limiting region is relative to the left and right sides of the value core region, with clause separators as boundaries and the upper limit of region length as constraints, the limiting region is redefined to obtain the positioning result. Only when step S311 or step S312 provides a stable and conflict-free positioning result is the positioning result considered successful, and the repositioned value core region position is used as the value range for steps S32 and S33. If the positioning result indicates unsuccessful repositioning, steps S32 and S33 are not executed, but are still executed during coverage detection; if the positioning result indicates successful repositioning, steps S32 and S33 are executed respectively to obtain the value core region comparison result and the limiting region comparison result.

[0036] S32. Extract fingerprint information from the repositioned value core area and compare it with the fingerprint information of the value core area to obtain the value core area comparison result. Perform NFKC normalization processing and whitespace character compression processing on the repositioned value core area, and then use the SHA-256 algorithm to calculate fingerprint information, and compare it with the value core area fingerprint information stored in the shadow table. If the two are equal, the value core area comparison result is consistent; otherwise, it is inconsistent.

[0037] S33. Extract fingerprint information and qualifiers from the repositioned qualifier region, and compare them with the fingerprint information and qualifiers of the qualifier region to obtain the qualifier comparison result.

[0038] For the repositioned qualifier region, fingerprint information is extracted using the same normalization and hashing methods as in step S32. The qualifier region text is then segmented and its intersection with the preset qualifier set is taken to obtain the re-extracted qualifiers. These re-extracted qualifiers are then compared with the fingerprint information and qualifiers stored in the shadow table. The comparison is performed in the order of hash short-circuit combined with symmetric difference: if the qualifier region fingerprint information is equal, the qualifier comparison result is directly consistent; if the qualifier region fingerprint information is not equal, the symmetric difference set between the re-extracted qualifier set and the original qualifier set is further calculated. The symmetric difference set is the union of the distinct elements in the two sets; when the symmetric difference set is empty, the qualifier comparison result is still consistent; when the symmetric difference set is not empty, the qualifier comparison result is inconsistent.

[0039] S34. Based on field labels, source time, and source role, retrieve later data in the current source document whose source time is later than the original record and whose values ​​are different, and obtain the later coverage detection results.

[0040] S35. When the positioning result is that repositioning is unsuccessful, or the value core area comparison result or the qualifying statement comparison result is inconsistent, or the subsequent coverage detection result is that coverage exists, the original record corresponding to the candidate field value is determined to be invalid, and the first determination result is obtained.

[0041] Based on field labels, source time, and source role, retrieve later data in the current source document whose source time is later than the original record and whose values ​​are different. Find records in the current source document that are equal to the normalized field label or whose edit distance does not exceed a preset threshold (which can be 1). Records matching this label are used as matching data, and their values, source time, and source role are obtained. If the matching data simultaneously meets the following criteria: the source role is the preset original source role, the source time is later than the source time in the baseline information, and the value does not meet the equality criterion with the candidate field value, then coverage is determined. The later coverage detection result is obtained using the following formula: ; in For subsequent coverage detection results, S=0 indicates the presence of coverage, and S=1 indicates the absence of coverage. To match the source role of the data, As the default original source character, To match the source time of the data, for The source time in the baseline information, To match the values ​​of the data, For candidate field values, This serves as the equality criterion. When the above formula is true, the subsequent coverage detection result is "coverage exists." The equality criterion is defined according to the type of candidate field value: for numeric fields, it judges whether the relative error is within the tolerance range according to the preset tolerance; for date fields, it judges whether they are equal after being normalized to the standard date format; for enumerated or text fields, it judges whether the characters are equal or whether they are mapped to the same preset enumeration code after normalization. Subsequent records whose source role is not the preset original source role are not included in this step of the judgment.

[0042] In one embodiment of the present invention, step S4 can be executed after the candidate field value is written to the shadow table. If the independent source values ​​are determined to be inconsistent before confirmation, the candidate field value will not be automatically confirmed. After confirmation, if the source document is updated or a new document arrives, step S4 and step S3 are executed in parallel.

[0043] like Figure 3 As shown, the source data of the candidate field value corresponding to the field in multiple source documents is obtained. The source data includes source time, source role, and original text fragment. Identifying the transcription relationship between source data includes: source roles include preset original source roles and paraphrased source roles. The original source role corresponds to the document type of first-hand records such as test reports and instrument reports, and the paraphrased source role corresponds to the document type of paraphrased records such as medical records and discharge summaries. The correspondence between document types and source roles is pre-compiled into a document type table according to the target business domain. For any first source data and second source data, based on the fact that the source time of the first source data is not later than the source time of the second source data, and the values ​​of the first source data and the second source data meet the equivalence criterion, the second source data is determined to be transcribed from the first source data if any of the following conditions are met: the source role of the first source data is the original source role and the source role of the second source data is the paraphrased source role; the original text fragment of the second source data contains the original text fragment of the first source data at the character level after normalization processing; the similarity between the original text fragments of the first source data and the second source data is not lower than a preset threshold. The similarity of the original text segments is calculated based on the edit distance. The similarity score is obtained by subtracting the larger of the edit distance and the length of the two segments from 1. A preset threshold of 0.85 is used. If the source role of either of the two source data cannot be determined, or if either the source time or the original text segment cannot be obtained, a copying relationship between the two source data is not determined. They are retained as independent sources and included in the independent source value consistency judgment.

[0044] Source data that is not derived from other source data is identified as an independent source. In a transfer chain with established transfer relationships, if a source data point is not the result of transferring any other source data, that source data itself is considered an independent source. Several source data points with transfer relationships, transferred sequentially, are merged into the independent source corresponding to the starting point of the same transfer chain, without duplicate counting. When there are at least two independent sources, and the values ​​of at least two of these independent sources do not meet the equivalence criterion, the independent source values ​​are determined to be inconsistent, resulting in a second determination.

[0045] In one embodiment of the present invention, in step S5, when the first determination result is that the original record is invalid, or the second determination result is that the values ​​from independent sources are inconsistent, a review task for the candidate field value is generated. The record corresponding to the candidate field value is marked as pending review, the current context of the record is frozen, and the current source document can be re-extracted. The extraction result is only used for comparison and does not automatically overwrite the existing value. Relevant information for review decision is presented: the verification interface is presented according to the specific reasons that led to entering the pending review state; when entering the pending review state due to the original record being invalid, the text of the new and old value core area, the difference in limiting terms, or the later source information are displayed according to the specific type of unsuccessful relocation, inconsistent comparison, or overwriting in step S3; when entering the pending review state due to inconsistent values ​​from independent sources, the values ​​of each independent source after merging and their source information are displayed. Business personnel make review decisions on the candidate field values ​​accordingly. The decision results include maintaining the original value, adopting the new value, or rewriting it. The review results record the decision content, the decision-maker, the decision time, and the reason code, and are written to the shadow table. The review process has a time limit; if no decision is made within the time limit, the process will be escalated. Records that have entered the pending review state will not reuse the routing rules from the initial write of candidate field values, and will not be automatically confirmed due to the existence of new high-confidence candidates. The record status can only be changed after manual decision-making.

[0046] In one embodiment of the present invention, such as Figure 4 As shown, in step S6, the baseline information, the first judgment result, the second judgment result, and the review result are sequentially written into the log. The hash value of each record in the log is calculated using the following formula: ; in The hash value of the nth log record. The hash value of the (n-1)th log record. The payload content of the nth log entry. This represents a string concatenation operation. This indicates the specific algorithm used for the hash operation. The hash value of the first log record is calculated by concatenating an unprecedented value or a fixed initial value with the first payload. The payload includes landmarks, relative displacement, value kernel area fingerprint information, qualifying text area fingerprint information, qualifying text, field labels, source time, source role, and the first judgment result, second judgment result, and review result involved in this write operation. The log uses an append-only write method and does not support modification or deletion of already written records. The hash value of each record is calculated based on the hash value of the previous record. If a historical log record is tampered with, the hash value of that record and all subsequent records will not be consistent with the recalculated result.

[0047] like Figure 5 As shown, the present invention also provides an AI-generated field evaluation system based on shadow table isolation, including a shadow table module, a document verification module, a source analysis module, and a log module.

[0048] The shadow table module is physically isolated from the formal data table. The formal data table is the data table where candidate field values ​​are written after confirmation or review. In specific implementations, it can be deployed using an independent database instance or an independent database mode. The shadow table module obtains and stores the candidate field values ​​generated by AI and their original positions in the source document. Before writing, it verifies whether the original position and original text fragment are complete; if missing, it switches to the manual input branch. The shadow table module stores the original records, baseline information, first judgment result, second judgment result, and review result, and maintains the record status, including the pending confirmation status and the pending review status. It provides interfaces for querying and exporting by record status. The export of the formal dataset only reads from the range of records that have been confirmed or reviewed. When the first judgment result is that the original record is invalid or the second judgment result is that the value is inconsistent from an independent source, the shadow table module generates a review task for the candidate field value, marks the corresponding record as pending review, freezes the current context, presents relevant information for review and adjudication, receives and stores the review result for the review task, and performs upgrade processing on records that have not been adjudicated within the review time limit.

[0049] The document verification module responds to manual confirmation or rewriting operations for candidate field values. Based on the original text location, it extracts the original text content of the candidate field values ​​from the source document version corresponding to the operation, using it as the original record, and extracts baseline information. During extraction, neighboring text is truncated, and the value core area and qualifying phrase area are divided according to clause boundaries. Within the neighboring search window excluding the value core area, landmarks are selected and their relative displacements are calculated. Normalization is performed on both the value core area and the qualifying phrase area, and fingerprint information is calculated. Qualifying phrases are distinguished by words and their intersection with a preset qualifying phrase set to obtain the qualifying phrases. Field labels are extracted within the vicinity of the value core area, and the source time and source role are recorded. Within the same database transaction, the original record and the above content are written as baseline information to the shadow table module. When the source document is updated or a new source document is received, the document verification module relocates the content corresponding to the original record in the current source document based on the baseline information. The relocated content and its context are compared with the original record, and the overlay is detected: candidate positioning positions are calculated based on landmarks and relative displacements, and the relocated value core area position is obtained by pattern matching using the original text fragments in the neighborhood of the value core area at the time of confirmation; if the landmark is not unique or is not matched, the unused substrings are selected in ascending order of length among the substrings that meet the uniqueness condition to replace the landmarks and the relative displacement is recalculated; if the unused substrings are exhausted and the position is still not located, back-matching is performed using edit distance approximate matching and text embedding vector semantic retrieval; after successful positioning, the limiting text area is re-divided according to the original position relationship, and fingerprint information and limiting text are extracted and compared for the relocated value core area and limiting text area respectively. The data is retrieved based on field labels, source time and source role, and the detection result is given according to the back-to-back overlay detection formula. If the relocation result indicates unsuccessful relocation, or if any of the value kernel comparison results or the qualifying statement comparison results are inconsistent, or if the subsequent coverage detection result indicates the existence of coverage, the original record is determined to be invalid, and the first determination result is obtained. If relocation fails, value kernel comparison and qualifying statement comparison are stopped, but subsequent coverage detection is still performed.

[0050] The source analysis module obtains source data for the candidate field values ​​in multiple source documents. This source data includes source time, source role, and original text fragments. Based on the temporal sequence, role relationships, and original text fragment associations among the source data, it identifies the transcription relationships between them. For any first and second source data, provided the source time sequence is valid and the values ​​meet the equivalence criterion, if the source role satisfies the pairing of the original source role and the relaying source role, or if the later original text fragment contains the earlier original text fragment, or the similarity of the original text fragments is not lower than a preset threshold, then it is determined that the later fragment is transcribed from the earlier one. Source data not transcribed from other source data is identified as independent sources. Source data with transcription relationships are merged into the independent source corresponding to the starting point of their transcription chain. The module then determines whether the values ​​of the merged independent sources are consistent, obtaining a second determination result. The source analysis module can be executed after the candidate field values ​​are written to the shadow table module. If the independent source values ​​are determined to be inconsistent before confirmation, the candidate field values ​​will not be automatically confirmed. After confirmation, it executes in parallel with the document verification module.

[0051] The log module sequentially writes the baseline information, the first judgment result, the second judgment result, and the review result into the log, which is linked by hash values. The log uses an append-only writing method. The hash value of the first record is calculated by concatenating the first payload with an unprecedented value or a fixed initial value. The hash value of each subsequent record is obtained by concatenating the hash value of the previous record with the current payload and then performing SHA-256 calculation. The payload includes landmarks, relative displacement, value core area fingerprint information, qualifying area fingerprint information, qualifying terms, field labels, source time, source role, and the judgment and review results involved in this case.

[0052] The modules described above can be deployed within the same server-side application as different functional components, or they can be deployed as independent microservices. Modules pass data through the business primary key of candidate field values. The shadow table module serves as a shared data storage layer for all modules. The judgment results from the document verification module and the source analysis module are both written to the shadow table module. The log module appends records to the write operations that occur in the shadow table module.

[0053] This invention decomposes the determination of the validity of original records into four non-compensating judgment conditions: location relocation, value kernel comparison, qualifier comparison, and subsequent evidence comparison. Different types of mismatches that may occur after source document revision, such as changes in the value itself, changes in the nature of qualifiers, missing evidence paragraphs, and subsequent evidence coverage, can be identified separately. Value kernel areas are excluded from the candidate range of road signs, and relative displacement is used to achieve cross-version coordinate transport, avoiding the common anchor point misjudging revisions as unchanged when source document revisions occur in adjacent value positions. The consistency judgment between multiple source data is based on the identification of transcription relationships; source data with transcription relationships are merged first, and then the consistency of values ​​between independent sources is judged. Baseline information is incorporated into the hash chain log payload, and subsequent tampering with baseline information can be detected through log integrity verification. In specific implementations, the above processing methods can be adjusted according to business scenarios, such as preset thresholds, time window ranges, and normalization rules, without changing the basic processing ideas of decomposition judgment, transcription merging, and hash chain tracing.

[0054] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for evaluating AI-generated fields based on shadow table isolation, characterized in that, Includes the following steps: S1. Obtain the candidate field values ​​generated by AI and the original text position of the candidate field values ​​in the source document. Write the candidate field values ​​into a shadow table. The shadow table is physically isolated from the formal data table. The formal data table is the data table into which the candidate field values ​​are written after confirmation or review. S2. In response to manual confirmation or rewriting operations for candidate field values, extract the original content of the candidate field values ​​from the source document version corresponding to the original document version when the operation is performed, based on the original text position, as the original record, and extract baseline information, and write the original record and baseline information into the shadow table. S3. When the source document is updated or a new source document is received, the content corresponding to the original record is relocated in the current source document based on the baseline information. The relocated content and its context are compared with the original record. The current source document is checked to see if there is data in the current source document whose source time is later than the original record and whose value is different, which covers the field corresponding to the candidate field value. Based on whether the relocation was successful, whether the comparison was consistent, and whether there was any overwriting, the first determination result of whether the original record is invalid is obtained; S4. Obtain the source data of the candidate field value corresponding to the field in multiple source documents. The source data includes source time, source role and original text fragment. Based on the time sequence, role relationship and original text fragment association between the source data, identify the copying relationship between the source data, merge the source data with copying relationship into the same independent source, determine whether the values ​​of each independent source are consistent after merging, and obtain the second determination result. S5. When the first judgment result is that the original record is invalid, or the second judgment result is that the values ​​of independent sources are inconsistent, a review task for the candidate field values ​​is generated, relevant information for review and adjudication is presented, the review result for the review task is received, and the review result is written to the shadow table. S6. Write the baseline information, the first judgment result, the second judgment result, and the review result into the log in sequence. The records in the log are linked together by hash value.

2. The AI-generated field evaluation method based on shadow table isolation as described in claim 1, characterized in that, Step S2 specifically includes: S21. Based on the original text location, extract the neighboring text containing the original record from the source document version corresponding to the manual confirmation or rewriting operation, and divide the neighboring text into a value core area and a limiting clause area according to the clause boundaries. The value core area corresponds to the original record, and the limiting clause area corresponds to the context of the original record. S22. Extract landmarks within the neighborhood after excluding the core area and calculate the relative displacement of the landmarks relative to the starting point of the core area. S23. Extract fingerprint information from the value kernel area and the qualifier area respectively, and extract the qualifiers belonging to the preset qualifier set from the qualifier area; S24. Extract field labels within the vicinity of the value kernel area and record the source time and source role of the candidate field values; S25. Write the original record into the shadow table, and write the landmark, relative displacement, value core area fingerprint information, qualifier area fingerprint information, qualifier, field label, source time and source role into the shadow table as baseline information.

3. The AI-generated field evaluation method based on shadow table isolation as described in claim 2, characterized in that, In step S22, the neighborhood search window is determined according to the length of the value kernel region. Within the neighborhood search window, excluding the value kernel region, the shortest substring that satisfies the uniqueness condition is selected as the landmark. The relative displacement of the landmark is calculated using the following formula: ； in This is relative displacement. To determine the starting character position of the value core area, For the selected road signs, This indicates the starting character position of the substring in the source document version corresponding to the manual confirmation or rewrite operation.

4. The AI-generated field evaluation method based on shadow table isolation as described in claim 2, characterized in that, In step S23, normalization is performed on the value kernel region and the qualifying region respectively before calculating the fingerprint information: ； ； in To obtain the fingerprint information of the core area, This refers to the fingerprint information of the restricted text area, where D represents the source document version corresponding to the manual confirmation or rewrite operation. For the value core region, For the qualifier section, This indicates normalization processing. This represents a hash operation; after distinguishing the qualifier words, the intersection with the preset qualifier set is taken to obtain the qualifier.

5. The AI-generated field evaluation method based on shadow table isolation as described in claim 2, characterized in that, Step S3 specifically includes: S31. Based on landmarks and relative displacement, relocate the value core area and the qualifier area in the current source document to obtain the location result; S32. Extract fingerprint information from the repositioned value core area and compare it with the value core area fingerprint information in the baseline information to obtain the value core area comparison result; S33. Extract fingerprint information and limiting terms from the repositioned limiting term region, and compare them with the limiting term region fingerprint information and limiting terms in the baseline information to obtain the limiting term comparison result. S34. Based on field labels, source time, and source role, retrieve later data in the current source document whose source time is later than the original record and whose values ​​are different, and obtain the later coverage detection results. S35. When the positioning result is that repositioning is unsuccessful, or the value core area comparison result or the qualifying statement comparison result is inconsistent, or the subsequent coverage detection result is that coverage exists, the original record corresponding to the candidate field value is determined to be invalid, and the first determination result is obtained.

6. The AI-generated field evaluation method based on shadow table isolation as described in claim 5, characterized in that, Step S31 specifically includes: S311. Find the starting character position of the landmark in the current source document, add it to the relative displacement to obtain the candidate positioning position, and perform matching in the vicinity of the candidate positioning position to obtain the repositioned value core area position; when the landmark is not unique in the current source document or no repositioned value core area position is matched, in the neighborhood range after excluding the value core area, select the next unselected substring in the order of increasing length of the substring that meets the uniqueness condition to replace the landmark, recalculate the relative displacement and repeat this step until the repositioned value core area position is obtained or no unselected substring exists; S312. When there is no unselected substring and the value core area position after repositioning has not been obtained, a matching method different from character matching is used to match the neighboring text to obtain the value core area position after repositioning or the positioning result is that the repositioning is unsuccessful. S313. Based on the positional relationship between the repositioned value core area and the qualifier area, the repositioned qualifier area is obtained, and the positioning result is obtained.

7. The AI-generated field evaluation method based on shadow table isolation as described in claim 5, characterized in that, In step S34, data matching the field label and belonging to the same source time window is searched in the current source document to obtain the value, source time, and source role of the matching data. When the matching data simultaneously meets the following criteria: the source role is the preset original source role, the source time is later than the source time in the baseline information, and the value does not meet the equivalence criterion with the candidate field value, it is determined that there is coverage. The coverage detection result is then obtained by the following formula: ； in For subsequent coverage detection results, S=0 indicates the presence of coverage, and S=1 indicates the absence of coverage. To match the source role of the data, As the default original source character, To match the source time of the data, The source time is the baseline information. To match the values ​​of the data, For candidate field values, This serves as an equivalence criterion.

8. The AI-generated field evaluation method based on shadow table isolation as described in claim 7, characterized in that, In step S4, the source role includes a preset original source role and a relay source role; The identification of the transcription relationship between source data specifically includes: For any first source data and second source data, provided that the source time of the first source data is no later than the source time of the second source data, and the values ​​of the first source data and the second source data satisfy a preset equivalence criterion, the second source data is determined to be copied from the first source data if any of the following conditions are met: ①The source role of the first source data is the original source role, and the source role of the second source data is the relay source role; ②The original text fragments of the second source data contain the original text fragments of the first source data; ③ The similarity between the original text fragments of the first source data and the second source data is not lower than the preset threshold.

9. The AI-generated field evaluation method based on shadow table isolation as described in claim 8, characterized in that, In step S4, source data that is not copied from other source data is identified as independent source data, and source data with copying relationship is merged into the independent source corresponding to the starting point of its copying chain; when the number of independent sources is not less than two, and the values ​​of at least two of the independent sources do not meet the equivalence criterion, it is determined that the values ​​of the independent sources are inconsistent, and a second determination result is obtained.

10. An AI-generated field evaluation system based on shadow table isolation, characterized in that, The system is used to implement the method as described in any one of claims 1-9, comprising: The shadow table module is physically isolated from the formal data table, which is the data table written after the candidate field values ​​are confirmed or reviewed. It obtains and stores the candidate field values ​​generated by AI and their original positions in the source document, as well as the original records, baseline information, first judgment result, second judgment result, and review result. When the first judgment result is that the original record is invalid or the second judgment result is that the values ​​from independent sources are inconsistent, a review task for the candidate field value is generated, relevant information for review and adjudication is presented, and the review result is received and stored. The document verification module responds to manual confirmation or rewriting operations for candidate field values. Based on the original text position, it extracts the original text content of the candidate field value from the source document version corresponding to the operation as the original record and extracts baseline information. When the source document is updated or a new source document is received, it relocates the content corresponding to the original record in the current source document based on the baseline information. It compares the relocated content and its context with the original record and detects whether there is data in the current source document with a later source time and different value that covers the field corresponding to the candidate field value. Based on whether the relocation is successful, whether the comparison is consistent, and whether there is overwriting, a first determination result is obtained as to whether the original record is invalid. The source analysis module obtains the source data of the candidate field value corresponding to the field in multiple source documents. The source data includes source time, source role, and original text fragment. Based on the time sequence, role relationship, and original text fragment association between the source data, the module identifies the transcription relationship between the source data, merges the source data with transcription relationship into the same independent source, and determines whether the values ​​of each independent source are consistent after merging to obtain a second determination result. The log module writes the baseline information, the first judgment result, the second judgment result, and the review result into the log in sequence. The records in the log are linked by hash values.

Images