Heterogeneous medical big data version conflict detection and field-level adaptive repair method and system, electronic device and storage medium

Abstract

The application discloses a heterogeneous medical big data version conflict detection and field-level adaptive repair method, system, electronic equipment and storage medium. The method comprises version fingerprint calculation, conflict calculation, ambiguity resolution, repair, storage evidence tracing and the like. The application can solve the defects of the prior art, such as dependence on manual configuration, low version identification accuracy, coarse conflict detection granularity, low ambiguity mapping resolution efficiency, excessive repair range and lack of traceability.

Description

Technical Field

[0001] This application belongs to the technical field of medical data processing, specifically relating to a method, system, electronic device, and storage medium for heterogeneous medical big data version conflict detection and field-level adaptive repair. Background Technology

[0002] Clinical disease databases are core data infrastructure supporting medical research and clinical decision-making. They typically aggregate medical data from multiple heterogeneous medical information systems, such as HIS (Hospital Information System), EMR (Electronic Medical Record System), LIS (Laboratory Information System), PACS (Picture Archiving System), and pharmacy management systems. These heterogeneous medical information systems are independent in terms of construction time, vendor selection, and version iteration rhythm. This results in the same type of medical concept using different versions of coding standards in different systems and at different historical periods, leading to a situation where multiple versions of codes coexist in the same field of the clinical disease database. Taking disease diagnosis coding as an example, ICD-9, ICD-10 standard version, ICD-10 national clinical version 1.0, and ICD-10 national clinical version 2.0 are used in parallel in hospitals at all levels in China. The same disease, "type 2 diabetes," corresponds to codes 410.00, E11, E11.900, and E11.902 in the above four versions, respectively. The literal values ​​of the four codes are completely different, but they all point to the same clinical concept. Regarding surgical procedure coding, the Beijing version, the Shanghai version, and the national clinical version of ICD-9-CM-3 coexist; regarding laboratory test coding, three systems are used interchangeably: the LOINC international version, the National Health Commission's laboratory test code standard, and hospital-defined coding. This situation is even more pronounced when multiple hospital campuses are merged to create a new database.

[0003] In the existing technology, there are three main types of mainstream technical solutions to address the above problems.

[0004] The first type is the manually configured version mapping rule table solution. Data administrators manually compile the coding versions of each system and create a version mapping rule table. When data is entered into the database, a unified version conversion is performed according to the rule table. This solution has three drawbacks: First, manual compilation is costly and prone to omissions. In scenarios involving dozens of systems and hundreds of coding fields during hospital mergers, the manual compilation process can take months. Second, the mapping rule table is statically configured and cannot adapt to continuous iterations of coding standards; each standard version upgrade requires a complete re-compilation. Third, manual configuration lacks a quality verification mechanism, making it difficult to detect mapping errors.

[0005] The second type is version identification schemes based on exact matching of encoded literals. This method identifies the version by precisely matching the encoded literal with a known version encoding dictionary. This scheme has two drawbacks: First, it can only identify codes already included in the dictionary and cannot identify custom codes or legacy codes not included in the dictionary; second, different versions may have the same encoded literal value but different semantics (such as some overlapping ICD-9 and ICD-10 encoded literals), which can lead to misclassification of the version through exact matching.

[0006] The third type is a unified data restoration solution for the entire row, which performs a unified version conversion on the entire patient record after detecting a version conflict. This solution has two drawbacks: First, reprocessing the entire row may introduce the risk of accidental modification of non-coded fields, compromising the integrity of the original data; second, it lacks field-level version snapshot records, making it unable to support comparison of differences before and after restoration, one-click rollback, and quantitative assessment of the credibility of research data, thus failing to meet the regulatory compliance requirements for medical data.

[0007] Furthermore, existing solutions generally employ field-level binary judgments (i.e., only checking whether a conflict exists) in conflict detection granularity, failing to quantify the severity of conflicts and resulting in a lack of basis for prioritizing repairs; they generally rely on manual, item-by-item review for handling ambiguous mappings, which is inefficient and inconsistent; and they lack traceability of repair operations, failing to record version snapshots of repair operations, thus unable to support comparisons of differences before and after repair, one-click rollback, and quantitative assessment of the credibility of scientific research data, and failing to meet the regulatory compliance requirements for medical data. Summary of the Invention

[0008] To address one or more of the aforementioned problems, the first aspect of this application provides a method for detecting and adaptively repairing version conflicts in heterogeneous medical big data at the field level; the second aspect provides a system for detecting and adaptively repairing version conflicts in heterogeneous medical big data at the field level; the third aspect provides an electronic device; and the fourth aspect provides a storage medium for use in this application.

[0009] The technical solution of this application is as follows.

[0010] The first aspect is a method for detecting and adaptively repairing version conflicts in heterogeneous medical big data at the field level, including: The code set of the target field in the clinical disease database is obtained and preprocessed. Then, a multidimensional statistical feature vector is extracted from the code set. The multidimensional statistical feature vector is matched with the pre-stored version fingerprint reference database to obtain the version identification result and identification confidence. When the identification confidence level is lower than the preset confidence threshold, the target field is marked as version uncertain and pushed for manual confirmation; when the identification confidence level is not lower than the preset confidence threshold, each code in the code set is classified according to its version, and the corresponding number of coexisting versions and version distribution vector are determined; when there is only one coexisting version, the conflict intensity index is defined as zero; when there is more than one coexisting version, the conflict intensity index is calculated based on the normalized information entropy of the version distribution vector. Based on the conflict intensity index and two preset trigger thresholds, tiered disambiguation is performed on conflicting codes. Tiered disambiguation includes at least three levels: routine monitoring, automatic repair, and mandatory manual review. For codes that trigger automatic repair, their mapping table is queried to determine the mapping type. If a mapping record exists in the mapping table, codes judged as one-to-one mappings are set as directly replaceable. For codes judged as one-to-multi-ambiguous mappings, multi-dimensional clinical context features of the same patient record are extracted. The matching score is obtained by weighted similarity calculation between the multi-dimensional clinical context features and the corresponding clinical feature profiles of the multi-ambiguous codes. Based on the matching score, the multi-ambiguous codes that can be replaced are determined. If no mapping record exists in the mapping table, mandatory manual review is initiated. For conflicting codes in the target field, perform field-level targeted replacement and generate a field-level version snapshot record based on the determined codes that can be used for replacement; field-level targeted replacement only performs the replacement operation on the field where the conflicting codes are located; Calculate the hash value for each version snapshot record and construct a field-level hash chain. Then, write the hash value corresponding to the initial version snapshot of the hash chain into the audit log of the root node in the hash chain.

[0011] As one implementation of the first aspect, the multidimensional statistical feature vector includes one or more of the following: encoding length distribution vector, first character type distribution vector, decimal point proportion, pure number proportion, and first character frequency distribution vector; Similarity matching uses cosine similarity calculation to select the version with the highest similarity value as the version identification result; The highest similarity value is set as the recognition confidence level.

[0012] As one implementation of the first aspect, when the conflict intensity index is lower than the first of the two trigger thresholds, only routine monitoring is performed. Automatic repair is performed when the conflict intensity index is between two trigger thresholds; When the conflict intensity index is not lower than the second trigger threshold among the two trigger thresholds, a mandatory manual review is performed and repair suggestions are generated simultaneously.

[0013] As one implementation of the first aspect, the multi-dimensional clinical context features include one or more of the following: a set of abnormal test indicators within a preset time window before consultation, a set of medication records within a preset time window before consultation, a set of other diagnostic codes during the same period, and a set of surgical operation codes within a preset time window before consultation. The weighted similarity calculation adopts the generalized weighted Jaccard similarity method, which sums the Jaccard similarities of different dimensions according to their weight vectors to obtain the matching score.

[0014] As one implementation of the first aspect, the version snapshot record includes one or more of the following fields: snapshot identifier, record identifier, field name, version number, field value before operation, field value after operation, source version identifier, target version identifier, creation timestamp, operation event type, repair basis, conflict intensity index value, operation subject identifier, and preceding hash value. Operation event types include one or more of the following: initial inbound event, automatic repair event, assisted confirmation repair event, manual review repair event, and rollback operation event.

[0015] As one implementation method of the first aspect, the hash value is calculated as follows: Calculate the SHA-256 hash value of the concatenated string of core fields in the version snapshot record; The core fields include one or more of the following: snapshot identifier, record identifier, field name, version number, field value before operation, field value after operation, source version identifier, target version identifier, creation timestamp, operation event type, operation subject identifier, and preceding hash value. The preceding hash value in the SHA-256 hash value of the initial version snapshot record is set to a preset initial value. The SHA-256 hash value of the current version snapshot record is used as the preceding hash value of the SHA-256 hash value of the next version snapshot record to form a hash chain.

[0016] As one implementation of the first aspect, it also includes performing integrity verification on the hash chain; Integrity verification includes first-level root node comparison and second-level hierarchical chain verification; The first-level root node comparison involves comparing the root node hash value stored in the root node's audit log with the current hash value of the corresponding field's initial version snapshot record. The second-level hierarchical chain verification is as follows: in order of version number, the preceding hash value of each version snapshot record is verified to see if it is equal to the current hash value of the previous version snapshot record.

[0017] Secondly, a heterogeneous medical big data version conflict detection and field-level adaptive repair system includes: The version fingerprint calculation module is used to obtain the encoding set of the target field in the clinical disease database and perform preprocessing. Then, it extracts multidimensional statistical feature vectors from the encoding set and performs similarity matching between the multidimensional statistical feature vectors and the pre-stored version fingerprint reference database to obtain the version identification result and identification confidence. The conflict calculation module is used to mark the target field as having an uncertain version and push it for manual confirmation when the identification confidence level is lower than a preset confidence threshold; when the identification confidence level is not lower than the preset confidence threshold, it classifies each code in the code set according to its version and determines the corresponding number of coexisting versions and the version distribution vector; when there is only one coexisting version, the conflict intensity index is defined as zero; when there is more than one coexisting version, the conflict intensity index is calculated based on the normalized information entropy of the version distribution vector. The ambiguity resolution module performs tiered disambiguation on conflicting codes based on a conflict intensity index and two preset trigger thresholds. Tiered disambiguation includes at least three levels: routine monitoring, automatic repair, and mandatory manual review. For codes that trigger automatic repair, the module queries their mapping table and determines the mapping type. If a mapping record exists in the mapping table, codes with a one-to-one mapping type are set as directly replaceable. For codes with a one-to-many ambiguous mapping type, the module extracts multi-dimensional clinical context features from the same patient record. The matching score is calculated by weighted similarity calculation between the multi-dimensional clinical context features and the corresponding clinical feature profile of the ambiguous code. The matching score determines the ambiguous code that can be replaced. If no mapping record exists in the mapping table, the module proceeds to mandatory manual review. The repair execution module is used to perform field-level targeted replacement and generate field-level version snapshot records for conflicting codes in the target field, based on the determined codes that can be used for replacement; the field-level targeted replacement only performs the replacement operation on the field where the conflicting code is located. The evidence storage and traceability module is used to calculate the hash value of each version snapshot record and build a field-level hash chain. Then, the hash value of the initial version snapshot of the hash chain is written to the audit log of the root node in the hash chain.

[0018] In a third aspect, an electronic device includes a memory, a processor, and a computer program stored on the memory and running on the processor, wherein the processor, when executing the computer program, implements the heterogeneous medical big data version conflict detection and field-level adaptive repair method of the first aspect.

[0019] Fourthly, a storage medium is a computer-readable storage medium storing computer instructions for enabling a computer to implement the heterogeneous medical big data version conflict detection and field-level adaptive repair method of the first aspect.

[0020] Compared with the prior art, the advantages of this application are as follows: It eliminates reliance on manual configuration and adapts to the continuous iteration of coding standards; it enables quantitative calculation and priority ranking of conflicts; it utilizes clinical context to accurately resolve one-to-many ambiguity mappings; it achieves minimum-granularity repair at the field level to avoid the risk of accidental modification of non-coded fields; and it uses tamper-proof hash chain evidence storage and trusted quantification assessment methods to achieve data security. Attached Figure Description

[0021] Figure 1 This is a flowchart illustrating one of the heterogeneous medical big data version conflict detection and field-level adaptive repair methods in this application.

[0022] Figure 2 This is a schematic diagram of the structural framework of one of the heterogeneous medical big data version conflict detection and field-level adaptive repair systems in this application.

[0023] Figure 3 This is a schematic diagram of the frame of one of the electronic devices in this application. Detailed Implementation

[0024] Referring to the illustrations, the principles of this application are illustrated by way of example implementation in a suitable operating environment. The following description is based on the illustrative specific embodiments of this application and should not be construed as limiting other specific embodiments not detailed herein, such as corresponding adjustments to the order of steps of the method of this application based on the principles of the technical concept.

[0025] It should be noted that this application only standardizes the names of examination items in the medical institution's examination data and does not involve sensitive medical data such as patient personal identification information, medical record content, and diagnostic conclusions. It complies with the relevant requirements of laws, regulations, and ethical norms regarding de-identification and minimal processing of personal information.

[0026] like Figure 1 As shown in the figure, this application discloses a method for detecting version conflicts and adaptively repairing field-level conflicts in heterogeneous medical big data, including the following steps.

[0027] S1. Obtain the encoding set S(F) of the target field F in the clinical disease database, then extract the multidimensional statistical feature vector from the encoding set S(F), and perform similarity matching between the multidimensional statistical feature vector and the pre-stored version fingerprint reference database to obtain the version identification result and identification confidence.

[0028] In this embodiment, the multidimensional statistical feature vector includes one or more of the following: encoding length distribution vector, first character type distribution vector, decimal point ratio, pure number ratio, and first character frequency distribution vector.

[0029] Optionally in this embodiment, the version with the highest similarity value is selected as the version identification result by calculating the cosine similarity, and the highest similarity value is set as the identification confidence level.

[0030] In step S1, different versions of medical coding standards exhibit statistically significant differences in character structure, length distribution, and prefix patterns. These differences constitute the statistical fingerprint features of the corresponding versions. Taking common coding standards as examples, ICD-9 primarily uses pure numeric encoding with a length concentrated between 3 and 5 characters. ICD-10 Clinical Version 1.0 has a encoding length concentrated between 5 characters, includes a decimal point, and begins with a letter. ICD-10 Clinical Version 2.0 has an encoding length concentrated between 7 characters and begins with a letter. LOINC encoding is entirely numeric and its length is concentrated between 5 and 7 characters. In contrast, most hospitals' custom internal coding is typically a mixed structure of pure numeric characters or specific prefixes. These differences can be accurately captured by multidimensional statistical feature vectors and used for automatic version inference.

[0031] In this embodiment, the encoding set S(F) can be preprocessed before the multidimensional statistical feature vector: empty strings and NULL values ​​are filtered out, and invalid values ​​do not participate in the statistical calculation of any feature components; if the number of valid codes is lower than the preset minimum sample threshold after filtering, the target field F is marked as insufficient sample, the version fingerprint calculation is not performed, and it is pushed to manual confirmation. This preprocessing ensures that all components of the multidimensional statistical feature vector can be calculated based on valid codes, avoiding recognition bias caused by sparse data.

[0032] In this embodiment, optionally, the multidimensional statistical feature vector is constructed as follows: multiple statistical features are extracted from the preprocessed encoding set and concatenated into a normalized multidimensional statistical feature vector FP(F), where all components of the multidimensional statistical feature vector FP(F) remain in the [0,1] interval; in this embodiment, it is preferred to extract four types of features and concatenate them into a 57-dimensional feature vector: (1) Coding length distribution vector (16 dimensions): The frequency of character length distribution of the statistical coding; the corresponding dimension components are formed by the proportion of coding with length L (L ranges from 0 to 15), and the sum of each component is 1. Codings with a length of more than 15 bits are uniformly included in the 16th dimension, that is, the physical meaning of the 16th dimension is "the proportion of coding with a length ≥ 15 bits"; the coding length in the existing mainstream medical coding standards does not exceed 15 bits. The super long coding is usually a hospital-defined non-standard format. Therefore, merging it into the highest bit component has a negligible impact on version differentiation. Setting it to 16 dimensions can avoid the high-dimensional sparsity problem at the same time. (2) First character type distribution vector (3-dimensional): The proportion of the first character of the code is uppercase letter, numeric character, and other characters, forming a 3-dimensional normalized vector. The sum of the three components is 1. The first character of ICD-9 is almost entirely numeric, while the first character of ICD-10 clinical version 2.0 is more than 96% uppercase letter. There is a significant difference between the two in this dimension. (3) Decimal point ratio (1-dimensional) and pure number ratio (1-dimensional): Decimal point ratio is the ratio of codes containing decimal points, and pure number ratio is the ratio of codes where all characters are numbers; both can distinguish between ICD-10 clinical version codes containing decimal points, pure number LOINC codes without decimal points, and hospital-customized codes with mixed structures. (4) First character frequency distribution vector (36-dimensional): A predefined standard character set is used, which includes 26 uppercase letters A to Z and 10 numeric characters 0 to 9. It covers all legal first character types in the existing mainstream medical coding standards. For each code, the first character is taken and the frequency of each character is counted to form a 36-dimensional normalized vector. For codes whose first character does not belong to the standard character set, the "other characters" component of the first character type distribution is set to capture the first character.

[0033] In this embodiment, the multidimensional statistical feature vector FP(F) of the normalized numerical value is compared with the standard fingerprint vector of each version entry in the version fingerprint reference library. The version corresponding to the highest cosine similarity value is taken as the identification result Version(F), and the similarity value is the identification confidence Conf(F).

[0034] In this embodiment, each version entry in the version fingerprint reference library is obtained by randomly selecting a sufficient number of samples from the official release file of the corresponding version coding standard in the same manner as step S1, and is configured to support dynamic expansion to cover new version coding standards and the hospital's custom coding system; when the confidence level is lower than the preset confidence threshold, the field is marked as an uncertain version and pushed for manual confirmation, without triggering the subsequent automatic repair process.

[0035] S2. When the identification confidence level is lower than the preset confidence threshold, the target field is marked as version uncertain and then pushed for manual confirmation. When the identification confidence level is not lower than the preset confidence threshold, each code in the code set is classified according to its version and the corresponding number of coexisting versions K and version distribution vector are determined. When there is only one coexisting version K, the conflict intensity index VCSI(F) is defined as zero. When there is more than one coexisting version K, the conflict intensity index VCSI(F) is calculated based on the normalized information entropy of the version distribution vector.

[0036] Optionally in this embodiment, when the number of concurrent versions K is greater than 1, the calculation formula for the conflict intensity index VCSI(F) is as follows: ; in, Let pᵢ be the information entropy of the version distribution vector, and pᵢ be the encoding proportion of the i-th version.

[0037] Optionally, in this embodiment, a first trigger threshold and a second trigger threshold are set, and the first trigger threshold is set to be significantly smaller than the second trigger threshold. When the conflict intensity index VCSI(F) is lower than the first trigger threshold, subsequent graded disambiguation only performs routine monitoring. When the conflict intensity index VCSI(F) is between the two trigger thresholds, subsequent graded disambiguation is forced to perform automatic repair. When the conflict intensity index VCSI(F) is not lower than the second trigger threshold, subsequent graded disambiguation is forced to be manually reviewed and repair suggestions are generated simultaneously.

[0038] Optionally in this embodiment, after step S1 completes the overall version inference of the target field, step S2 queries the standard version mapping table for each code in the code set S(F), classifies them according to their respective versions, and then counts the actual number of coexisting versions K in the target field F and the code proportion of each version to form a version distribution vector P(F)={p1, p2, ..., p k The sum of all components is 1.

[0039] In this embodiment, optionally, taking the number of coexisting versions K as 2 as an example, the first trigger threshold can be set to 0.30 and the second trigger threshold to 0.60. When the conflict intensity index VCSI(F) is 0.30, the corresponding version code ratio is about 10%, which is enough to introduce statistical bias, so it is set as the trigger lower limit for automatic repair. When the conflict intensity index VCSI(F) is 0.60, the corresponding version code ratio is about 35%. When this ratio is exceeded, the reliability of automatic repair decreases, so it is set as the trigger threshold for forced manual review.

[0040] In this embodiment, optionally, after calculating the Conflict Intensity Index (VCSI) (F) for all coded target fields in the clinical disease database, a conflict heatmap matrix is ​​generated with the target field name on the horizontal axis and the VCSI (F) value mapped to different color depths. The target fields are then sorted in descending order of VCSI (F) value to automatically determine the highest priority repair target.

[0041] In this embodiment, optionally, time windows are divided according to the timestamp of data entry into the database. The conflict intensity index (VCSI) (F) values ​​within each time window are calculated to form a time-series trend sequence. A minimum preset number of valid windows is required for effective time-series analysis. If the number of valid windows is insufficient, conflict detection is skipped, and only the current VCSI (F) distribution is output, without generating trend alarms to avoid false alarms due to insufficient sample size. The optional effective time-series analysis window processing rules are: linear interpolation is performed to fill no more than two consecutive empty windows; more than two consecutive empty windows are marked as missing data segments; conflict detection skips window pairs with missing segments; when the change in VCSI (F) between two adjacent valid windows exceeds a preset mutation threshold, it is determined to be a version conflict emergency. Then, the upstream data source change log is automatically associated to locate the possible version upgrade time point, assisting data administrators in tracing the source of the conflict.

[0042] S3. Based on the Conflict Intensity Index (VCSI) (F) and the preset first and second trigger thresholds, perform graded disambiguation on conflicting codes. Graded disambiguation includes at least three levels: routine monitoring, automatic repair, and forced manual review. For codes that trigger automatic repair, query their mapping table and determine the mapping type. If there is a mapping record in the mapping table, codes that are judged to be one-to-one mapping types are set as directly replaceable. For codes that are judged to be one-to-many ambiguous mapping types, extract multi-dimensional clinical context features from the same patient record. Calculate the matching score by weighted similarity between the multi-dimensional clinical context features and the corresponding clinical feature profile mapped to the multi-ambiguous code. Determine the multi-ambiguous code that can be replaced based on the multiple matching scores of the corresponding multiple dimensions. If there is no mapping record in the mapping table, proceed to forced manual review.

[0043] In this embodiment, the multi-dimensional clinical context features include one or more of the following: a set of abnormal test indicators within a preset time window before the visit, a set of medication records within a preset time window before the visit, a set of other diagnostic codes during the same period, and a set of surgical operation codes within a preset time window before the visit. The weighted similarity calculation uses a generalized weighted Jaccard similarity method, where the matching score Simlab is obtained by weighting and summing the Jaccard similarities of different dimensions according to their weight vectors. The calculation method is as follows: Simlab = Σⱼ (wAj × wBj) / Σⱼ max(wAj, wBj); Where j represents the union of multi-dimensional clinical context features and clinical feature profiles; wAj is the existence weight of multi-dimensional clinical context features, which is 1 when the item is abnormal and 0 when it is absent or normal; wBj is the frequency weight of clinical feature profiles, which is the frequency of occurrence of the item in historical cases; when the union is empty, the corresponding dimension score is 0. The calculation method for medication, diagnosis, and surgery dimensions is the same as that for test dimensions.

[0044] Optionally, in this embodiment, a corresponding difference threshold, a first score threshold, and a second score threshold are set for the matching scores. When the highest matching score among multiple matching scores is not lower than the first score threshold and the difference between it and the second highest matching score is not lower than the difference threshold, it is pushed to automatic repair. When the highest matching score is between the second score threshold and the first score threshold and the difference between it and the second highest matching score is not lower than the difference threshold, it is pushed to auxiliary confirmation, and subsequent repair is performed after manual confirmation. When the highest matching score is lower than the second score threshold or the difference between it and the second highest matching score is lower than the difference threshold, it is pushed to mandatory manual review, and subsequent repair is performed only after a mandatory manual review result. Further, in this embodiment, a timeout limit is set for the content queue entering the repair operation. If a timeout occurs, and the highest matching score is not lower than a preset downgrade score threshold and the difference between the highest matching score and the second highest matching score is not lower than a preset downgrade difference threshold, then automatic downgrade repair is performed. If the aforementioned downgrade conditions are not met, the repair operation is upgraded to mandatory manual review.

[0045] In this embodiment, the mapping type of each conflict code in the target field F is determined by querying the standard version of the mapping relationship table, and then the corresponding processing path is set as follows: (a) One-to-one deterministic mapping: The old version code has a unique target code in the new version, which is set to allow direct field replacement; (b) One-to-many ambiguous mapping: the old version encoding corresponds to two or more candidate target encodings in the new version, and is set to enter the process of calculating the matching score using multi-dimensional clinical context features; (c) No mapping record: The old version code has no corresponding record in the new version, and directly enters the mandatory manual review without automatic repair.

[0046] In this embodiment, the boundary handling of the number of mapped polyambiguous codes is as follows: when the number of codes mapped to by an old version code in the mapping relationship table is exactly 1, it is logically a one-to-one deterministic mapping and field replacement can be directly performed; when the number of codes mapped to by an old version code exceeds 1, the corresponding code corresponds to a Profile entry in the clinical feature profile library, recording the four types of features typically associated with the code in historical clinical data and their frequency weights. The four types of features are the set of typical abnormal test indicators, the set of typical medication features, the set of typical accompanying diagnoses, and the set of typical surgical operations. The weight value of each type of feature is the frequency of occurrence of the feature in the historical cases of the code.

[0047] Optionally in this embodiment, the clinical feature profiling database uses a three-stage cold start scheme to address the circular dependency problem when data is insufficient. The three stages are as follows: Phase 1 (Pre-built General Library): During initial deployment, a pre-built Profile library is used, which is built based on publicly available clinical benchmark datasets and domestic clinical diagnosis and treatment guidelines feature libraries. It covers typical clinical features of common codes and can be used directly. Phase Two (Hybrid Transition): As in-hospital manual verification data accumulates, in-hospital data will be gradually introduced according to the hybrid weighting formula; Phase 3 (purely hospital-based data-driven): After the accumulated manually verified data within the hospital exceeds the pre-set Profile library, the Profile entries are set entirely by hospital-based data, updated regularly on a quarterly basis, and continuously reflect the distribution of real clinical characteristics within the hospital.

[0048] S4. For conflicting codes in the target field, perform field-level targeted replacement and generate a field-level version snapshot record based on the determined codes that can be used for replacement; the field-level targeted replacement only performs the replacement operation on the target field where the conflicting codes are located.

[0049] In this embodiment, the version snapshot record includes one or more of the following fields: snapshot identifier, record identifier, field name, version number, field value before operation, field value after operation, source version identifier, target version identifier, creation timestamp, operation event type, repair basis, conflict intensity index value, operation subject identifier, and preceding hash value; wherein, the operation event type includes one or more of the following: initial entry event, automatic repair event, assisted confirmation repair event, manual review repair event, and rollback operation event.

[0050] In this embodiment, optionally, after determining the hierarchical disambiguation method, a credibility score is also included: all content to be processed is divided into five categories: automatic repair, assisted confirmation repair, manual review repair, unresolved, and version uncertain. The data credibility score is obtained by weighting and summing the scores according to the proportion of each category and a set credibility coefficient. The credibility coefficient for the unresolved category is 1 minus the conflict intensity index of that field, and the credibility coefficient for the version uncertain category is zero. Optionally, in this embodiment, the sum of the proportions of the five categories of records is strictly equal to 1, covering all record types. The automatic repair category uses the average of the highest matching scores of all records, the assisted confirmation repair category has a fixed value of 0.95, the manual review repair category has a fixed value of 1.00, the unresolved category has 1 minus VCSI(F), and the version uncertain category has a fixed value of 0.

[0051] Optionally, in this embodiment, for the first data entry into the clinical disease database, an initial entry version snapshot is generated for each coding field of each record, recording the original entry value of the field; the records before and after the repair of the initial entry version snapshot are both the original entry values, indicating that there is no repair operation for this version, the version number is 1, and the preceding hash value is a fixed initial value; for historical existing data, initial entry version snapshots are generated in batches when the existing data migration scan is performed for the first time. The batch generation process uses database transactions to ensure the atomicity of writing to the main version snapshot table and anchoring to the root node audit log, ensuring that there is no inconsistency in partial writes. S5, calculate the hash value for each version snapshot record and construct a field-level hash chain, and then write the corresponding hash value of the initial version snapshot in the hash chain to the audit log of the root node in the hash chain.

[0052] In this embodiment, the hash value is calculated as follows: (I) Calculate the SHA-256 hash value of the concatenated string of core fields in the version snapshot record; the core fields include one or more of the following: snapshot identifier, record identifier, field name, version number, field value before operation, field value after operation, source version identifier, target version identifier, creation timestamp, operation event type, operation subject identifier, and previous hash value; (II) The preceding hash value in the SHA-256 hash value of the initial version snapshot record is set to the preset initial value, and the SHA-256 hash value of the current version snapshot record is used as the preceding hash value of the SHA-256 hash value of the next version snapshot record to form a hash chain.

[0053] In this embodiment, the integrity of the hash chain is further verified. The integrity verification includes a first-level root node comparison and a second-level hierarchical chain verification. The first-level root node comparison involves comparing the root node hash value stored in the root node's audit log with the current hash value of the corresponding field's initial version snapshot record. The second-level hierarchical chain verification involves verifying, in version sequence, whether the preceding hash value of each version snapshot record is equal to the current hash value of the previous version snapshot record. Optionally, in this embodiment, if the first-level root node comparison fails or the second-level hierarchical chain verification is broken, an alarm is triggered, and subsequent repair operations for the affected fields are locked until manual review and confirmation.

[0054] Optionally in this embodiment, the fields of the version snapshot record are connected by a fixed separator "||". All fields are uniformly converted to UTF-8 encoded strings and then concatenated to ensure cross-platform consistency of hash calculation. After the calculation is completed, the current hash value is filled back into the current version snapshot record and used as the input of the preceding hash value of the next version snapshot, forming an immutable field-level hash chain. The preceding hash value of the initial version snapshot is fixed as a 64-bit "0" string.

[0055] Optionally in this embodiment, to defend against root node forgery attacks where an attacker deletes the entire hash chain and then forges a new chain from the beginning, the following mechanism is set up: When the initial version snapshot of each field hash chain is generated, its current hash value is synchronously written to the physically isolated root node audit log table. This write operation is performed in the same database transaction as the write operation of the main version snapshot table to ensure atomicity. If the main version snapshot table is written successfully but the audit log is not written, or if the audit log is written successfully but the main version snapshot table is not written, a transaction rollback will be triggered to prevent inconsistencies in partial writes.

[0056] Compared with the prior art, the advantages of this application are as follows: Eliminating reliance on manual configuration and adapting to continuous iteration of coding standards, this application uses a version fingerprint method that matches multidimensional statistical feature vectors with similarity to automatically infer the version by extracting statistical patterns of characters. This eliminates the need for manual configuration of version mapping rules for each field. When a new version of the coding standard is released, the new version can be automatically assigned by manually annotating a few low-confidence fields, which significantly reduces the maintenance costs caused by version standard iteration. This application achieves quantitative calculation of conflicts and priority ranking of repairs. It quantifies the severity of version conflicts at the field level by using the conflict intensity index of normalized information entropy and forms a priority ranking of repairs, which improves the configuration efficiency of data processing. At the same time, the conflict detection through the time window can locate the time node of the version upgrade event, providing data support for version iteration tracing. This application utilizes clinical context to accurately resolve one-to-many ambiguous mappings. For scenarios involving one-to-many ambiguous mappings, it extracts four types of clinical context features related to the patient's concurrent testing, medication, diagnosis, and surgery. Weighted Jaccard similarity calculation is then performed on the clinical feature profiles corresponding to the candidate field codes. A three-level processing method is set up to resolve ambiguous mappings, avoiding the shortcomings of traditional solutions that require manual review of each ambiguous mapping. This improves processing efficiency while ensuring accuracy. This application achieves minimal granular repair at the field level, avoiding the risk of accidental modification of non-coded fields. The repair execution module only performs targeted replacement and writing operations on conflicting codes in the target field, while other fields in the same patient record are not triggered to perform any writing operations. This minimizes the repair scope, effectively avoids the risk of accidental modification of non-coded fields that may be introduced by reprocessing the entire row, and ensures the integrity of the original data. Data security is achieved by using tamper-proof hash chain evidence storage and trusted quantification assessment. This application generates a field-level version snapshot record for each repair operation. By using SHA-256 hash chain storage combined with the root node atomic transaction anchoring mechanism, the repair operation achieves end-to-end tamper-proof evidence storage. Through a two-layer integrity verification mechanism (root node comparison and hierarchical chain verification), data tampering behavior can be detected in a timely manner, meeting the regulatory compliance audit requirements of medical data.

[0057] like Figure 2 As shown in the figure, this application discloses a heterogeneous medical big data version conflict detection and field-level adaptive repair system, including: The version fingerprint calculation module is used to obtain the encoding set of the target field in the clinical disease database and perform preprocessing. Then, it extracts multidimensional statistical feature vectors from the encoding set and performs similarity matching between the multidimensional statistical feature vectors and the pre-stored version fingerprint reference database to obtain the version identification result and identification confidence. The conflict calculation module is used to mark the target field as having an uncertain version and push it for manual confirmation when the identification confidence level is lower than a preset confidence threshold; when the identification confidence level is not lower than the preset confidence threshold, it classifies each code in the code set according to its version and determines the corresponding number of coexisting versions and the version distribution vector; when there is only one coexisting version, the conflict intensity index is defined as zero; when there is more than one coexisting version, the conflict intensity index is calculated based on the normalized information entropy of the version distribution vector. The ambiguity resolution module performs tiered disambiguation on conflicting codes based on a conflict intensity index and two preset trigger thresholds. Tiered disambiguation includes at least three levels: routine monitoring, automatic repair, and mandatory manual review. For codes that trigger automatic repair, the module queries their mapping table and determines the mapping type. If a mapping record exists in the mapping table, codes with a one-to-one mapping type are set as directly replaceable. For codes with a one-to-many ambiguous mapping type, the module extracts multi-dimensional clinical context features from the same patient record. The matching score is calculated by weighted similarity calculation between the multi-dimensional clinical context features and the corresponding clinical feature profile of the ambiguous code. The matching score determines the ambiguous code that can be replaced. If no mapping record exists in the mapping table, the module proceeds to mandatory manual review. The repair execution module is used to perform field-level targeted replacement and generate field-level version snapshot records for conflicting codes in the target field, based on the determined codes that can be used for replacement; the field-level targeted replacement only performs the replacement operation on the field where the conflicting code is located. The evidence storage and traceability module is used to calculate the hash value of each version snapshot record and build a field-level hash chain. Then, the hash value of the initial version snapshot of the hash chain is written to the audit log of the root node in the hash chain.

[0058] The system described in the above embodiments is used to execute the corresponding heterogeneous medical big data version conflict detection and field-level adaptive repair methods in the foregoing embodiments, and has the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0059] like Figure 3 As shown, based on the same application concept, corresponding to any of the above embodiments, this application also discloses an electronic device, including a memory, a processor, and a computer program stored on the memory and running on the processor. When the processor executes the computer program, it implements the above-mentioned heterogeneous medical big data version conflict detection and field-level adaptive repair method.

[0060] Specifically, the device includes: a processor 1010, a memory 1020, an input / output interface 1030, a communication interface 1040, and a bus 1050. The processor 1010, memory 1020, input / output interface 1030, and communication interface 1040 are interconnected within the device via the bus 1050.

[0061] The processor 1010 can be implemented using a general-purpose CPU (Central Processing Unit), microprocessor, application-specific integrated circuit (ASIC), GPU (Graphics Processing Unit), or one or more integrated circuits, to implement relevant programs and achieve the technical solutions provided in the embodiments of this specification.

[0062] The memory 1020 can be implemented in the form of ROM (Read Only Memory), RAM (Random Access Memory), static storage device, dynamic storage device, etc. The memory 1020 can store the operating system and other application programs. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program code is stored in the memory 1020 and called by the processor 1010. The input / output interface 1030 is used to connect input / output modules to realize information input and output. Input / output modules can be configured as components in the device (not shown in the figure) or externally connected to the device to provide corresponding functions. Input devices can include keyboards, mice, touch screens, microphones, various sensors, etc., and output devices can include displays, projectors, speakers, vibrators, indicator lights, etc.

[0063] The communication interface 1040 is used to connect a communication module (not shown in the figure) to enable communication between this device and other devices. The communication module can communicate via wired means (such as USB (Universal Serial Bus), network cable, etc.) or wireless means (such as mobile network, WIFI (Wireless Fidelity), Bluetooth, etc.).

[0064] Bus 1050 includes a pathway for transmitting information between various components of the device, such as processor 1010, memory 1020, input / output interface 1030, and communication interface 1040.

[0065] It should be noted that although the above-described device only shows the processor 1010, memory 1020, input / output interface 1030, communication interface 1040, and bus 1050, in specific implementations, the device may also include other components necessary for normal operation. Furthermore, those skilled in the art will understand that the above-described device may only include the components necessary for implementing the embodiments of this specification, and not necessarily all the components shown in the figures.

[0066] The electronic device described in the above embodiments is used to implement the corresponding heterogeneous medical big data version conflict detection and field-level adaptive repair method in any of the foregoing embodiments, and has the beneficial effects of the corresponding method embodiments, which will not be elaborated here.

[0067] Based on the same application concept, and corresponding to any of the above embodiments, this application also discloses a computer-readable storage medium that stores computer instructions for enabling a computer to implement the above-described heterogeneous medical big data version conflict detection and field-level adaptive repair method.

[0068] The computer-readable storage medium of this embodiment includes permanent and non-permanent, removable and non-removable media, and information storage can be implemented by any method or technology. Information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transfer medium, which can be used to store information accessible by a computing device. The computer instructions stored in the storage medium of the above embodiments are used to enable the computer to implement the heterogeneous medical big data version conflict detection and field-level adaptive repair method as described in any of the above embodiments, and have the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0069] The above description is merely a preferred embodiment and the technical principles employed in this application. This application is not limited to the specific embodiments or combinations thereof, and various obvious changes, readjustments, and substitutions that can be made by those skilled in the art will not depart from the scope of protection of this application. Therefore, although this application has been described in detail through the above embodiments, this application is not limited to the above embodiments, and may include more other equivalent embodiments without departing from the concept of this application, the scope of which is determined by the scope of the claims.

Claims

1. A method for detecting version conflicts and adaptively repairing field-level conflicts in heterogeneous medical big data, characterized in that, include: The code set of the target field in the clinical disease database is obtained and preprocessed. Then, a multidimensional statistical feature vector is extracted from the code set. The multidimensional statistical feature vector is matched with the pre-stored version fingerprint reference database to obtain the version identification result and identification confidence. When the identification confidence level is lower than a preset confidence threshold, the target field is marked as having an uncertain version and then pushed for manual confirmation; when the identification confidence level is not lower than the preset confidence threshold, each code in the code set is classified according to its version, and the corresponding number of coexisting versions and version distribution vector are determined; when there is only one coexisting version, the conflict intensity index is defined as zero; when there is more than one coexisting version, the conflict intensity index is calculated based on the normalized information entropy of the version distribution vector. Based on the conflict intensity index and two preset trigger thresholds, hierarchical disambiguation is performed on the conflicting codes. The hierarchical disambiguation corresponds to at least three levels: routine monitoring, automatic repair, and forced manual review. For codes that trigger automatic repair, their mapping relationship table is queried to determine the mapping type. If there is a mapping record in the mapping relationship table, the codes that are determined to be one-to-one mapping types are set as directly replaceable. For codes that are determined to be one-to-multi-ambiguous mapping types, multi-dimensional clinical context features of the same patient record are extracted. The matching score is obtained by weighted similarity calculation of the multi-dimensional clinical context features and the corresponding clinical feature profiles of the multi-ambiguous codes. Based on the matching score, the multi-ambiguous codes that can be replaced are determined. If there is no mapping record in the mapping table, then a mandatory manual review will be initiated. For conflicting codes in the target field, a field-level targeted replacement is performed based on the determined codes that can be used for replacement, and a field-level version snapshot record is generated; the field-level targeted replacement is performed only on the field where the conflicting codes are located; For each version snapshot record, a hash value is calculated and a field-level hash chain is constructed. Then, the hash value corresponding to the initial version snapshot of the hash chain is written to the audit log of the root node in the hash chain.

2. The method for heterogeneous medical big data version conflict detection and field-level adaptive repair according to claim 1, characterized in that, The multidimensional statistical feature vector includes one or more of the following: encoding length distribution vector, first character type distribution vector, decimal point ratio, pure number ratio, and first character frequency distribution vector; The similarity matching is performed by calculating the cosine similarity and taking the version corresponding to the highest similarity value as the version identification result; The highest similarity value is set as the recognition confidence level.

3. The method for heterogeneous medical big data version conflict detection and field-level adaptive repair according to claim 1, characterized in that, When the conflict intensity index is lower than the first of the two trigger thresholds, only routine monitoring is performed; Automatic repair is performed when the conflict intensity index is between two trigger thresholds; When the conflict intensity index is not lower than the second trigger threshold among the two trigger thresholds, a mandatory manual review is performed and a repair suggestion is generated simultaneously.

4. The method for heterogeneous medical big data version conflict detection and field-level adaptive repair according to claim 1, characterized in that, The multi-dimensional clinical context features include one or more of the following: a set of abnormal test indicators within a preset time window before the visit, a set of medication records within a preset time window before the visit, a set of other diagnostic codes during the same period, and a set of surgical operation codes within a preset time window before the visit; The weighted similarity calculation adopts the generalized weighted Jaccard similarity method, which sums the Jaccard similarities of different dimensions according to their weight vectors to obtain the matching score.

5. The method for heterogeneous medical big data version conflict detection and field-level adaptive repair according to claim 1, characterized in that, Version snapshot records include one or more of the following fields: snapshot identifier, record identifier, field name, version number, field value before operation, field value after operation, source version identifier, target version identifier, creation timestamp, operation event type, repair basis, conflict intensity index value, operation subject identifier, and preceding hash value. The operation event types include one or more of the following: initial data entry event, automatic repair event, assisted confirmation repair event, manual review repair event, and rollback operation event.

6. The method for heterogeneous medical big data version conflict detection and field-level adaptive repair according to claim 1, characterized in that, The hash value is calculated as follows: Calculate the SHA-256 hash value of the concatenated string of core fields in the version snapshot record; The core fields include one or more of the following: snapshot identifier, record identifier, field name, version number, field value before operation, field value after operation, source version identifier, target version identifier, creation timestamp, operation event type, operation subject identifier, and preceding hash value. The preceding hash value in the SHA-256 hash value of the initial version snapshot record is set to a preset initial value, and the SHA-256 hash value of the current version snapshot record is used as the preceding hash value of the SHA-256 hash value of the next version snapshot record to form a hash chain.

7. The method for heterogeneous medical big data version conflict detection and field-level adaptive repair according to claim 1, characterized in that, It also includes integrity verification of the hash chain; The integrity verification includes a first-level root node comparison and a second-level hierarchical chain verification. The first-level root node comparison is as follows: the root node hash value stored in the root node's audit log is compared with the current hash value of the corresponding field's initial version snapshot record; The second layer of hierarchical chain verification is as follows: in order of version number, the preceding hash value of each version snapshot record is verified to see if it is equal to the current hash value of the previous version snapshot record.

8. A heterogeneous medical big data version conflict detection and field-level adaptive repair system, characterized in that, include: The version fingerprint calculation module is used to obtain the encoding set of the target field in the clinical disease database and perform preprocessing. Then, it extracts a multidimensional statistical feature vector from the encoding set and performs similarity matching between the multidimensional statistical feature vector and the pre-stored version fingerprint reference database to obtain the version identification result and identification confidence. The conflict calculation module is used to mark the target field as having an uncertain version and push it for manual confirmation when the identification confidence level is lower than a preset confidence threshold; when the identification confidence level is not lower than the preset confidence threshold, it classifies each code in the code set according to its version and determines the corresponding number of coexisting versions and the version distribution vector; when there is only one coexisting version, the conflict intensity index is defined as zero; when there is more than one coexisting version, the conflict intensity index is calculated based on the normalized information entropy of the version distribution vector. The ambiguity resolution module is used to perform graded disambiguation on conflicting codes based on the conflict intensity index and two preset trigger thresholds. The graded disambiguation corresponds to at least three levels: routine monitoring, automatic repair, and forced manual review. For codes that trigger automatic repair, the module queries their mapping table and determines the mapping type. If there is a mapping record in the mapping table, codes that are determined to be one-to-one mappings are set as directly replaceable. For codes that are determined to be one-to-many ambiguous mappings, the module extracts multi-dimensional clinical context features from the same patient record, calculates a matching score by weighted similarity between the multi-dimensional clinical context features and the corresponding clinical feature profiles of the ambiguous codes, and determines the ambiguous codes that can be replaced based on the matching score. If there is no mapping record in the mapping table, then a mandatory manual review will be initiated. The repair execution module is used to perform field-level targeted replacement and generate field-level version snapshot records for conflicting codes in the target field, based on the determined codes that can be replaced. The field-level targeted replacement only performs the replacement operation on the field containing the conflicting encoding; The evidence storage and traceability module is used to calculate the hash value of each version snapshot record and construct a field-level hash chain, and then write the hash value of the initial version snapshot of the hash chain into the audit log of the root node in the hash chain.

9. An electronic device, characterized in that, The method includes a memory, a processor, and a computer program stored in the memory and running on the processor. When the processor executes the computer program, it implements the heterogeneous medical big data version conflict detection and field-level adaptive repair method according to any one of claims 1-7.

10. A storage medium, which is a computer-readable storage medium, characterized in that, The system stores computer instructions for enabling a computer to implement the heterogeneous medical big data version conflict detection and field-level adaptive repair method as described in any one of claims 1-7.

Images