Clinical Information Collection and Management System

By integrating front-end interaction, data anonymization, layout awareness, semantic error correction, and confidence level processing, the system addresses the issues of manual reliance and multimodal recognition errors in clinical data collection. It enables efficient and accurate data transformation into structured data, adapting to heterogeneous cross-center scenarios and improving the accuracy and efficiency of data collection.

CN123120168APending Publication Date: 2026-11-17曾庆伦 +1
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202611030489.6
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2026-07-11
Publication Date
2026-11-17

AI Technical Summary

Technical Problem

Existing technologies suffer from several drawbacks in clinical data collection, including reliance on manual data entry, lack of a unified structured entry point for multimodal data, frequent recognition errors due to image quality issues, and difficulty in adapting to cross-center format differences and complex distortions, resulting in low data accuracy and failing to meet the needs of precision medicine research.

Method used

The system employs a front-end acquisition and interaction module for image quality assessment and data integrity verification, a desensitization module for identity information desensitization, a layout awareness and multimodal extraction module for reconstruction and recognition, a semantic error correction and encoding normalization module for optimization, a confidence and human-computer collaboration module for traffic diversion, and a data knowledge base module to provide template data and confidence thresholds, forming a self-reinforcing closed loop to gradually eliminate uncertainty.

Benefits of technology

It improves the accuracy and efficiency of data collection, reduces error propagation, meets privacy compliance requirements, significantly improves the recognition accuracy and adaptation efficiency in multi-center heterogeneous scenarios, avoids the limitations of a single end-to-end model, and realizes efficient and accurate data transformation into structured data.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN123120168A_ABST
    Figure CN123120168A_ABST
Patent Text Reader

Abstract

This invention discloses a clinical information collection and management system, relating to the field of medical data collection and management technology. Through its front-end collection and interaction module, de-identification processing module, layout awareness and multimodal extraction module, semantic error correction and encoding normalization module, data knowledge base module, and confidence and human-computer collaboration module, this invention efficiently, accurately, and compliantly transforms heterogeneous, unstructured, privacy-information-containing, and inconsistent-quality source data (such as paper, images, and audio) into structured data usable on EDC platforms, significantly improving the precision and efficiency of medical data quality control.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] One or more embodiments of the present invention relate to the field of medical data acquisition and management technology, specifically to a clinical information acquisition and management system. Background Technology

[0002] The quality of clinical data directly affects the reliability of the conclusions of the entire clinical medical research and the safety of the trial participants. Currently, there are bottlenecks in efficiency and quality in the process of integrating clinical data from raw files into the clinical data management system. The main problems are as follows: (1) Data entry is highly dependent on manual labor. Traditional data entry platforms rely on manual transcription. The data involves paper test reports, medical records, medical orders, images and electrocardiogram reports, etc., which require manual input of each field into the system. Some key fields also require two people to enter independently for mutual verification. (2) The source data presents strong multimodal and strong unstructured characteristics. Clinical source documents include paper forms, scanned copies, photos taken by mobile phones or tablets, images and waveforms exported by devices, voices spoken by researchers, etc. The formats and carriers are different. The same information may appear in different modalities and different expressions, lacking a unified structured entry point. (3) Mobile acquisition has introduced serious image quality problems. Taking source documents with mobile phones and tablets has become the norm, but this brings problems such as perspective distortion, tilt, curling, reflection, shadow, local blurring, missing pages or repeated shooting. After the source document enters the clinical management system, it will be magnified into field misalignment and value error in subsequent recognition, which seriously affects the development of clinical medical research.

[0003] It is worth noting that while some solutions exist in the industry, they are still insufficient to solve the aforementioned problems. Solutions relying solely on a single end-to-end "black box" model, while offering some improvement in image understanding, suffer a sharp drop in robustness to distortion, blurring, and complex layout overlays, and lack interpretability and controllability, making it difficult to locate and correct errors. Solutions relying solely on fixed template matching struggle to adapt to cross-center layout differences and interference from handwriting and annotations. Meanwhile, simple general OCR plus dictionary-based error correction modes often suffer from overcorrection or undercorrection due to a lack of field-level semantic constraints. The common limitation of these solutions is that they entrust complex problems to a single step, lacking a systematic adaptation to the characteristics of clinical data.

[0004] Existing technologies rely heavily on manual labor or delegate the entire identification and error correction task to a single "black box" model, resulting in low data accuracy and difficulty in meeting the increasingly precise needs of clinical medical research. Summary of the Invention

[0005] To address the aforementioned issues, this invention provides a clinical information collection and management system. The aim is to efficiently, accurately, and compliantly transform heterogeneous, unstructured, privacy-involved, and inconsistent quality modal data sources such as paper, images, and audio into structured data usable on an EDC platform, thereby improving the precision and efficiency of medical data quality control.

[0006] This invention specifically discloses a clinical information collection and management system, including: a front-end collection and interaction module, a desensitization processing module, a layout awareness and multimodal extraction module, a semantic error correction and encoding normalization module, a data knowledge base module, and a confidence and human-computer collaboration module;

[0007] The front-end data acquisition and interaction module is used to guide the acquisition of multimodal source data for clinical data and to verify the integrity of the acquired multimodal source data. The desensitization processing module is used to receive multimodal source data acquired by the front-end acquisition and interaction module, desensitize the identity information of the multimodal source data, and output desensitized multimodal data. The layout awareness and multimodal extraction module is used to reconstruct, slice and recognize the desensitized multimodal data in sequence, using the matched template data as a prior constraint, and output structured data. The semantic error correction and encoding normalization module is used to optimize structured data and output normalized data while satisfying the constraints of the field's controlled domain, combined with image evidence evaluation, field approximation distance and field domain rationality. The confidence and human-computer collaboration module is used to compare the confidence values ​​and corresponding confidence thresholds output by the layout awareness and multimodal extraction module and the semantic error correction and encoding normalization module, and to perform traffic splitting based on the comparison results. The data knowledge base module is used to summarize and extract graphic and textual structure data based on historical source data from different data sources to form template data. These template data are then provided to the layout awareness and multimodal extraction module, the semantic error correction and encoding normalization module, and the confidence and human-machine collaboration module for reuse. Simultaneously, the module receives feedback data after manual review and continuously updates the template data.

[0008] Optionally, when the front-end acquisition and interaction module acquires multimodal source data of clinical data, it uses the built-in image quality assessment module to calculate image tilt, edge integrity and sharpness parameters in real time to assess image quality, and outputs correction guidance information for multimodal source data that fails the assessment before submission for re-acquisition.

[0009] Optionally, the front-end acquisition and interaction module performs data integrity verification on the acquired multimodal source data, including: For multi-page documents, page order splicing and missing page detection are performed based on header features and page number anchor points, and supplementary information is output by the front-end acquisition and interaction module based on the missing pages; The collected multimodal source data is compared in real time according to the preset list of required fields, and supplementary collection information is output by the front-end acquisition and interaction module based on the missing fields. Based on the target area and fields to be collected, the front-end collection interaction module outputs target area guidance information and guides re-collection.

[0010] Optionally, when the desensitization processing module desensitizes the identity information of the multimodal source data, it includes: The identity information is located by means of a named entity recognition channel and a layout area positioning channel working together; the identity information includes identity information belonging to a direct identifier and identity information belonging to a quasi-identifier; Generate consistent alternative identifiers for the same trial participant across different modal data sources, and establish a vertical correlation mapping between the alternative identifiers and the trial participant; Perform authorized and reversible encryption masking processing on the identity information directly attribution; The identity information of the attribution quasi-identifier is subjected to binning and generalization processing, and k-anonymity protection is implemented.

[0011] Optionally, after the desensitization processing module completes the desensitization processing, it performs a second scan on the desensitized multimodal data and calculates the residual risk value of the residual identity information: if the residual risk value exceeds a preset threshold, the desensitization processing flow is rolled back and manual review is triggered; if the residual risk value does not exceed the preset threshold, the desensitized multimodal data is released.

[0012] Optionally, when the layout awareness and multimodal extraction module reconstructs, slices, and identifies the desensitized multimodal data, it includes: Based on feature points or positioning markers, perspective transformation is performed on the desensitized multimodal data to map the distorted image onto a standard plane, and the physical mesh of the desensitized multimodal data is reconstructed through morphological line detection to obtain the corrected data; The corrected data is registered with the template data to establish a unified coordinate system. Based on the physical boundaries of the template data matched in the data knowledge base module, the corrected data is graphically segmented to generate multiple mutually isolated recognition sub-regions. Each of the identified sub-regions is subjected to field identification, and structured data containing field value candidates, coordinate information and field identification confidence is output.

[0013] Optionally, the semantic error correction and encoding normalization module optimizes the structured data, including: For each target field, match the corresponding field-controlled domain to constrain the candidate value space of the target field to the range of the field-controlled domain; Construct a joint optimization objective function, which includes an image evidence probability term, a character approximation distance penalty term, and a register reasonableness prior term: ; in To select the optimal field value; For visual evidence; This is a preliminary identification result; For field controlled domain Candidate values ​​within; For the target field; For recognition models based on image evidence For candidate values The given probability is the probability term for the image evidence; The approximate distance between the candidate value and the preliminary recognition result at the character shape or pronunciation level is the character approximation distance penalty term; This refers to the register reasonableness of the candidate value in the context of the target field, i.e., the register reasonableness prior. and Non-negative weights; Based on the joint optimization objective function, the optimal field value is obtained within the controlled domain of the field, and the optimal field value and the corresponding correction confidence are output. Based on the optimal field values, unit conversion and normalization processing, as well as standard encoding mapping processing, are performed to obtain standardized data.

[0014] Optionally, when the confidence level and human-machine collaboration module performs traffic splitting based on the comparison results, it includes: The confidence scores output by the layout awareness and multimodal extraction module and the semantic error correction and encoding normalization module are obtained respectively. The comprehensive confidence score of the output results of the corresponding modules is calculated based on the confidence scores output by the layout awareness and multimodal extraction module and the semantic error correction and encoding normalization module. The process for calculating the overall confidence level is as follows: ; in For overall confidence level; To determine the probability of the candidate value output by the identification model or error correction model; Consistency score for the same field value across sources and modalities; The register appropriateness of candidate values ​​within the context of the target field; These are the weight values, and i = 1, 2, 3; The overall confidence level is compared with the corresponding preset triage threshold: if the overall confidence level is greater than or equal to the preset triage threshold, it is determined to be a high confidence result, and the corresponding field data is directly written into the electronic data acquisition system; if the overall confidence level is less than the preset triage threshold, it is determined to be a low confidence result, and the corresponding field data is pushed to the manual review interface or the front-end acquisition interaction module is triggered to perform local re-acquisition of the target area corresponding to the field data.

[0015] Optionally, when the same field data is obtained from source data of different modalities, the field data obtained from the source data of different modalities are associated with the same alternative identifier. The layout awareness and multimodal extraction module obtains the field value candidates and field recognition confidence of the field data from the source data of different modalities respectively. The semantic error correction and encoding normalization module calculates the corrected field value by weighting the field value candidates determined by the source data of different modalities based on the field recognition confidence. When the confidence of the corrected field value is lower than a preset threshold, the manual review interface is triggered.

[0016] Optionally, the confidence and human-machine collaboration module calculates a structural deviation metric between the format representation of the newly collected source data and the template data within the data knowledge base module for each new source data. ; in For the source data to be tested Structural deviation measure; For the source data to be tested Layout representation; For the main body of data source Template data accumulated in the data knowledge base; The normalized layout alignment, with a value range of [0,1]. The structural deviation metric is compared with a preset threshold: if the structural deviation metric exceeds the preset threshold, the current source data is determined to be abnormal or questionable, and the process is transferred to the manual review interface; if the structural deviation metric does not exceed the preset threshold, the current source data is determined to be normal, and the layout awareness and multimodal extraction module uses the current template representation data as a priori constraint.

[0017] Compared with the prior art, the present invention has the following beneficial effects: (1) The system of this invention moves image quality verification, missing page / item detection, and directional region acquisition to the front end of the acquisition process, so that problems can be discovered and corrected during the acquisition process, avoiding the high rework costs caused by errors being propagated to subsequent stages. At the same time, it can perform encryption masking on direct identity identifiers, generate consistent alternative identifiers for the same test participants to retain vertical correlation, perform binning generalization on aligned identifiers to meet k-anonymity constraints, and cover non-textual identity clues and residual risks for secondary scanning. While meeting privacy compliance requirements, it retains the ability to connect data across documents and modalities. The system completes geometric correction and mesh reconstruction, template alignment and spatial isolation, controlled domain identification and error correction in sequence according to the physical layer, structural layer, and semantic layer. Each stage works on the input that has had some uncertainties eliminated in the previous stage. The overall error converges step by step rather than being coupled and amplified, effectively solving the problems of field value mismatch and crosstalk in low-quality, distorted, and complex format tables. Furthermore, this system constrains the error correction space to a specific controlled domain, jointly optimizing the solution by combining image evidence, character approximation distance, and domain reasonableness. This approach corrects systematic errors while suppressing over-correction, avoiding the systematic flaw of general error correction models that correct specific domain terms as generic terms. Simultaneously, the system assigns calibrated confidence levels to the outputs of each stage as a unified control signal, automatically triaging high-confidence results for direct adoption and low-confidence results for manual review or partial re-collection based on thresholds. This achieves maximum accuracy gain with minimal human intervention. The feedback of review results continuously calibrates the confidence model and knowledge base, forming a self-reinforcing closed loop. Utilizing the structural consistency characteristics of the same data source entity, a knowledge base containing format templates, field value distributions, and encoding mapping relationships is formed. This narrows the recognition problem down to a constrained recognition problem under known templates, providing structural prior support for extraction, error correction, and anomaly detection, significantly improving recognition accuracy and adaptation efficiency in multi-center heterogeneous scenarios.

[0018] (2) The system of the present invention decomposes the collection, desensitization, extraction, semantic normalization and knowledge base into mutually decoupled functional modules. Each module interacts through standardized data contracts, and the specific models or algorithms used internally can be replaced and upgraded independently, avoiding the defects of a single end-to-end black box model that is difficult to locate problems and difficult to optimize locally, so that the system can continuously iterate with the evolution of the underlying model technology. Attached Figure Description

[0019] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings of the embodiments will be briefly introduced below. Obviously, the drawings described below only relate to some embodiments of the present invention, and are not intended to limit the present invention.

[0020] Figure 1 A schematic block diagram of the composition of a clinical data acquisition and management system provided in an embodiment of the present invention is shown; Figure 2 This diagram illustrates the overall architecture of a clinical data acquisition and management system provided in an embodiment of the present invention. Figure 3 A schematic flowchart of the identity / content desensitization mechanism in an embodiment of the present invention is shown.

[0021] Figure 4 A flowchart illustrating the three-stage extraction pipeline mechanism in an embodiment of the present invention is shown.

[0022] Figure 5 The diagram illustrates the flow of semantic error correction and encoding normalization module in the field-controlled domain decoding error correction in an embodiment of the present invention.

[0023] Figure 6 The diagram illustrates the process of continuous data accumulation and updating of the data knowledge base module in an embodiment of the present invention.

[0024] Figure 7 The diagram illustrates a flowchart of the confidence level and human-machine collaboration module performing threshold-based traffic splitting in an embodiment of the present invention.

[0025] Figure 8 A schematic flowchart illustrating the collaborative operation of various modules in the clinical data acquisition and management system of this invention is shown in an embodiment of the present invention. Detailed Implementation

[0026] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the described embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0027] Unless otherwise defined, the technical or scientific terms used in this invention shall have the ordinary meaning understood by one of ordinary skill in the art to which this invention pertains. It is understood that before using the technical solutions disclosed in the various embodiments of this invention, relevant users should be informed of the type, scope of use, and usage scenarios of the information involved in this invention through appropriate means in accordance with relevant laws and regulations, and authorization should be obtained from the relevant users. Relevant users may include any type of rights holder, such as individuals, enterprises, or groups.

[0028] To facilitate understanding, the main terms in this article are briefly explained below: EDC, or Electronic Data Capture System, is used to collect and manage clinical trial data.

[0029] Source data refers to documents that record the original observation results in clinical trials, such as test reports, medical records, imaging and electrocardiogram reports, etc.

[0030] Desensitization refers to the process of removing or masking personally identifiable information from data.

[0031] Pseudonymic consistency refers to mapping the same participant to the same anonymous identifier in different documents in order to preserve longitudinal associations under the premise of anonymity.

[0032] K-anonymity is a privacy-preserving model that requires any combination of quasi-identifiers to correspond to at least k individuals.

[0033] A controlled domain refers to a finite set of allowed values ​​for a field that is constrained by a standard or dictionary.

[0034] Confidence level refers to the quantitative estimate of the correctness of a certain output by the system. After calibration, it can be compared with a preset threshold and used for automatic diversion decision-making.

[0035] A research center's data knowledge base refers to a knowledge base that a research center continuously builds during the experimental process, accumulating the graphic and textual structure data (typical format, field composition and value distribution, etc.) of the center's historical report forms, which serves as a priori source for extraction, error correction and quality control.

[0036] Structural deviation from baseline refers to a measure of the degree of deviation of the report under test from a typical template of a certain center.

[0037] Perspective correction refers to the geometric transformation that maps an image with perspective distortion back to a standard plane using a reference point.

[0038] Morphological line detection refers to the method of extracting horizontal and vertical lines from a table using image morphological operators to reconstruct the physical mesh.

[0039] like Figure 1As shown, the clinical data acquisition and management system involved in this embodiment of the invention includes six main modules: a front-end acquisition interaction module, a de-identification processing module, a layout awareness and multimodal extraction module, a semantic error correction and encoding normalization module, a data knowledge base module, and a confidence level and human-machine collaboration module. The front-end acquisition interaction module belongs to the front-end and is a data acquisition application deployed on mobile or web-based terminals. It allows users to capture, upload, or input source data at the center and carries out quality guidance and integrity verification at the acquisition end. The de-identification processing module, layout awareness and multimodal extraction module, semantic error correction and encoding normalization module, data knowledge base module, and confidence level and human-machine collaboration module all belong to the back-end, organized in a service-oriented manner, and interface with the EDC (Electronic Data Capture) platform and hospital information system through standard interfaces. The service modules communicate with each other through standardized data interfaces, and the specific models or algorithms within any service can be independently upgraded or replaced. The system can be deployed as needed on-premises at the center, in a private cloud, or under a federated architecture to meet different data-within-domain requirements. The clinical data acquisition and management system described in this invention can eliminate the dependence on specific underlying models, significantly improving the efficiency, accuracy, and compliance of data entered into the EDC platform, while also possessing good interpretability and evolvability. This clinical data acquisition and management system can be deployed in multi-center clinical trial scenarios to efficiently, accurately, and compliantly transform heterogeneous, unstructured, privacy-information-containing, and inconsistent-quality source data, such as paper, images, and audio, into structured data usable on the EDC platform.

[0040] Combination Figure 2 As shown, the front-end data acquisition and interaction module, namely module A, is used to guide the collection of multimodal source data for clinical data from mobile or web-based terminals, and to perform quality guidance and data integrity verification on the collected multimodal source data, thereby moving data quality control forward to the source of data acquisition. Regarding the data acquisition process, once data is incorrectly acquired, the later the error is discovered, the higher the cost of correction. Therefore, in this embodiment of the invention, the front-end data acquisition and interaction module moves the quality control of data acquisition from post-event verification to control within the acquisition stage. Specifically, it performs framing and clarity guidance, page and item missing detection, and on-demand targeted area acquisition at the moment of shooting or submission, so that data quality problems are exposed and corrected at the source of acquisition. Its underlying principle is to place quality constraints as early as possible to the moment information is generated, thereby amortizing rework costs throughout the entire data chain.

[0041] Regarding framing and sharpness guidance: When collecting multimodal source data for clinical data, the front-end acquisition and interaction module can use its built-in image quality assessment module to calculate image tilt, edge integrity, and sharpness parameters in real time to assess image quality. For multimodal source data that fails the assessment, correction guidance information is output before submission for re-acquisition. For example, when collecting multimodal source data for clinical data in the form of images, the front-end acquisition and interaction module will calculate image quality indicators in real time, such as using image gradient energy or Laplacian variance to measure sharpness, edge detection to estimate tilt angle, image coverage and corner detection to determine missing corners, and highlight area proportion to detect reflections. It assesses image tilt, missing corners, and sharpness indicators. If any indicator falls below a preset acquisition quality threshold, a correction prompt is given before submission, such as a targeted prompt on the interface: "Image tilted, please align," or "Reflection present, please adjust angle," guiding the user to adjust on the spot. Whether the image information acquired after guidance meets the requirements is judged based on whether the same set of quality indicators passes the re-examination after adjustment or re-shooting: if it meets the standard, it is allowed to be submitted; if it does not meet the standard, guidance continues. Furthermore, if the captured image meets the standards and is released by the front-end acquisition interaction module, but still produces a low-confidence output due to image quality after being recognized by the layout perception and multimodal extraction module, then the confidence level and human-machine collaboration mechanism will send a directional re-acquisition instruction to the front-end acquisition interaction module, and the front-end acquisition interaction module will guide the user to re-capture the corresponding area.

[0042] For missing page and missing item detection: The front-end acquisition and interaction module performs page sequence splicing and missing page detection on multi-page documents based on header features and page number anchors, and outputs supplementary acquisition information based on the missing pages. Simultaneously, it can compare the already acquired content in the document with the pre-set list of required fields for this visit in real time to detect missing items. Based on the specific fields corresponding to the missing items, it determines the target areas and fields that need to be supplemented, and the front-end acquisition and interaction module outputs target area guidance information to guide re-acquisition. Users can select target areas and specify fields on the image, thus recognizing only specific areas. Furthermore, if a missing item is confirmed and the document is still submitted, the missing item can be marked by the confidence and human-machine collaboration module and routed to manual processing.

[0043] For on-demand targeted area collection: When the confidence level and human-machine collaboration module or manual review determines that the data of a specific area is unavailable, such as a value in a reflective area that has low confidence and cannot be disambiguated, the confidence level and human-machine collaboration mechanism sends a re-collection instruction carrying the area coordinates and target field to the front-end collection interaction module. At this time, the front-end collection interaction module will guide the user to select and retake only the area and identify only within that area, avoiding rework of the entire page.

[0044] Combination Figure 2As shown, the anonymization module, namely module B, is used to receive multimodal source data acquired by the front-end data acquisition and interaction module, and to anonymize the identity information of the multimodal source data, outputting anonymized multimodal data. For clinical data that needs to be entered into the EDC platform, the source files generally contain personal identity information such as name, hospital number, ID number, contact information, ID photo, QR code, identifiable signature, and even voiceprint. Based on clinical trial compliance requirements, sensitive identity information must be removed, while data usability requires maintaining vertical correlation under anonymity.

[0045] Combination Figure 2 and Figure 3 As shown, the desensitization module in this embodiment of the invention decouples identity and content within multimodal source data to balance both aspects, and performs desensitization processing on different identity identifiers in the multimodal source data respectively: Firstly, the desensitization processing module can locate the identity information of multimodal source data through the collaboration of the named entity recognition channel and the layout area positioning channel.

[0046] By generating a consistent alternative identifier for the same trial participant across different modal data sources and establishing a vertical association mapping between the alternative identifier and the trial participant, both identity anonymization and vertical associativity are achieved to link the test results, disease course, and imaging of the same trial participant.

[0047] Irreversible encryption and masking are applied to directly identified identity information. For example, hospital numbers, ID card numbers, and ID photos can be directly replaced with alternative identifiers, making the original identity information unobtainable after data anonymization.

[0048] Binning and generalization are performed on the identity information of the quasi-identifier, and k-anonymity protection is applied. For example, information such as precise date of birth, rare diagnosis, and location of medical treatment can be binned and generalized so that each equivalence class contains at least a specified number of k individuals to mitigate the risk of re-identification.

[0049] It should be noted that the desensitization processing module in this embodiment of the invention performs desensitization processing not only on text, but also on non-textual identity clues such as ID photos, QR codes, identifiable signatures and voiceprints.

[0050] Furthermore, in this embodiment of the invention, after the desensitization processing module completes the desensitization processing, it can perform a second scan on the desensitized multimodal data and calculate the residual risk value of the residual identity information: if the residual risk value exceeds the preset threshold, the desensitization processing flow is rolled back and manual review is triggered; if the residual risk value does not exceed the preset threshold, the desensitized multimodal data is released.

[0051] Combination Figure 2As shown, the layout perception and multimodal extraction module, namely module C, uses the matched template data as a priori constraint to sequentially reconstruct, slice, and recognize the desensitized multimodal data, outputting structured data. Common clinical data source files include paper tables, scanned copies, photos taken with mobile phones or tablets, images and waveforms exported from devices, and researcher dictation, with varying layouts and media. For example, table-type source files may have multi-column layouts, merged cells, linkage between reference intervals and anomaly markers, and printing misalignments or exceeding borders, easily leading to "field-value" mismatches and row misalignments, resulting in misattributed data. Furthermore, the same information may appear in different modalities and with different expressions in clinical data source files, and existing methods lack a unified structured entry point. Traditional methods read structured data directly from low-quality images in one step, which is essentially a highly coupled and often pathological problem: physical distortion, complex layouts, and illegible handwriting are intertwined, leading to significant errors in recognition results.

[0052] In this embodiment of the invention, the layout awareness and multimodal extraction module is responsible for converting anonymized multimodal data into "field-value" data pairs. The core of this module's operation employs a three-stage extraction pipeline, comprising image preprocessing and physical alignment, spatial localization and data isolation, and constraint recognition and semantic correction. This module decomposes the aforementioned structured data extraction problem into sequentially conditional sub-problems along the "physical layer—structural layer—semantic layer," gradually eliminating uncertainty. All three stages are executed sequentially within the layout awareness and multimodal extraction module. The second stage, spatial localization and data isolation, utilizes template priors provided by the data knowledge base module, while the third stage, constraint recognition and semantic correction, is seamlessly integrated with the field-controlled domain constraints of the semantic correction and encoding normalization module.

[0053] Combination Figure 4 As shown, specifically, the extraction process of this layout awareness and multimodal extraction module is as follows: The first stage of image preprocessing and physical alignment is performed internally by the layout awareness and multimodal extraction module. It estimates perspective relationships using feature points or positioning markers, performs perspective transformation on the desensitized multimodal data (e.g., photographs) based on these markers, maps the distorted image to a standard plane, and reconstructs the physical grid of the desensitized multimodal data using morphological line detection to obtain corrected data. For example, morphological line detection extracts the horizontal and vertical lines of a table, restoring the true row and column boundary structure, significantly reducing the risk of misaligned rows from a physical perspective. It should be noted that this risk of misaligned rows refers to the potential for incorrect pairing of "field-value" across rows in the subsequent image-to-text (recognition) stage due to photographic tilt, perspective distortion, and misalignment during printing, leading to misattribution of data. This first stage, by performing geometric correction and physical grid reconstruction before recognition, ensures that each data item is locked to its true row and column position before entering recognition, thus eliminating this risk beforehand, rather than adjusting for misalignment in the recognition or distorted image processing results afterward.

[0054] The second stage of spatial localization and data isolation is performed internally by the layout awareness and multimodal extraction module, while simultaneously invoking the template prior provided by the data knowledge base module. This module registers and aligns the corrected data with the template data provided by the data knowledge base module, establishes a unified coordinate system, and performs graphic segmentation on the corrected data based on the known physical boundaries of the template data, generating multiple mutually isolated recognition sub-regions. Recognition is performed cell by cell, thereby significantly reducing contextual crosstalk caused by cross-row, cross-column, and merged cells through physical isolation.

[0055] The third stage, constraint identification and semantic correction, is performed internally by the layout awareness and multimodal extraction module. The candidate distribution output is then disambiguated by the semantic correction and encoding normalization module within the controlled domain of the field. This module performs field identification on the aforementioned sub-regions, outputting structured data containing candidate field values, coordinate information, and field identification confidence. For example, for ambiguous characters, no hard judgment is made; instead, the candidate distribution is output and passed down to the subsequent semantic correction and encoding normalization module for constraint disambiguation. Simultaneously, reference intervals and anomaly markers are extracted in tandem, and the same field from different modalities and sources is weighted and fused according to source reliability, with conflicts marked.

[0056] The three-stage extraction pipeline described above transfers the structural complexity of the source data to a known template; replaces the model's understanding of complex layouts with physical isolation; and uses strong priors and business rules as a final safety net, rather than relying entirely on a single model for accuracy. The accuracy of the entire pipeline depends on the quality of the template, thus it is strongly bound to the data knowledge base module. Traditional methods employ an end-to-end black-box structure that simultaneously undertakes geometric, structural, and semantic inferences in a single step, leading to mutually coupled and amplified errors. In this embodiment, the phased processing of the layout awareness and multimodal extraction module ensures that each stage builds upon the accurate results of the previous stage, which has already resolved some uncertainties: physical distortions are first eliminated by geometric correction, layout complexity is then isolated by templates and segmented regions, and finally, only semantic ambiguities are resolved by the field-controlled domain. Based on this process, the overall extraction error of the source data is no longer a simple summation of the difficulties of each stage, but rather closer to gradual convergence. This makes the phased processing of the layout awareness and multimodal extraction module in this embodiment significantly superior to the single black-box approach.

[0057] Combination Figure 2 As shown, the semantic error correction and encoding normalization module, namely module D, is used to optimize structured data and output normalized data under the constraints of controlled field domains, combined with image evidence evaluation, field approximation distance, and field domain reasonableness. In this embodiment of the invention, this module is responsible for correcting errors, disambiguating, normalizing units, and mapping standard codes to the extraction results under the constraints of field semantics and medical ontology, outputting comparable and analyzable normalized data. Traditional methods generally use general model recognition and error correction, which are performed within an open vocabulary, thus correcting certain domain-specific fields to non-domain fields that are similar in form or sound.

[0058] In this embodiment of the invention, for each target field, its set of terms in a specific domain is matched as a controlled domain for the field. The identification and error correction are then remodeled as a joint optimization problem within that controlled domain. Figure 5 As shown, the specific optimization process is as follows: First, match the corresponding field control domain for each target field, and constrain the candidate value space of the target field to the range of the field control domain.

[0059] Subsequently, a joint optimization objective function is constructed, which includes an image evidence probability term, a character approximation distance penalty term, and a register reasonableness prior term: ; in To select the optimal field value; For visual evidence; This is a preliminary identification result; For field controlled domain Candidate values ​​within; For the target field; For recognition models based on image evidence For candidate values The given probability, i.e., the probability term for image evidence, belongs to the evidence fidelity term. This term requires that each candidate value be re-evaluated within the context of the image evidence to prevent error correction from arbitrarily altering the original image. It should be noted that the evaluation object of this evidence fidelity term is the candidate value. It should be the value itself, rather than a fixed preliminary identification result; otherwise, this item would have the same value for all candidate values ​​and would lose its distinguishing effect. The approximate distance between the candidate value and the initial recognition result at the character shape or pronunciation level is called the character approximate distance penalty term. It is a conservative term. Generally, the farther the candidate value deviates from the initial recognition result, the greater the penalty, thereby suppressing overcorrection. This refers to the register reasonableness of the candidate value in the context of the target field, i.e., the register reasonableness prior, which belongs to the domain prior. and These are non-negative weights used to adjust the relative strength of image evidence, conservative constraints, and domain priors, and can be optimized on a small amount of labeled data.

[0060] Then, based on the joint optimization objective function, the optimal field value is solved within the controlled domain of the field, and the optimal field value and the corresponding correction confidence are output. It should be noted that the optimization process of this semantic error correction and encoding normalization module is fundamentally different from the traditional naive fuzzy matching method that automatically corrects based solely on similarity thresholds: traditional methods only look at literal similarity, which is prone to overcorrection; while this module combines image evidence, approximate distance, and medical rationality, which can correct systematic misidentifications such as "oral → oral lip balm" and suppress inappropriate corrections. Its essence is to constrain the candidate space of decoding from an "open vocabulary" to a "field-controlled domain", so that the prior is switched from a general language prior to a domain-controlled prior, thereby reducing the output entropy in an information theory sense.

[0061] Finally, based on the optimal field values, unit conversion and normalization processing, and standard encoding mapping processing are performed to obtain standardized data.

[0062] In this embodiment of the invention, the semantic error correction and encoding normalization module performs constrained decoding and error correction within the controlled medical terminology domain, selecting the optimal value by combining model probability, approximation distance, and medical rationality. It disambiguates similar-looking or similar-sounding medical terms by considering context and field type; it standardizes abbreviations, aliases, and trade names to generic names; it marks unwritten fields (such as route of administration and dosage form) as "inferred values" based on contextual inference, providing inference justification for verification; it identifies and converts testing units to standard units, storing the conversion process; and it automatically maps diagnoses, medications, and testing items to standard medical codes and provides confidence levels, with low-confidence items marked as requiring manual confirmation. Furthermore, this module can perform deterministic format and value range validation on columns based on header attributes, serving as a fallback for the layout awareness and multimodal extraction module.

[0063] Combination Figure 2 As shown, the confidence level and human-machine collaboration module, serving as control signals throughout the entire process, can be used to compare the confidence values ​​output by the layout perception and multimodal extraction modules, as well as the semantic error correction and encoding normalization modules, with their corresponding confidence thresholds, and then perform triage processing based on the comparison results. Any automatic recognition algorithm has errors and cannot completely guarantee accurate recognition, but the key is whether the system can identify where it might make a mistake, and thus accurately locate the error when it occurs. This invention calculates a calibrated confidence level for each output in the extraction and error correction stages, and uses the confidence level as a unified control signal to automatically triage based on thresholds: high-confidence results are directly adopted; low-confidence results are routed to targeted manual review, or trigger the front end to re-acquire the corresponding area.

[0064] like Figure 7 As shown, the specific process of the confidence level and the human-machine collaboration module performing traffic splitting is as follows: The confidence scores output by the layout awareness and multimodal extraction module and the semantic error correction and encoding normalization module are obtained respectively. The comprehensive confidence score of the output results of the corresponding modules is calculated based on the confidence scores output by the layout awareness and multimodal extraction module and the semantic error correction and encoding normalization module. The calculation process for the overall confidence level is as follows: ; in For overall confidence level; To identify the probability of the model or error correction model outputting candidate values, reflecting the model's own level of confidence; For the consistency score of the same field value across sources and modalities, when the value of the same field can be obtained from multiple independent sources, such as the recognition results of a single image, the data stream directly collected by the device, and the extracted value from free nursing text, the score tends to be high if the sources match each other, and low if there is a discrepancy, thus using information redundancy to detect random errors from a single source; when there is only a single source, the score can degenerate to a neutral value or have its weight reduced. The register reasonableness of candidate values ​​in the context of the target field is jointly evaluated by controlled word list hit, reference interval and dimensional verification, domain ontology constraints, etc., and is used to suppress the confidence of values ​​that are "linguistically fluent but unreasonable in a specific domain". These are the weight values, and they satisfy... ,i=1,2,3; make It falls within the [0,1] interval and is comparable across different fields and batches.

[0065] The overall confidence level is compared with the corresponding preset triage threshold: if the overall confidence level is greater than or equal to the preset triage threshold, it is determined to be a high confidence result, and the corresponding field data is directly written into the electronic data acquisition system; if the overall confidence level is less than the triage threshold, it is determined to be a low confidence result, and the corresponding field data is pushed to the manual review interface or the front-end acquisition interaction module is triggered to perform local re-acquisition of the target area corresponding to the field data.

[0066] In this embodiment of the invention, the confidence level model of the confidence level and human-machine collaboration module can be fitted and calibrated based on the correctness and error results of historical manual review, so that the comprehensive confidence level of different modules approximates the "true probability that the output is correct", thus giving the diversion threshold a clear probabilistic meaning. This forms a closed loop of "collection - extraction - confidence - review or recollection". This module "locates" errors based on the comprehensive confidence level, accurately allocating scarce human resources to the few samples with the highest marginal benefit, thereby obtaining the greatest accuracy gain with minimal human intervention.

[0067] Combination Figure 2As shown, this data knowledge base module, or module S, serves as a horizontal knowledge foundation. It is used to extract and summarize textual and graphical structure data from historical source data of different data sources to form template data. This template data is then provided to the layout awareness and multimodal extraction module, the semantic error correction and encoding normalization module, and the confidence and human-machine collaboration module for reuse. Simultaneously, it receives feedback data after manual review to continuously update the template data. For example, a research center continuously builds a data knowledge base during experiments, accumulating textual and graphical structure data (typical layouts, field composition, and value distribution, etc.) from its historical report forms. The knowledge source is the structured data output by the layout awareness and multimodal extraction module and the standardized data output by the semantic error correction and encoding normalization module (which, after manual review and confirmation, is fed back into the data knowledge base module) for reuse by other modules.

[0068] In clinical trials, similar reports produced by the same research center are typically constrained by the same template, printing or information system, and writing standards. Consequently, they exhibit a high degree of homogeneity in layout, field composition, and wording, a characteristic known as "low entropy." This invention identifies and consolidates this structural consistency into a reusable data knowledge base for the corresponding research center.

[0069] like Figure 6 As shown, in this embodiment of the invention, template data is formed by summarizing and extracting textual and graphical structure data based on historical source data from the same data source subject. This template data can function in four directions: Firstly, as a template prior for the layout awareness and multimodal extraction module, it enables new source data from the same data source subject to quickly apply the template data of that data source subject to accelerate positioning and matching. This is used by the layout awareness and multimodal extraction module in the second stage of template alignment and segmentation region positioning.

[0070] Secondly, as a center-level reference set, it provides localized value distribution and mapping relationships for the semantic error correction and coding normalization module; this center-level reference set is used by the semantic error correction and coding normalization module in the error correction and coding normalization process.

[0071] Third, as a structural deviation from the baseline, it is used to identify abnormal or questionable reports. It is used by the confidence and human-machine collaboration module in the quality assessment triage, which can converge the "general identification problem in the open world" into the "restricted identification problem under a known template", thereby systematically reducing uncertainty.

[0072] Specifically, when acquiring newly collected source data, such as a new test report obtained by a research center, the confidence level and human-machine collaboration module will calculate the format representation of the current test report and compare it with the corresponding template data of the research center in the data knowledge base module to measure the structural deviation between the two: ; in For the source data to be tested Structural deviation measure; For the source data to be tested Layout representation; For the main body of data source Template data accumulated in the data knowledge base; The normalized layout alignment has a value range of [0,1]. Generally, a perfect match is set to 1, and a complete mismatch is set to 0. It also follows the trend of "the greater the deviation, the larger the value". It can be directly compared with a preset threshold to determine whether the report is abnormal or questionable.

[0073] Compare the structural deviation metric with the preset threshold: If the structural deviation metric exceeds the preset threshold, the current source data is determined to be abnormal or questionable, and the process is transferred to the manual review interface; if the structural deviation metric does not exceed the preset threshold, the current source data is determined to be normal, and the layout awareness and multimodal extraction module uses the current template representation data as a priori constraint.

[0074] Fourth, as a self-supervised training or reference dataset, it provides low-cost samples for model adaptation of the layout awareness and multimodal extraction module and the semantic error correction and encoding normalization module, respectively.

[0075] It should be noted that for newly launched data source entities or those lacking historical data, templates from similar mature centers can be used as initial priors, and then the corresponding template data can be updated online as the data source entity accumulates data.

[0076] In the specific operation phase of the clinical data acquisition and management system of this invention embodiment, the corresponding data flow process is as follows: Source data of different modalities enters the system through the front-end acquisition and interaction module. First, the identity desensitization module completes the identity decoupling, then the layout awareness and multimodal extraction module completes the structuring, and then the semantic error correction and encoding normalization module completes the standardization, and finally writes it into the EDC platform. The data knowledge base module provides priors for the layout awareness and multimodal extraction module and the semantic error correction and encoding normalization module, while the confidence and human-machine collaboration module determines "whether manual intervention or re-collection is required" at each stage.

[0077] Specifically, the data knowledge base module provides template priors for the layout awareness and multimodal extraction module, which are template representations of a certain data source subject, including typical layout, cell boundary coordinates, and field composition. These are used by the layout awareness and multimodal extraction module to complete template alignment and segmentation region positioning in the second stage. The data knowledge base module provides a central reference set for the semantic error correction and encoding normalization module, which are the field value distribution, terminology and unit writing habits, and field-standard encoding mapping relationships of the historical reports of the corresponding data source subject. These are used by the semantic error correction and encoding normalization module to build a field-controlled vocabulary and support error correction and encoding normalization.

[0078] The unified principle for the confidence level and human-machine collaboration module's judgment methods and conditions for the other five modules is "to compare the calibration confidence level (or corresponding quality or risk measure) output by each stage with the preset threshold of that stage": For the front-end acquisition and interaction module, if the image quality indicators (sharpness, tilt, integrity) calculated in real time during framing are lower than the acquisition quality threshold, it is determined that the user needs to be guided to re-acquire the image at the source. For the desensitization module, if the residual identity risk score obtained from the second scan after desensitization exceeds the risk threshold, it is determined that it needs to be rolled back and transferred to manual review. For the layout awareness and multimodal extraction module, if the confidence level of a certain field is lower than the threshold or the candidate distribution does not converge, it is determined that the candidate distribution of the field needs to be sent down, transferred to manual review, or the front-end data collection interaction module is triggered to re-collect the corresponding area. For the semantic error correction and coding normalization module, if the confidence level is lower than the threshold, it is determined that the normalization or coding result needs to be manually confirmed. For the data knowledge base module, if the structural deviation metric exceeds the threshold, a certain source data item is determined to be abnormal, questionable, or transferred to manual review.

[0079] The modules described above are interconnected via standardized data interfaces: the front-end data acquisition and interaction module outputs raw multimodal data with quality labels; the de-identification module outputs de-identified data and a consistent pseudonym mapping; the layout awareness and multimodal extraction module outputs field-value data pairs with coordinates and candidate distributions, along with their confidence scores; and the semantic error correction and encoding normalization module outputs standardized, encoded fields, their confidence scores, and supporting annotations. Each module can input data into the next through the interface, enabling mutual calls. The system can also selectively enable or degrade certain modules, such as disabling template alignment in the absence of a template center and reverting to a general layout understanding. Therefore, the specific models or algorithms used within any layer can be independently replaced and upgraded without affecting the overall architecture.

[0080] In this embodiment of the invention, the clinical data acquisition and management system relies primarily on the collaboration between various modules, rather than on single-point optimization. For example... Figure 8As shown, the data knowledge base module provides template priors to the layout awareness and multimodal extraction module, a central reference set to the semantic error correction and encoding normalization module, and structural deviation baselines to the quality assessment module, serving as a multi-layered knowledge foundation. The three-stage pipeline of the layout awareness and multimodal extraction module provides "clean and isolated" candidates for the semantic error correction and encoding normalization module, enabling the fields of the semantic error correction and encoding normalization module to function effectively on low-noise inputs under controlled domain constraints. Conversely, the semantics of the fields in the semantic error correction and encoding normalization module provides a basis for candidate disambiguation of fuzzy characters in the layout awareness and multimodal extraction module, forming a closed loop that echoes each other. In this embodiment, the confidence level mechanism spans three layers: the front-end data acquisition and interaction module, the layout awareness and multimodal extraction module, and the semantic error correction and encoding normalization module. The layout awareness and multimodal extraction module and the semantic error correction and encoding normalization module each output their corresponding confidence levels. Based on this, the front-end data acquisition and interaction module triggers partial re-acquisition. The results of manual review are then fed back to calibrate the confidence levels and update the template data in the data knowledge base module. The de-identification module is located at the data entry point, ensuring compliance while preserving the "connection key" required for cross-document and cross-modal associations through consistent pseudonyms. This provides a prerequisite for multi-source fusion in the layout awareness and multimodal extraction module and the semantic error correction and encoding normalization module. Based on the mutual support of the above "prior—extraction—semantics—confidence—human-machine" mechanisms, the system remains robust even with distorted, ambiguous, heterogeneous, and privacy-involved real data.

[0081] Based on the above collaborative relationship, any layer can independently replace its underlying model or algorithm without destroying the whole, such as upgrading a recognition model to a stronger version or connecting a new preprocessing path for a new modality; the modules communicate with each other through standardized data contracts, enabling the system to continuously iterate as medical terminology standards, compliance requirements and model capabilities change, without having to reconstruct the whole.

[0082] This embodiment of the invention illustrates the collaborative operation of the clinical data collection and management system of this invention using a specific process. Imagine a research coordinator taking a photo of a paper biochemistry test report with their mobile phone during a visit. The photo is tilted and has perspective distortion; the lower half of the page is partially blurred due to glare; and due to misalignment during printing, one row is shifted upwards by half a grid.

[0083] The first step is for the front-end data acquisition and interaction module to prompt the viewfinder to be tilted and guide it to be aligned when framing the shot. Before submission, it completes the sharpness and integrity verification and prompts the viewfinder to retake the shot for areas with severe reflection.

[0084] The second step involves the desensitization module detecting and masking identity information such as name, hospital number, and ID number at the top of the page, replacing the hospital number with a substitute identifier, and mapping the participant to an existing consistent pseudonym identifier.

[0085] The third step involves the layout perception and multimodal extraction module performing perspective correction on the photo to "flatten" it, and reconstructing the physical grid using morphological line detection to correct misalignment caused by tilting and overlay misalignment from a physical perspective. The second step aligns the corrected image with the central template, divides it into graphic segmentation regions, and identifies each segment individually. For numbers blurred due to reflection, no hard judgment is made, and candidate distributions are output.

[0086] The fourth step involves semantic error correction and encoding normalization modules performing constrained decoding within the controlled domains of fields such as "inspection item," "value," "unit," and "route of administration." For example, it corrects the suspected "oral liposome" to "oral" under the constrained word list of "route of administration," converts the blood glucose unit from mg / dL to a unified standard unit and retains the conversion trajectory, and disambiguates fuzzy numbers by combining the reference interval with the context.

[0087] The fifth step involves the confidence level and human-machine collaboration module outputting the confidence level for each field. For individual low-confidence fields (such as a value in a reflective area), the data is routed to manual review or a re-collection of that area is triggered.

[0088] The sixth step involves manually reviewing and confirming the data, then writing the standardized, coded, and anonymized data into the EDC platform. Simultaneously, feedback from the review and re-collection is fed back to update the central template and confidence model.

[0089] The entire process embodies the overall principles of "layered decoupling, prior-driven, confidence-based, and human-machine collaboration".

[0090] Furthermore, in this embodiment of the invention, when the same field data is obtained from source data of different modalities, the field data obtained from the source data of different modalities can be associated with the same alternative identifier. The layout awareness and multimodal extraction module obtains the field value candidates and field recognition confidence of the corresponding field data from the source data of different modalities respectively; the semantic error correction and encoding normalization module calculates the corrected field value by weighted summation of the field value candidates determined by the source data of different modalities based on the field recognition confidence; when the confidence of the corrected field value is lower than a preset threshold, the manual review interface is triggered. For example, consider the information of "heart rate" from the same visit appearing simultaneously in three places: the monitoring or test report image, the connected device data stream, and the nursing free text, and the values ​​in the three places are slightly different. First, a consistent substitution identifier is used to associate the three data points with the same participant and the same visit. The layout awareness and multimodal extraction module obtains candidate values ​​and their confidence levels from the three modalities respectively. The semantic error correction and encoding normalization module weights and fuses the candidates according to the reliability of the source, outputs a single value, and marks any discrepancies. If the confidence level after fusion is still lower than the threshold, a manual review is triggered by the confidence level mechanism. The above process demonstrates how the consistent substitution identifier of the de-identification module, the multi-source candidates of the layout awareness and multimodal extraction module, the reliability fusion and confidence level of the semantic error correction and encoding normalization module, and the human-machine collaboration module work together to complete a cross-modal consistent merging.

[0091] For example, consider the information "heart rate" from the same visit appearing simultaneously in three places: monitoring or test report images, connected device data streams, and nursing free text, with slight discrepancies in the values. The system first associates the three data points with the same trial participant and the same visit using a consistent substitution identifier; the layout awareness and multimodal extraction module obtains candidate values ​​and their confidence levels from the three modalities respectively; the semantic error correction and encoding normalization module weights and fuses the candidates according to source reliability, outputs a single value, and marks any discrepancies; if the fused confidence level is still below the threshold, the confidence level and human-machine collaboration module triggers manual review. This embodiment demonstrates how the consistent substitution identifier of the desensitization processing module, the multi-source candidates of the layout awareness and multimodal extraction module, the reliability fusion and confidence level of the semantic error correction and encoding normalization module, and the human-machine collaboration module work together to complete a cross-modal consistency merging.

[0092] The two embodiments described above correspond to two typical tasks: "robust extraction from a single complex table" and "consistent merging of the same field across modalities." They share the same system architecture and underlying mechanism, demonstrating the universality of this invention under different data formats.

[0093] Furthermore, the clinical data acquisition and management system of this invention can adopt a template-free degradation strategy for data sources lacking historical templates: using general layout understanding instead of template alignment, and switching to template-driven approach after data accumulation. For non-tabular modal data sources, such as free text medical records, audio, ECG waveforms, and images, the "physical alignment" and "cell segmentation" in the three-stage pipeline can be replaced with preprocessing and region localization of the corresponding modality, while principles such as "field-controlled domain constraints" and "confidence threshold triage" remain unchanged. The underlying models in the system, such as OCR, named entity recognition, and encoding mapping, can be replaced or upgraded as needed without affecting the overall architecture. The system can also run under federated or local deployment to meet compliance requirements for data not leaving the domain.

[0094] It should also be noted that the operating principle of this system is not limited to testing a single source file, but can also be extended to various source materials such as medical records, medical orders, informed consent forms, imaging and electrocardiogram reports; for source materials that are mainly free text, the "cell segmentation region" can be weakened while the "field controlled domain constraint" and "context inference plus basis annotation" can be strengthened, while the other principles remain unchanged. Thus, the same system architecture can cover most scenarios of clinical trial data entry.

[0095] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the technical principles of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A clinical information collection and management system, characterized in that, include: The module includes a front-end data collection and interaction module, a desensitization processing module, a layout awareness and multimodal extraction module, a semantic error correction and encoding normalization module, a data knowledge base module, and a confidence and human-computer collaboration module. The front-end data acquisition and interaction module is used to guide the acquisition of multimodal source data for clinical data and to perform data integrity verification on the acquired multimodal source data. The desensitization processing module is used to receive multimodal source data acquired by the front-end acquisition and interaction module, and to desensitize the identity information of the multimodal source data, and output desensitized multimodal data. The layout awareness and multimodal extraction module is used to reconstruct, slice and identify the desensitized multimodal data in sequence, using the matched template data as a priori constraint, and output structured data. The semantic error correction and encoding normalization module is used to optimize the structured data and output normalized data by combining image evidence evaluation, field approximation distance and field domain rationality under the constraint of the field controlled domain. The confidence and human-computer collaboration module is used to compare the confidence values ​​output by the layout awareness and multimodal extraction module and the semantic error correction and encoding normalization module with the corresponding confidence thresholds, and to perform traffic splitting based on the comparison results. The data knowledge base module is used to summarize and extract graphic and textual structure data based on historical source data from different data sources to form template data, and provide it to the layout awareness and multimodal extraction module, the semantic error correction and encoding normalization module, and the confidence and human-machine collaboration module for reuse. At the same time, it receives feedback data after manual review and continuously updates the template data.

2. The clinical information collection and management system according to claim 1, characterized in that, When collecting multimodal source data of clinical data, the front-end acquisition and interaction module uses the built-in image quality assessment module to calculate image tilt, edge integrity and sharpness parameters in real time to assess image quality. For multimodal source data that fails the assessment, correction guidance information is output before submission for re-acquisition.

3. The clinical information collection and management system according to claim 2, characterized in that, The front-end data acquisition and interaction module performs data integrity verification on the acquired multimodal source data, including: For multi-page documents, page order splicing and missing page detection are performed based on header features and page number anchor points, and supplementary information is output by the front-end acquisition and interaction module based on the missing pages; The collected multimodal source data is compared in real time according to the preset list of required fields, and supplementary collection information is output by the front-end acquisition and interaction module based on the missing fields. Based on the target area and fields to be collected, the front-end collection interaction module outputs target area guidance information and guides re-collection.

4. The clinical information collection and management system according to claim 1, characterized in that, When the desensitization processing module desensitizes the identity information of the multimodal source data, it includes: The identity information is located by means of a named entity recognition channel and a layout area positioning channel working together; the identity information includes identity information belonging to a direct identifier and identity information belonging to a quasi-identifier; Generate consistent alternative identifiers for the same trial participant across different modal data sources, and establish a vertical correlation mapping between the alternative identifiers and the trial participant; Perform authorized and reversible encryption masking processing on the identity information directly attribution; The identity information of the attribution quasi-identifier is subjected to binning and generalization processing, and k-anonymity protection is implemented.

5. The clinical information collection and management system according to claim 4, characterized in that, After the desensitization processing module completes the desensitization processing, it performs a second scan on the desensitized multimodal data and calculates the residual risk value of the residual identity information. If the residual risk value exceeds the preset threshold, the desensitization processing flow is rolled back and manual review is triggered. If the residual risk value does not exceed the preset threshold, the desensitized multimodal data is allowed.

6. The clinical information collection and management system according to claim 1, characterized in that, When the layout awareness and multimodal extraction module reconstructs, slices, and identifies the desensitized multimodal data, it includes: Based on feature points or positioning markers, perspective transformation is performed on the desensitized multimodal data to map the distorted image onto a standard plane, and the physical mesh of the desensitized multimodal data is reconstructed through morphological line detection to obtain the corrected data; The corrected data is registered with the template data to establish a unified coordinate system. Based on the physical boundaries of the template data matched in the data knowledge base module, the corrected data is graphically segmented to generate multiple mutually isolated recognition sub-regions. Each of the identified sub-regions is subjected to field identification, and structured data containing field value candidates, coordinate information and field identification confidence is output.

7. The clinical information collection and management system according to claim 1, characterized in that, The semantic error correction and encoding normalization module optimizes the structured data, including: For each target field, match the corresponding field-controlled domain to constrain the candidate value space of the target field to the range of the field-controlled domain; Construct a joint optimization objective function, which includes an image evidence probability term, a character approximation distance penalty term, and a register reasonableness prior term: ; in To select the optimal field value; For visual evidence; This is a preliminary identification result; For field controlled domain Candidate values ​​within; For the target field; To identify models based on image evidence For candidate values The given probability is the probability term for the image evidence; The approximate distance between the candidate value and the preliminary recognition result at the character shape or pronunciation level is the character approximation distance penalty term; This refers to the register reasonableness of the candidate value within the context of the target field, i.e., the register reasonableness prior. and Non-negative weights; Based on the joint optimization objective function, the optimal field value is obtained within the controlled domain of the field, and the optimal field value and the corresponding correction confidence are output. Based on the optimal field values, unit conversion and normalization processing, as well as standard encoding mapping processing, are performed to obtain standardized data.

8. The clinical information collection and management system according to claim 1, characterized in that, When the confidence level and human-machine collaboration module performs traffic splitting based on the comparison results, it includes: The confidence scores output by the layout awareness and multimodal extraction module and the semantic error correction and encoding normalization module are obtained respectively. The comprehensive confidence score of the output results of the corresponding modules is calculated based on the confidence scores output by the layout awareness and multimodal extraction module and the semantic error correction and encoding normalization module. The process for calculating the overall confidence level is as follows: ; in For overall confidence level; To determine the probability of the candidate value output by the identification model or error correction model; Consistency score for the same field value across sources and modalities; The register appropriateness of candidate values ​​within the context of the target field; These are the weight values, and , i =1,2,3; The overall confidence level is compared with the corresponding preset triage threshold: if the overall confidence level is greater than or equal to the preset triage threshold, it is determined to be a high confidence result, and the corresponding field data is directly written into the electronic data acquisition system; if the overall confidence level is less than the preset triage threshold, it is determined to be a low confidence result, and the corresponding field data is pushed to the manual review interface or the front-end acquisition interaction module is triggered to perform local re-acquisition of the target area corresponding to the field data.

9. The clinical information collection and management system according to claim 6, characterized in that, When the same field data is obtained from source data of different modalities, the field data obtained from the source data of different modalities are associated with the same alternative identifier. The layout awareness and multimodal extraction module obtains the field value candidates and field recognition confidence of the field data from the source data of different modalities respectively. The semantic error correction and encoding normalization module calculates the corrected field value by weighting the field value candidates determined by the source data of different modalities based on the field recognition confidence. When the confidence of the corrected field value is lower than the preset threshold, the manual review interface is triggered.

10. The clinical information collection and management system according to claim 1, characterized in that, The confidence and human-machine collaboration module calculates the structural deviation metric between the format representation of the newly collected source data and the template data within the data knowledge base module. ; in For the source data to be tested Structural deviation measure; For the source data to be tested Layout representation; For the main body of data source Template data accumulated in the data knowledge base; The normalized layout alignment, with a value range of [0,1]. Compare the structural deviation metric with a preset threshold: if the structural deviation metric exceeds the preset threshold, the current source data is determined to be abnormal or questionable, and the process is transferred to the manual review interface; If the structural deviation metric does not exceed the preset threshold, the current source data is determined to be normal, and the layout awareness and multimodal extraction module applies the current template representation data as a priori constraint.