A method and device for standardizing and converting medical clinical data based on the SDTM standard.

Abstract

The application provides a medical clinical data standardization conversion method and device based on an SDTM standard, and relates to the technical field of medical clinical data processing. The method comprises the following steps: acquiring medical clinical data of different sources; parsing the medical clinical data into two-dimensional metadata and data entities; taking the two-dimensional metadata as a retrieval condition, and performing retrieval enhancement generation by interacting with a knowledge base by using a text generation model to obtain a mapping relationship between the two-dimensional metadata and domains and variables in the SDTM standard, wherein the knowledge base is a multi-dimensional knowledge base for medical clinical data standardization processing, which is constructed based on the SDTM standard; converting the two-dimensional metadata and the data entities according to the mapping relationship to obtain a preliminary data set meeting the SDTM standard; and performing quality detection processing on the preliminary data set to meet the SDTM standard to obtain a standard data set.

Description

Technical Field

[0001] This application relates to the field of medical clinical data processing technology, and in particular to a method and apparatus for standardizing and converting medical clinical data based on the SDTM standard. Background Technology

[0002] Medical clinical data is characterized by its wide range of sources (including hospital information systems and pharmaceutical companies' electronic data capture systems), diverse types (including structured medical records and case report forms), and varied formats (e.g., databases, application programming interfaces). The quality defects and non-standardization of this data not only affect the reliability of clinical trial results but also increase the difficulty of regulatory review, thereby delaying the approval of new drugs and the market launch of medical devices. Multinational pharmaceutical companies, biomedical research institutions, and clinical research institutions face further challenges when conducting cross-border collaborations, including differences in data standards, inconsistent formats, and privacy compliance requirements across countries. Therefore, there is an urgent need to achieve effective integration and cross-regional consistency management of multi-source heterogeneous data.

[0003] To address the standardization of medical clinical data, the Clinical Data Interchange Standards Consortium (CDISC) has developed a series of international standards, including the Study Data Tabulation Model (SDTM), which require medical clinical data to be classified by specific domains and converted into the SDTM standardized format to support regulatory submissions.

[0004] However, existing solutions for converting medical clinical data into the SDTM standard format rely on manually configuring mapping tables and writing programs to achieve the SDTM standard format conversion. These solutions suffer from low automation, high time consumption, high cost, and inability to handle multi-source heterogeneous data. Summary of the Invention

[0005] In view of the above problems, this application provides a method and apparatus for standardizing and converting medical clinical data based on the SDTM standard, so as to overcome the above problems or at least partially solve the above problems.

[0006] A first aspect of this application discloses a method for standardizing and converting medical clinical data based on the SDTM standard, the method comprising:

[0007] Obtain medical clinical data from different sources;

[0008] The medical clinical data is parsed into two-dimensional metadata and data entities. The two-dimensional metadata represents the data attribute information and feature information described in a two-dimensional table structure, and the data entities represent the actual recorded medical clinical data.

[0009] Using the two-dimensional metadata as search criteria, the search is enhanced by interacting with the text generation model and the knowledge base to obtain the mapping relationship between the two-dimensional metadata and the domains and variables in the SDTM standard of the research data tabulation model. The knowledge base is a multi-dimensional knowledge base for standardized processing of medical clinical data built based on the SDTM standard.

[0010] Based on the mapping relationship, the two-dimensional metadata and the data entities are transformed to obtain a preliminary dataset that meets the SDTM standard;

[0011] The preliminary dataset is subjected to quality inspection processing that meets the SDTM standard to obtain the standard dataset.

[0012] Optionally, medical clinical data from various sources may be acquired, including:

[0013] Medical clinical data from different sources can be obtained through data access methods or data import methods.

[0014] The medical clinical data from different sources includes: patient daily medical records from the hospital information system, electronic medical record forms from the electronic data acquisition system, and other medical clinical data from sources other than the hospital information system and the electronic data acquisition system.

[0015] Optionally, the medical clinical data is parsed into two-dimensional metadata and data entities, including:

[0016] The medical clinical data is parsed into metadata and data entities, wherein the metadata is used to describe data attribute information and feature information;

[0017] Based on the semantic information of the medical clinical data and the provisions of the SDTM standard, the metadata is converted into a standardized two-dimensional table structure to obtain the two-dimensional metadata.

[0018] Optionally, the knowledge base includes: a rule base with data processing rules and data mapping rules, a standard base integrating the SDTM standard, and a terminology base with medical terminology;

[0019] Using the two-dimensional metadata as search criteria, and leveraging a text generation model in conjunction with a knowledge base for enhanced search generation, the mapping relationship between the two-dimensional metadata and domains and variables in the SDTM standard is obtained, including:

[0020] Using the two-dimensional metadata as search criteria, the text generation model is used to search the rule base, the standard base, and the terminology base respectively, to obtain search results composed of target rules, target domain knowledge, and target terms;

[0021] The search results are integrated into system prompts, and the two-dimensional metadata is placed into user prompts.

[0022] The mapping relationship is generated using the text generation model based on the system prompts and the user prompts.

[0023] Optionally, the two-dimensional metadata and the data entities are transformed according to the mapping relationship to obtain a preliminary dataset that meets the Research Data Tabulation Model (SDTM) standard, including:

[0024] Based on the mapping relationship, the two-dimensional metadata is converted into standard metadata, which includes domain identifiers, variable labels, data types, and controlled terms;

[0025] Based on the standard metadata and the data entities, a preliminary dataset that meets the SDTM standard is generated.

[0026] Optionally, the preliminary dataset is subjected to quality inspection processing that meets the SDTM standard to obtain a standard dataset, including:

[0027] The preliminary dataset is detected according to the target detection strategy, which includes: dataset integrity check, data format and type verification, controlled terminology correction, logical contradiction check, and domain rule verification.

[0028] If an error is detected, the error message and the corresponding two-dimensional metadata are sent to the text generation model, and the following steps are returned: using the two-dimensional metadata as a retrieval condition, the text generation model interacts with the knowledge base to perform retrieval enhancement generation, and the mapping relationship between the two-dimensional metadata and the domains and variables in the standard is obtained;

[0029] If no error is triggered during detection, the preliminary dataset of the current detection will be used as the standard dataset.

[0030] Optionally, the method further includes:

[0031] If the number of times the preliminary dataset is tested according to the target detection strategy reaches a threshold, it is determined that the medical clinical data has quality problems, and the quality problems are output to indicate that manual verification is required.

[0032] Optionally, the method further includes:

[0033] Obtain the generation rules of derived variables described in natural language, wherein the derived variables represent variables that are related to the clinical trial protocol and are calculated based on recorded variables;

[0034] Using the text generation model, data processing code for configuring the derived variables is generated according to the generation rules of the derived variables;

[0035] The data processing code is executed based on the standard dataset to generate derived variable data, and the derived variable data is inserted into the standard dataset to obtain the final standard dataset.

[0036] Optionally, the final standard dataset may be manually reviewed.

[0037] After confirming that the final standard dataset meets regulatory requirements, the final standard dataset and target format file are output, wherein the target format file records the standard metadata of the final standard dataset.

[0038] A second aspect of this application discloses a medical clinical data standardization and conversion device based on the SDTM standard, the device comprising:

[0039] The acquisition module is used to acquire medical clinical data from different sources;

[0040] The parsing module is used to parse the medical clinical data into two-dimensional metadata and data entities. The two-dimensional metadata represents data attribute information and feature information described in a two-dimensional table structure, and the data entities represent the actual recorded medical clinical data.

[0041] The interaction module is used to use the two-dimensional metadata as search conditions, and to perform search enhancement generation by interacting with the knowledge base using a text generation model, so as to obtain the mapping relationship between the two-dimensional metadata and the domains and variables in the SDTM standard. The knowledge base is a multi-dimensional knowledge base built on the SDTM standard for standardized processing of medical clinical data.

[0042] The conversion module is used to convert the two-dimensional metadata and the data entities according to the mapping relationship to obtain a preliminary dataset that meets the SDTM standard;

[0043] The detection module is used to perform quality detection processing on the preliminary dataset to meet the SDTM standard, so as to obtain a standard dataset.

[0044] A third aspect of this application discloses an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the steps of the medical clinical data standardization and conversion method based on the SDTM standard described in the first aspect of this application.

[0045] A fourth aspect of this application discloses a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the steps of the medical clinical data standardization and conversion method based on the SDTM standard described in the first aspect of this application.

[0046] A fifth aspect of this application discloses a computer program product, including a computer program that, when executed by a processor, implements the steps of the medical clinical data standardization and conversion method based on the SDTM standard described in the first aspect of this application.

[0047] The embodiments of this application have the following advantages:

[0048] This application's embodiments overcome the limitations of existing methods that target single data sets. Medical clinical data from different sources is parsed into two-dimensional metadata and data entities. Using the two-dimensional metadata as search criteria, a text generation model interacts with a knowledge base to enhance retrieval and generate a mapping relationship between the two-dimensional metadata and the domains and variables in the SDTM standard. This achieves automated and accurate mapping between the two-dimensional metadata and the domains and variables in the SDTM standard, significantly reducing the professional threshold for manual configuration and improving standardization efficiency. Based on the mapping relationship, the two-dimensional metadata and data entities are transformed to obtain a preliminary dataset that meets the SDTM standard. This preliminary dataset undergoes quality checks to ensure it meets the SDTM standard, resulting in a high-quality standard dataset. Thus, low-cost, efficient, and accurate intelligent transformation of medical clinical data for standardization is achieved, reducing reliance on manual labor and high professional requirements. Attached Figure Description

[0049] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the description of the embodiments of this application will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0050] Figure 1 This is a flowchart illustrating the steps of a medical clinical data standardization and conversion method based on the SDTM standard provided in this application embodiment;

[0051] Figure 2 This is a schematic diagram of a medical clinical data standardization and conversion system based on the SDTM standard provided in an embodiment of this application;

[0052] Figure 3 This is a schematic diagram of the structure of a medical clinical data standardization and conversion device based on the SDTM standard provided in an embodiment of this application;

[0053] Figure 4This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0054] To make the above-mentioned objectives, features, and advantages of this application more apparent and understandable, the technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0055] To better understand the technical solution of this application, the technical concepts involved in this application will be explained first.

[0056] Existing methods for converting medical clinical data to the SDTM standard format involve manually defining the mapping between source and target variables using an Excel spreadsheet, then writing a SAS (Statistical Analysis System) program to perform the conversion, transforming the raw medical clinical data into the SDTM format; or defining the conversion process from source to target variables using an Excel mapping table, and then implementing the specific conversion by writing a SAS macro program.

[0057] However, these solutions have the following problems: 1) Low automation: The solutions still rely on manual Excel spreadsheets for variable mapping and rule configuration, which is inefficient and prone to human error. 2) Poor cost and ease of use, requiring a high level of expertise; personnel need to be very familiar with the SDTM standard and proficient in writing SAS programs. 3) Inability to handle multi-source heterogeneous data: Existing solutions struggle to effectively integrate heterogeneous data from diverse data sources such as HIS (Hospital Information System) and EDC (Electronic Data Capture), and can only be adapted to a specific system. 4) Poor dynamic configuration and flexible scalability: Over-reliance on the SAS ecosystem; its closed architecture limits collaboration with open-source tools such as Python, and each trial requires the redevelopment of customized macro programs, lacking reusable standardized modules. 5) Complex generation of derived variables: Derived variables are diverse and complex; complex statistical logic and clinical rules require extensive manual writing of SAS code, a process that is time-consuming and prone to data quality issues due to programming errors.

[0058] To overcome the limitations of related technologies, this application provides a method for standardizing and converting medical clinical data based on the SDTM standard. This method intelligently converts medical clinical data from different sources (e.g., HIS, EDC, etc.) into SDTM standard datasets. The method utilizes a text generation model to transform metadata in the medical clinical data into a two-dimensional form based on semantic information, and maps it to the domains and variables defined by the SDTM standard. Based on the metadata mapping relationship, the medical clinical data is progressively converted, and the initial SDTM dataset undergoes quality checks to meet the SDTM standard (e.g., implementing multi-level quality control measures including range checks, logical verification, and consistency verification) to ensure that the output data conforms to the technical specifications of the SDTM standard, thus obtaining the SDTM standard dataset.

[0059] In some embodiments, the data processing code is further generated through natural language dialogue in conjunction with the clinical trial protocol to intelligently configure derived variables, and finally outputs an SDTM standard dataset and Define-XML metadata (Define-XML is a source data description file format in CDISC used to define dataset variables, controlled terms, and specific source data). This application will help improve the review efficiency of regulatory agencies, while supporting subsequent statistical analysis, cross-study data integration, and electronic submission, thereby improving the processing efficiency and quality of clinical research data, reducing labor costs and technical barriers, and providing technical support for accelerating new drug development and medical research.

[0060] The following description, in conjunction with the accompanying drawings, details the method for standardizing and converting medical clinical data based on the SDTM standard according to embodiments of this application.

[0061] Reference Figure 1 As shown, Figure 1 This is a flowchart illustrating the steps of a medical clinical data standardization and conversion method based on the SDTM standard provided in an embodiment of this application. Figure 1 As shown, the medical clinical data standardization and transformation method based on the SDTM standard may include steps S110 to S150:

[0062] Step S110: Obtain medical clinical data from different sources.

[0063] Medical clinical data is typically structured or semi-structured, and data from different sources often have different structures. This application's embodiments overcome the limitations of existing methods that target single data sets by converting medical clinical data from different sources—that is, converting multi-source heterogeneous medical clinical data.

[0064] In one optional embodiment, acquiring medical clinical data from different sources includes: acquiring medical clinical data from different sources through a data access method or a data import method; wherein, the medical clinical data from different sources includes: patient daily medical record data from the hospital information system, electronic medical record forms from the electronic data acquisition system, and other medical clinical data from sources other than the hospital information system and the electronic data acquisition system.

[0065] In this embodiment, the patient's daily medical records from the hospital information system, including structured or semi-structured data generated from outpatient, inpatient, and laboratory examinations, while having clinical value, were not originally designed to directly serve clinical trial research. Therefore, when a new clinical trial project needs to reuse historical data (i.e., patient's daily medical records) from the hospital information system, this embodiment converts it into a standard dataset (i.e., the SDTM standard dataset) according to the specific requirements of the current trial protocol to ensure that it meets the inclusion and exclusion criteria, observation indicator definitions, and data collection specifications of the clinical trial.

[0066] The electronic case report form from the electronic data acquisition system strictly follows the data structure requirements of the clinical trial protocol during the database design phase. Each field has a clear clinical semantic and acquisition standard predefined, which makes the database structure have a data logic framework that is consistent with the trial protocol.

[0067] Other medical clinical data (i.e. other structured data) can also be converted into SDTM standard datasets using the methods implemented in this application.

[0068] Specifically, the acquisition of medical clinical data can be divided into data access and data import. Data access is a dynamic data interaction mode that uses APIs (Application Programming Interfaces), database connections, or streaming to access source data in real time or at regular intervals without directly acquiring files. This method eliminates the need for physical file transmission and storage, thus maintaining the timeliness and consistency of the data. Data import directly provides structured files of medical clinical data, allowing for the batch loading of the corresponding medical clinical data.

[0069] Step S120: Parse the medical clinical data into two-dimensional metadata and data entities. The two-dimensional metadata represents the data attribute information and feature information described in a two-dimensional table structure, and the data entities represent the actual recorded medical clinical data.

[0070] In this embodiment, two-dimensional metadata refers to metadata in a two-dimensional table structure, where rows represent observations (or actual records) and columns represent variables. Each piece of medical clinical data is described by its corresponding two-dimensional metadata, which contains its data attribute and feature information. Data entities represent the actual recorded medical clinical data, which can be understood as actual observations. For medical clinical data from different sources, their inherent patterns are identified through parsing, thereby uniformly extracting two-dimensional metadata and data entities.

[0071] In one optional embodiment, parsing the medical clinical data into two-dimensional metadata and data entities includes: parsing the medical clinical data into metadata and data entities, wherein the metadata is used to describe data attribute information and feature information; and converting the metadata into a standardized two-dimensional table structure according to the semantic information of the medical clinical data and the provisions of the SDTM standard to obtain the two-dimensional metadata.

[0072] Specifically, after obtaining the metadata through parsing, based on the semantic information of medical clinical data and the provisions of the SDTM standard, the metadata is organized into a standardized two-dimensional table structure through an intelligent mapping algorithm. This ensures that the fields of each single table can accurately correspond to the domains defined in the SDTM dataset during subsequent transformation, thereby improving the accuracy of data standardization transformation.

[0073] Step S130: Using the two-dimensional metadata as a retrieval condition, the text generation model interacts with the knowledge base to perform retrieval enhancement generation, thereby obtaining the mapping relationship between the two-dimensional metadata and the domains and variables in the SDTM standard of the research data tabulation model. The knowledge base is a multi-dimensional knowledge base for standardized processing of medical clinical data built based on the SDTM standard.

[0074] The mapping relationship between two-dimensional metadata and domains and variables in the SDTM standard can also be understood as a conversion template for performing medical clinical data (i.e., two-dimensional metadata and data entities).

[0075] The text generation model can be an open-source general-purpose large language model (e.g., Tongyi Qianwen, DeepSeek, etc.). It can be migrated to the closed environment of a local medical data center through private deployment, thus avoiding the risk of leakage of sensitive medical clinical data.

[0076] The text generation model can also be fine-tuned locally. Specifically, the locally fine-tuned model is a customized model that is fine-tuned for domain adaptation based on local medical corpora (such as professional texts such as electronic medical records and clinical trial reports). Through continuous training and parameter optimization in the medical vertical field, the locally fine-tuned model can more accurately understand medical terminology and grasp the logic of clinical context, and show stronger domain adaptability in professional scenarios such as data standardization conversion and case report generation.

[0077] In this embodiment, an intelligent mapping technology based on a knowledge base and a text generation model is used to construct a multidimensional knowledge base for standardized processing of medical clinical data based on the SDTM standard. This allows for enhanced generation through interaction between the text generation model and the knowledge base, achieving automated and accurate mapping between two-dimensional metadata and domains and variables in the SDTM standard. This significantly reduces the professional threshold for manual configuration and improves standardization efficiency.

[0078] In one optional embodiment, the knowledge base includes: a rule base with data processing rules and data mapping rules, a standard base integrating the SDTM standard, and a terminology base with medical terminology.

[0079] The rule base includes structured processing rules such as data cleaning, logical verification, and anomaly detection. Data mapping rules can be understood as establishing standardized transformation templates for field mapping relationships between different data sources and the SDTM standard. The standard library integrating the SDTM standard can be understood as a standard library that integrates domain knowledge from SDTM standard technical documents and implementation guidelines. The terminology library containing medical terminology can be understood as a system of professional terminology related to pharmaceutical clinical trials.

[0080] Furthermore, step S130 specifically includes sub-steps S130-1 to S130-3:

[0081] Step S130-1: Using the two-dimensional metadata as search conditions, the text generation model is used to search the rule base, the standard base, and the terminology base respectively to obtain search results composed of target rules, target domain knowledge, and target terms.

[0082] Step S130-2: Integrate the search results into the system prompts, and place the two-dimensional metadata into the user prompts.

[0083] Step S130-3: Using the text generation model, generate the mapping relationship based on the system prompt words and the user prompt words.

[0084] Specifically, using two-dimensional metadata as search criteria, the system retrieves target rules relevant to the current data processing task by searching the rule base, target domain knowledge relevant to the current data processing task by searching the standard library, and target terms relevant to the current data processing task by searching the terminology library. The current data processing task can be determined based on the user's experimental plan and description information. The search results are integrated into system prompts to form a processing context, and the two-dimensional metadata is placed into the user prompts as specific processing objects. This allows a text generation model to generate a mapping relationship between the two-dimensional metadata and the domains and variables in the SDTM standard.

[0085] Step S140: Convert the two-dimensional metadata and the data entity according to the mapping relationship to obtain a preliminary dataset that meets the SDTM standard.

[0086] In this embodiment of the application, since the mapping relationship contains the precise matching relationship between two-dimensional metadata and domains and variables in the SDTM standard, the two-dimensional metadata and data entities can be accurately converted into a preliminary dataset that meets the SDTM standard based on the mapping relationship.

[0087] In an optional embodiment, step S140 specifically includes sub-steps S140-1 to S140-2:

[0088] Step S140-1: Based on the mapping relationship, convert the two-dimensional metadata into standard metadata, which includes domain identifiers, variable labels, data types, and controlled terms.

[0089] Step S140-2: Generate a preliminary dataset that meets the SDTM standard based on the standard metadata and the data entity.

[0090] In this dataset, domain identifiers are used to identify specific domains; variable labels are used to identify specific variables, and each variable label in the SDTM dataset should be unique; data types can include character or numeric types; controlled terms refer to data stored in the SDTM standard dataset, where variable values ​​are presented through terms.

[0091] In some embodiments, standard metadata also includes: variable name, source (to indicate where the data comes from), role (to determine how the variable is used in the relevant dataset), and other relevant and necessary information for drug regulatory agencies to communicate about the variable and its contents.

[0092] In this embodiment, the mapping relationship is parsed, and standard metadata is constructed according to the data structure requirements specified in the SDTM standard implementation guidelines; wherein, the standard metadata is also metadata with a two-dimensional table structure. Then, the data entities and the standard metadata are integrated to generate a preliminary dataset that meets the SDTM standard.

[0093] Step S150: Perform quality inspection processing on the preliminary dataset to meet the SDTM standard to obtain a standard dataset.

[0094] After obtaining the preliminary dataset, a quality check process is performed to ensure that the generated standard dataset conforms to the SDTM standard's technical specifications, resulting in a high-quality standard dataset. This quality check process includes multi-level quality control measures such as range checks, logical verification, and consistency validation.

[0095] In an optional embodiment, step S150 specifically includes sub-steps S150-1 to S150-3:

[0096] Step S150-1: Detect the preliminary dataset according to the target detection strategy, which includes: dataset integrity check, data format and type verification, controlled terminology correction, logical contradiction check, and domain rule verification.

[0097] Step S150-2: If an error is detected, the error message and the corresponding two-dimensional metadata are sent to the text generation model, and the process returns to step S130: Using the two-dimensional metadata as a retrieval condition, the text generation model interacts with the knowledge base to perform retrieval enhancement generation, and obtains the mapping relationship between the two-dimensional metadata and the domains and variables in the standard.

[0098] Step S150-3: If no error is triggered during detection, the preliminary dataset of the current detection is used as the standard dataset.

[0099] The dataset integrity check includes key domain checks and required field filling checks. Key domain checks confirm the absence of core domains such as demographics and adverse events. Required field filling checks ensure that identifiers such as domain names and subject identifiers are present and not empty, and that key variables for specific domains are filled.

[0100] Data format and type validation includes date and time format checks, as well as numerical range checks. The date and time format check converts DTCs (Date / Time of Comment, time variables) into ISO 8601 format. In SDTM, if a variable ends with DTC, it indicates the time of comment, intervention, or check. For example, AESTDTC (Adverse EventStart Date / Time of Comment) indicates the start time of the adverse event. The numerical range check checks whether the numerical variable is within a reasonable range. For example, age cannot be negative.

[0101] Controlled terminology correction is used to identify erroneous terms and replace them with standard terms, and to ensure that any added variables are declared in Define-XML (Define-XML is not generated at the end of this method, but is initialized from the beginning of the medical clinical data parsing in step S120 and is continuously improved thereafter).

[0102] Logical contradiction checks are used to identify and correct contradictions and inconsistencies in the data, such as ensuring the end date is not earlier than the start date or that the subject's death date is within the study period. Domain rule validation is used to validate data according to predefined domain rules, such as the LB (Laboratory Test Results) domain unit consistency check.

[0103] In this embodiment, after obtaining preliminary data through step S140, step S150-1 is first executed to detect the preliminary dataset according to the target detection strategy. If an error is triggered during detection, step S150-2 is executed to send the error information and corresponding two-dimensional metadata to the text generation model, so as to return to step S130. The text generation model then provides the mapping relationship between the two-dimensional metadata and the domains and variables in the standard, and steps S140 and S150-1 are executed sequentially to generate a new preliminary dataset and continue to detect the newly generated preliminary dataset. Until no error is triggered during detection, step S150-2 is executed to use the currently detected preliminary dataset as the standard dataset.

[0104] Understandably, since the target detection strategy includes multiple types of detection (i.e., dataset integrity check, data format and type validation, controlled term correction, logical contradiction check, and domain rule validation), if multiple errors are triggered, all error information and corresponding two-dimensional metadata can be sent to the text generation model for processing after each detection is completed. Alternatively, if only one error is triggered, the corresponding error information and corresponding two-dimensional metadata can be sent to the text generation model for processing.

[0105] Furthermore, the above-mentioned S150 also includes sub-step S150-4:

[0106] Step S150-4: If the number of times the preliminary dataset is detected according to the target detection strategy reaches the threshold, it is determined that there is a quality problem in the medical clinical data, and the quality problem is output to indicate that manual verification is required.

[0107] In this embodiment of the application, a number of detection thresholds can be set. When the number of detections on the preliminary dataset reaches (or equals) the number of detection thresholds, step S150-4 is executed to output quality problems, thereby prompting that manual verification is required.

[0108] Through the above implementation process, the preliminary dataset obtained based on the mapping relationship transformation is subjected to quality inspection processing that meets the SDTM standard to ensure that a high-quality standard dataset is obtained; thereby achieving low-cost, efficient and accurate standardized intelligent transformation of medical clinical data, reducing reliance on manual labor and high professional requirements.

[0109] In an optional embodiment, step S150 is further included after step S160:

[0110] Step S160: Obtain the generation rules of derived variables described in natural language, wherein the derived variables represent variables that are related to the clinical trial protocol and are calculated based on recorded variables; use the text generation model to generate data processing code for configuring the derived variables according to the generation rules of the derived variables; execute the data processing code according to the standard dataset to generate derived variable data, and insert the derived variable data into the standard dataset to obtain the final standard dataset.

[0111] In this embodiment, natural language interaction technology is used to achieve intelligent configuration of clinical trial derived variables. The generation rules of derived variables described by the user in natural language are obtained (e.g., "Please create a new derived variable: calculate BMI = weight (kg) / (height (m)^2), where weight and height data are in the VS domain," where BMI refers to Body Mass Index and VS refers to Vital Signs). The text generation model then automatically converts the generation rules of derived variables (i.e., the unstructured requirements input by the user) into program code (i.e., data processing code). Executing the data processing code enables intelligent configuration of the derived variables.

[0112] Thus, the natural language-driven intelligent generation technology for derived variables, combined with clinical trial protocols, uses natural language dialogue to dynamically generate executable data processing code, replacing traditional manual programming and solving the problems of complex and error-prone configuration of derived variables.

[0113] In an optional embodiment, step S160 is further included after step S170:

[0114] Step S170: The final standard dataset is manually reviewed; after confirming that the final standard dataset meets regulatory requirements, the final standard dataset and target format file are output, wherein the target format file records the standard metadata of the final standard dataset.

[0115] In this embodiment, the target format file refers to the Define-XML metadata document. After the conversion of medical clinical data is completed through steps S110 to S160, step S170 is executed, where the clinical data management personnel manually review the final standard dataset. Finally, the final standard dataset and the target format file are output. The final standard dataset follows the domain structure and variable definitions of the SDTM standard, and the standard metadata recorded in the target format file includes variable labels, value ranges, controlled terms, etc.

[0116] The technical solution implemented in this application overcomes the limitations of existing methods that target single data sets. It parses medical clinical data from different sources into two-dimensional metadata and data entities. Using the two-dimensional metadata as search criteria, it leverages a text generation model and knowledge base interaction to enhance retrieval and generate a mapping relationship between the two-dimensional metadata and the domains and variables in the SDTM standard. This achieves automated and accurate mapping between the two-dimensional metadata and the domains and variables in the SDTM standard, significantly reducing the professional threshold for manual configuration and improving standardization efficiency. Based on this mapping relationship, the two-dimensional metadata and data entities are transformed to obtain a preliminary dataset that meets the SDTM standard. This preliminary dataset undergoes quality checks to ensure it meets the SDTM standard, resulting in a high-quality standard dataset. Thus, it achieves low-cost, efficient, and accurate intelligent standardization transformation of medical clinical data, reducing reliance on manual labor and the high level of professional expertise required.

[0117] like Figure 2 As shown, Figure 2 This is a schematic diagram of a medical clinical data standardization and conversion system based on the SDTM standard provided in this application embodiment. The medical clinical data standardization and conversion method based on the SDTM standard provided in this application embodiment can be implemented through a medical clinical data standardization and conversion system based on the SDTM standard. The system includes a preprocessing module, a derived variable generation module, a mapping execution module, a quality management module, a text generation module interface, a user dialogue interface, a text generation model, and a knowledge base.

[0118] Specifically, medical clinical data from different sources (e.g., hospital information systems, electronic data acquisition systems, other structured data) are acquired through data access methods or data import methods, and the medical clinical data from different sources are parsed into two-dimensional metadata and data entities through a preprocessing module.

[0119] The text generation module interface integrates the text generation model and knowledge base (rule base, standard base, and terminology base) into the system. Then, using two-dimensional metadata as search criteria, the text generation model searches the rule base, standard base, and terminology base respectively to obtain the target rules, target domain knowledge, and target terms most relevant to the current data processing task. The current data processing task can be determined based on the user's experimental plan and user description information.

[0120] The search results, consisting of target rules, target domain knowledge, and target terms, are integrated into system prompts, and two-dimensional metadata is placed into user prompts. Based on the system prompts and user prompts, a text generation model is used to generate a mapping relationship between the two-dimensional metadata and the domains and variables in the SDTM standard.

[0121] Next, the mapping execution module converts the two-dimensional metadata into standard metadata according to the mapping relationship; the quality management module generates a preliminary dataset that meets the SDTM standard based on the standard metadata and data entities, and performs quality checks on the preliminary dataset to meet the SDTM standard, thus obtaining the standard dataset.

[0122] Next, the generation rules of derived variables, described in natural language, are obtained through a user dialogue interface. Using a text generation model, data processing code for configuring these derived variables is generated based on these rules. The derived variable generation module executes the data processing code according to the standard dataset, generating derived variable data and inserting it into the standard dataset to obtain the final standard dataset. Finally, the final standard dataset is manually reviewed. After confirming that the final standard dataset meets regulatory requirements, the final standard dataset and target format file are output.

[0123] In this way, the system transforms medical clinical data from different sources into standard datasets that meet the SDTM standard. The transformation process uses a preprocessing module to parse the medical clinical data into two-dimensional metadata and data entities, and accurately matches them to the domains and variables defined by the SDTM standard based on a predefined knowledge base. At the same time, an integrated data quality control module ensures the accuracy and completeness of the transformation results, and finally generates a standardized dataset and Define-XML that can be directly submitted to regulatory agencies. This significantly reduces the time cost and technical threshold of traditional manual transformation methods, and provides an automated solution for the standardized transformation of multi-source heterogeneous clinical data.

[0124] This application also provides a medical clinical data standardization and conversion device based on the SDTM standard, referring to... Figure 3 As shown, Figure 3 This is a schematic diagram of a medical clinical data standardization and conversion device based on the SDTM standard provided in an embodiment of this application. The device includes:

[0125] The acquisition module 310 is used to acquire medical clinical data from different sources;

[0126] The parsing module 320 is used to parse the medical clinical data into two-dimensional metadata and data entities. The two-dimensional metadata represents data attribute information and feature information described in a two-dimensional table structure, and the data entities represent the actually recorded medical clinical data.

[0127] The interaction module 330 is used to use the two-dimensional metadata as a retrieval condition, and to perform retrieval enhancement generation by interacting with the knowledge base using a text generation model, so as to obtain the mapping relationship between the two-dimensional metadata and the domains and variables in the SDTM standard. The knowledge base is a multi-dimensional knowledge base built on the SDTM standard for standardized processing of medical clinical data.

[0128] The conversion module 340 is used to convert the two-dimensional metadata and the data entity according to the mapping relationship to obtain a preliminary dataset that meets the SDTM standard;

[0129] The detection module 350 is used to perform quality detection processing on the preliminary dataset to meet the SDTM standard, so as to obtain a standard dataset.

[0130] In an optional embodiment, the acquisition module is further configured to acquire medical clinical data from different sources through a data access method or a data import method.

[0131] The medical clinical data from different sources includes: patient daily medical records from the hospital information system, electronic medical record forms from the electronic data acquisition system, and other medical clinical data from sources other than the hospital information system and the electronic data acquisition system.

[0132] In one optional embodiment, the parsing module includes:

[0133] The first parsing submodule is used to parse the medical clinical data into metadata and data entities, wherein the metadata is used to describe data attribute information and feature information;

[0134] The second parsing submodule is used to convert the metadata into a standardized two-dimensional table structure according to the semantic information of the medical clinical data and the provisions of the SDTM standard, so as to obtain the two-dimensional metadata.

[0135] In one optional embodiment, the knowledge base includes: a rule base with data processing rules and data mapping rules, a standard library integrating the SDTM standard, and a terminology library with medical terminology; the interaction module includes:

[0136] The knowledge retrieval module is used to use the two-dimensional metadata as retrieval conditions and the text generation model to retrieve the rule base, the standard base and the terminology base respectively, and obtain retrieval results composed of target rules, target domain knowledge and target terms;

[0137] The prompt word module is used to integrate the search results into system prompt words, and to place the two-dimensional metadata into user prompt words;

[0138] The relationship generation module is used to generate the mapping relationship based on the system prompt words and the user prompt words using the text generation model.

[0139] In one optional embodiment, the conversion module includes:

[0140] The metadata conversion module is used to convert the two-dimensional metadata into standard metadata according to the mapping relationship. The standard metadata includes domain identifiers, variable labels, data types, and controlled terms.

[0141] The data generation module is used to generate a preliminary dataset that meets the SDTM standard based on the standard metadata and the data entities.

[0142] In one optional embodiment, the detection module includes:

[0143] The first detection submodule is used to detect the preliminary dataset according to the target detection strategy, which includes: dataset integrity check, data format and type verification, controlled terminology correction, logical contradiction check, and domain rule verification.

[0144] The second detection submodule is used to send the error information and corresponding two-dimensional metadata to the text generation model when an error is triggered by detection, and return the execution steps: using the two-dimensional metadata as a retrieval condition, using the text generation model to interact with the knowledge base to perform retrieval enhancement generation, and obtaining the mapping relationship between the two-dimensional metadata and the domains and variables in the standard;

[0145] The third detection submodule is used to use the preliminary dataset of the current detection as the standard dataset if no error is triggered during detection.

[0146] In an optional embodiment, the device further includes:

[0147] The prompting module is used to determine that the medical clinical data has quality problems when the number of times the preliminary dataset is detected according to the target detection strategy reaches a threshold, and outputs the quality problems to prompt that manual verification is required.

[0148] In an optional embodiment, the device further includes:

[0149] The rule acquisition module is used to acquire the generation rules of derived variables described in natural language, wherein the derived variables represent variables that are related to the clinical trial protocol and are calculated based on recorded variables;

[0150] The code generation module is used to generate data processing code for configuring the derived variables based on the text generation model and the generation rules of the derived variables.

[0151] The derived data module is used to execute the data processing code based on the standard dataset, generate derived variable data, and insert the derived variable data into the standard dataset to obtain the final standard dataset.

[0152] In an optional embodiment, the device further includes:

[0153] The verification module is used to manually verify the final standard dataset;

[0154] The output module is used to output the final standard dataset and the target format file after confirming that the final standard dataset meets regulatory requirements. The target format file records the standard metadata of the final standard dataset.

[0155] This application also provides an electronic device, see embodiments thereof. Figure 4 , Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. For example... Figure 4 As shown, the electronic device 400 includes a memory 410 and a processor 420. The memory 410 and the processor 420 are connected via a bus for communication. The memory 410 stores a computer program that can run on the processor 420 to implement the steps of the medical clinical data standardization and conversion method based on the SDTM standard described in the embodiments of this application.

[0156] This application also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the steps of the medical clinical data standardization and conversion method based on the SDTM standard described in this application.

[0157] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the medical clinical data standardization and conversion method based on the SDTM standard described in this application.

[0158] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.

[0159] This application describes embodiments of methods and apparatus according to flowchart illustrations and / or block diagrams. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing terminal device to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing terminal device, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0160] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing terminal device to operate in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0161] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal equipment, causing a series of operational steps to be performed on the computer or other programmable terminal equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable terminal equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0162] Although preferred embodiments of the present application have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of the embodiments of the present application.

[0163] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or terminal device that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or terminal device. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or terminal device that includes said element.

[0164] The above provides a detailed description of a medical clinical data standardization and conversion method and apparatus based on the SDTM standard provided in this application. Specific examples have been used to illustrate the principles and implementation methods of this application. The description of the above embodiments is only for the purpose of helping to understand the method and its core ideas. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A method for standardization conversion of medical clinical data based on SDTM standard, characterized in that, The method comprises: acquiring medical clinical data of different sources; parsing the medical clinical data into two-dimensional metadata and data entities, the two-dimensional metadata representing data attribute information and feature information described in a two-dimensional table structure, and the data entities representing actual recorded medical clinical data; using the two-dimensional metadata as a retrieval condition, performing retrieval enhancement generation by interacting with a knowledge base using a text generation model to obtain a mapping relationship between the two-dimensional metadata and domains and variables in a study data tabulation model (SDTM) standard, the knowledge base being a multi-dimensional knowledge base for medical clinical data standardization processing based on the SDTM standard; converting the two-dimensional metadata and the data entities according to the mapping relationship to obtain a preliminary data set meeting the SDTM standard; performing quality detection processing on the preliminary data set to meet the SDTM standard to obtain a standard data set; wherein the knowledge base comprises a rule base having data processing rules and data mapping rules, a standard library integrating the SDTM standard, and a terminology library having medical field professional terms; using the two-dimensional metadata as a retrieval condition, performing retrieval enhancement generation by interacting with a knowledge base using a text generation model to obtain a mapping relationship between the two-dimensional metadata and domains and variables in a study data tabulation model (SDTM) standard, comprising: using the two-dimensional metadata as a retrieval condition, using the text generation model to perform retrieval on the rule base, the standard library and the terminology library respectively to obtain a retrieval result composed of target rules, target domain knowledge and target terms; integrating the retrieval result into system prompt words, and placing the two-dimensional metadata into user prompt words; using the text generation model to generate the mapping relationship based on the system prompt words and the user prompt words.

2. The method for standardization conversion of medical clinical data based on SDTM standard according to claim 1, characterized in that, Acquiring medical clinical data of different sources comprises: acquiring medical clinical data of different sources through data access or data import methods; wherein the medical clinical data of different sources comprises patient daily visit record data from a hospital information system, electronic case report forms from an electronic data collection system, and other medical clinical data from sources other than the hospital information system and the electronic data collection system.

3. The method for standardization conversion of medical clinical data based on SDTM standard according to claim 1, characterized in that, Parsing the medical clinical data into two-dimensional metadata and data entities comprises: parsing the medical clinical data into metadata and data entities, the metadata being used to describe data attribute information and feature information; converting the metadata into a standard two-dimensional table structure according to semantic information of the medical clinical data and provisions of the SDTM standard to obtain the two-dimensional metadata.

4. The method for standardization conversion of medical clinical data based on SDTM standard according to claim 1, characterized in that, Converting the two-dimensional metadata and the data entities according to the mapping relationship to obtain a preliminary data set meeting the SDTM standard comprises: converting the two-dimensional metadata into standard metadata according to the mapping relationship, the standard metadata comprising domain identifiers, variable labels, data types and controlled terms; generating a preliminary data set meeting the SDTM standard according to the standard metadata and the data entities.

5. The method for standardization conversion of medical clinical data based on SDTM standard according to claim 1 or 4, characterized in that, The preliminary data set is subjected to quality detection processing meeting the SDTM standard to obtain a standard data set, including: According to the target detection strategy, the preliminary data set is detected, and the target detection strategy includes: data set integrity check, data format and type verification, controlled term correction, logical contradiction check, domain rule verification; In the case that the detection triggers an error report, the error report information and the corresponding two-dimensional metadata are sent to the text generation model, and the execution step of using the text generation model and the knowledge base to interact for retrieval enhancement generation by taking the two-dimensional metadata as a retrieval condition is returned to obtain the mapping relationship between the two-dimensional metadata and the domains and variables in the standard; In the case that the detection does not trigger an error report, the currently detected preliminary data set is taken as the standard data set.

6. The method for standardization conversion of medical clinical data based on SDTM standard according to claim 5, characterized in that, The method further includes: In the case that the number of times of detecting the preliminary data set according to the target detection strategy reaches a number threshold, it is determined that the medical clinical data has a quality problem, and the quality problem is output to prompt that manual checking needs to be performed.

7. The method for standardization conversion of medical clinical data based on SDTM standard according to claim 1, characterized in that, The method further includes: Obtaining a generation rule of a derived variable described by natural language, the derived variable representing a variable calculated based on a recorded variable and related to a clinical trial plan; Using the text generation model, generating data processing code for configuring the derived variable according to the generation rule of the derived variable; Executing the data processing code according to the standard data set, generating derived variable data, and inserting the derived variable data into the standard data set to obtain a final standard data set.

8. The medical clinical data standardization conversion method based on the SDTM standard according to claim 7, the method further comprising: reviewing the final standard data set by manual; after confirming that the final standard data set meets the regulatory requirements, outputting the final standard data set and a target format file, the target format file recording standard metadata of the final standard data set.

9. An apparatus for standardizing conversion of medical clinical data based on the SDTM standard, characterized by, includes: an acquisition module configured to acquire medical clinical data from different sources; an analysis module configured to analyze the medical clinical data into two-dimensional metadata and data entities, the two-dimensional metadata representing data attribute information and feature information described in a two-dimensional table structure, and the data entities representing actual recorded medical clinical data; an interaction module configured to use a text generation model and a knowledge base to interact for retrieval enhancement generation by taking the two-dimensional metadata as a retrieval condition to obtain a mapping relationship between the two-dimensional metadata and domains and variables in the SDTM standard, the knowledge base being a multi-dimensional knowledge base for medical clinical data standardization processing based on the SDTM standard; a conversion module configured to convert the two-dimensional metadata and the data entities according to the mapping relationship to obtain a preliminary data set meeting the SDTM standard; a detection module configured to perform quality detection processing meeting the SDTM standard on the preliminary data set to obtain a standard data set; wherein the knowledge base includes: a rule base having data processing rules and data mapping rules, a standard library integrating the SDTM standard, and a terminology library having medical field professional terms. The two-dimensional metadata is taken as a retrieval condition, retrieval enhancement generation is performed by interacting with a knowledge base by using a text generation model, and a mapping relationship between the two-dimensional metadata and domains and variables in a study data tabulation model (SDTM) standard is obtained, including: the two-dimensional metadata is taken as a retrieval condition, the rule base, the standard library, and the term library are retrieved by using the text generation model respectively, and a retrieval result composed of target rules, target domain knowledge, and target terms is obtained; the retrieval result is integrated into system prompt words, and the two-dimensional metadata is put into user prompt words; and the mapping relationship is generated based on the system prompt words and the user prompt words by using the text generation model.

Images