Automated methods, equipment, and media for standardizing clinical research analysis datasets

Abstract

The present invention relates to the field of data governance, specifically to an automated method, device and medium for standardization of clinical research analysis data sets. The method includes compiling a variable derivation method syntax standard and building a variable derivation rule library; identifying the multi-dimensional characteristics of the variable derivation rules in the user's variable description file by building a variable derivation rule parsing model, and dynamically matching the parsing rules to automatically generate SAS statements for deriving variables; based on the derivation logic of each variable in the user's variable description file, judging the directed acyclic dependency relationship between each variable, using a dynamic topological sorting algorithm to infer the optimal output order of the SAS statement, and outputting an analysis data set standardization program file, calling the file to generate a standardized analysis data set. Compared with traditional manual writing, the present invention realizes the automatic generation of an analysis data set standardization program, reduces the requirements for SAS programming experience, and improves work efficiency and quality.

Description

Technical Field

[0001] The present invention relates to the field of data governance technology, and specifically to automated methods, equipment, and media for standardizing clinical research and analysis data sets. Background Art

[0002] SAS (Statistical Analysis System) is a widely used statistical analysis software and a programming language, mainly used in data processing, statistical analysis, business intelligence, data mining and other fields.

[0003] In the field of clinical research, including both registered and non-registered clinical studies, accurate and clearly structured analytical datasets are a prerequisite for the accurate and efficient output of statistical analysis results. However, under normal circumstances, one experienced SAS programmer is required to write a set of analytical dataset description documents based on the data structure of the original dataset, the information in the annotated case report form, and the statistical analysis table template; and two SAS programmers with extensive programming experience and proficient programming skills independently write two sets of programs according to the requirements of the analytical dataset description documents to standardize the same set of original datasets, thereby generating two completely identical standardized analytical datasets.

[0004] The analysis dataset description file is equivalent to a programming plan. It consists of the attributes of the analysis dataset (name, structure, and key variables), the attributes of the variables in each dataset (name, type, length, and format), and the sources (derivation methods) of the variables. (This information together constitutes the variable derivation rules of the analysis dataset.) A preliminary draft is usually completed before the program is written. During the program writing process, it needs to be continuously revised according to the actual data until it is perfected. This means that the two programmers need to frequently modify their respective programs based on the revised analysis dataset description file. At the same time, due to poor project communication and other reasons, the programmers' programs may not be consistent with the requirements of the analysis dataset description file.

[0005] In summary, the existing analytical dataset standardization process requires highly skilled programmers, consumes a significant amount of time and effort, has a high programming threshold, and is inefficient. Therefore, there is an urgent need for an automated method, device, and medium for standardizing clinical research analytical datasets. Summary of the Invention

[0006] To avoid the above-mentioned problems existing in the prior art, the purpose of the present invention is to provide a method, device and medium for automating the standardization procedure of clinical research analysis data sets, which can use SAS programs to read and recognize the natural language in variable derivation rules, parse the variable derivation rules into SAS language; and automatically adjust the order of SAS statements according to the characteristics of the variable derivation rules, and finally output a SAS program that can generate an analysis data set.

[0007] To achieve the above objectives, the present invention provides the following technical solution: a method for automating the standardization process of clinical research analysis data sets, comprising the following steps:

[0008] S1: Prepare grammatical standards for variable derivation methods, standardize variable derivation rules from previous clinical research data based on the grammatical standards, and build a variable derivation rule library;

[0009] S2: Building a variable derivation rule parsing model based on the variable derivation rule library, identifying categories of multi-dimensional features of the variable derivation rules, and establishing differentiated parsing rules for different feature categories;

[0010] S3: Generate the variable description file required for the user's clinical research project based on the variable derivation rule in the variable derivation rule library selected by the user, or write a new variable derivation rule based on the grammatical standard;

[0011] S4: extracting multi-dimensional features from the variable description file using the variable derivation rule parsing model, dynamically matching the parsing rules using a rule engine according to the categories of the multi-dimensional features, and calling a SAS statement library to generate a SAS statement for deriving variables;

[0012] S5: Using a topological sorting algorithm, processing the directed acyclic dependency relationship between the variables in the variable description file according to the in-degree decay principle, outputting the variables with in-degree 0 in sequence, and dynamically inferring the optimal output order of the SAS statement;

[0013] S6: obtaining a standardized program file of the analysis data set based on the optimal output sequence output;

[0014] S7: calling the analysis data set standardization program file to directly generate a standardized analysis data set.

[0015] The present invention is further configured such that step S1 specifically comprises compiling a grammatical standard for the variable derivation method based on the variable derivation method of previous clinical research data and combined with the experience of previous clinical research projects; standardizing the variable derivation rules of previous clinical studies according to the grammatical standard, and constructing a variable derivation rule library based on the standardized variable derivation rules.

[0016] The present invention is further configured to use EXCEL software to establish the variable derivation rule base, and use EXCEL software to establish the variable derivation rule base, wherein the variable derivation rule base stores the standardized variable derivation rules.

[0017] The present invention is further configured such that, in step S2, the variable derivation rule parsing model is driven by natural language processing (NLP) logic and uses a text processing function or a regular expression function to identify the categories of the multi-dimensional features of the variable derivation rule.

[0018] The present invention is further configured such that the categories of the multi-dimensional features of the variable derivation rules include grammatical structure, semantic logic and keywords;

[0019] Based on natural language processing (NLP) logic, regular expression matching and keyword recognition are performed on grammatical structures; and the nesting level of conditional statements is determined based on semantic logic.

[0020] The present invention is further configured such that the text processing functions include SCAN, KSCAN, SUBSTR, LENGTH, KLENGTH, LENGTHN, LENGTHC, LENGTHM, UPCASE, LOWCASE, PROPCASE, COMPBL, COMPRESS, KCOMPRESS, STRIP, INDEX, FIND, COUNT, and TRANWRD;

[0021] The regular expression functions include PRXMATCH and PRXCHANGE.

[0022] The present invention is further configured such that the method for automating a standardization procedure for a clinical research analysis data set further comprises establishing a SAS statement library, wherein the SAS statement library stores segmented SAS statements corresponding to characteristics of the derived method, which are written based on previous clinical research experience;

[0023] The variable derivation rule parsing model can directly generate SAS statements according to the dynamically matched parsing rules, or generate SAS statements by calling a SAS statement library.

[0024] The present invention is further configured such that step S5 specifically includes the following steps:

[0025] S51: Construct directed acyclic dependency relationships between variables based on the derivation logic between variables in the variable description file;

[0026] S52: Calculate the in-degree of each variable, where the in-degree of the variable directly derived from the original dataset is 0;

[0027] For other variables, use the PRXCHANGE function to delete the variables that have generated SAS statements; use the PRXCHANGE function to identify the variables that have not generated SAS statements and are not repeated, and then use the COUNT function to count the number of variables with dependent relationships, which is the in-degree;

[0028] S53: Based on the in-degree decay principle, the order of outputting variables with in-degree 0 is used as the optimal output order of the SAS statement.

[0029] The present invention is further configured such that step S53 specifically includes first outputting the SAS statements corresponding to the variables with an in-degree of 0 and releasing the dependencies of the output variables; subtracting the number of released dependencies from the in-degrees of the remaining variables, and outputting the SAS statements corresponding to the variables with an in-degree of 0 again; and repeating the above steps until all variables are output.

[0030] The present invention is further configured such that step S5 is specifically, and step S6 is specifically, according to the optimal output order obtained by the topological sorting algorithm, the ordered execution sequence is used to reconstruct the code, and a standardized program file of the analysis data set that meets the preset standards is output through standardized code encapsulation technology.

[0031] The present invention further includes an electronic device, comprising:

[0032] a memory storing executable instructions;

[0033] A processor runs the executable instructions in the memory to implement the above-mentioned method for automating the standardization procedure of clinical research analysis data sets.

[0034] The present invention also includes a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the above-mentioned method for automating the standardization procedure of clinical research analysis data sets.

[0035] The present invention collects and organizes variable derivation rules described in analysis data set description documents used in previous clinical studies and constructs a variable derivation rule library. Using natural language processing logic, the present invention employs a SAS program to read and recognize the natural language in the variable derivation rules, parses the variable derivation rules into SAS language, and automatically adjusts the order of SAS statements based on the characteristics of the variable derivation rules, ultimately outputting a SAS program capable of generating an analysis data set.

[0036] In summary, the beneficial effects of the technical solution of the present invention are as follows:

[0037] (1) The present invention provides an automated method for standardizing clinical research analysis data sets, which lowers the threshold for writing analysis data set standardization programs and improves work efficiency. Users do not need to master SAS, R, or other programming languages. They only need to select the variables required for the clinical research project and their corresponding derivation rules from the variable derivation rule library to export the analysis data set description file for the project; at the same time, they only need to fill in the special information of the new project in the analysis data set description file using appropriate standardized natural language to quickly complete the analysis data set standardization program and the standardized analysis data set. The minimum work experience requirement for users is reduced from 2 years to 2 months; manual programming is changed to automatic programming, which improves programming efficiency by 90%. At the same time, new variable derivation rules written according to the grammatical standard of the present invention can be added to the variable derivation rule library for use in subsequent projects, thereby improving the marginal benefits of writing new variable derivation rules.

[0038] (2) The present invention automatically outputs programs by parsing the analysis dataset description file, avoiding errors that may result from manual coding, ensuring that the program complies with the requirements of the analysis dataset description file, and improving work quality. At the same time, based on defined parsing rules, the present invention avoids the risk of uncontrollable results brought about by the use of deep learning black box models, and can meet the high data quality requirements in the field of clinical research.

[0039] (3) Wide scope of application. The present invention is not only applicable to generating SDTM and AdaM data sets and their programs specified in the CDISC standard, but also applicable to generating non-CDISC standard analysis data sets and their programs. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for describing the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0041] Figure 1 Flowchart of the automated method for standardizing clinical research analysis datasets according to an embodiment of the present invention. DETAILED DESCRIPTION

[0042] In order to enable those skilled in the art to better understand the technical solution of the present invention, the technical solution of the present invention is clearly and completely described below in conjunction with the accompanying drawings of the present invention. Based on the embodiments of the present invention, other similar embodiments obtained by ordinary technicians in this field without making any creative work should fall within the scope of protection of the present invention.

[0043] The present invention will be further described below with reference to the accompanying drawings and preferred embodiments.

[0044] Example 1:

[0045] like Figure 1 As shown in FIG. 1 , a preferred embodiment of the present invention is a method for automating a standardization process of a clinical research analysis data set, comprising the following steps:

[0046] S1: Prepare grammatical standards for variable derivation methods, standardize variable derivation rules from previous clinical research data based on the grammatical standards, and build a variable derivation rule library;

[0047] Based on the variable derivation method of previous clinical research data and combined with the experience of previous clinical research projects, a grammatical standard for the variable derivation method is compiled; according to the grammatical standard, the variable derivation rules of previous clinical studies are standardized, and a variable derivation rule library is constructed based on the standardized variable derivation rules.

[0048] The variable derivation rule base is established using EXCEL software, and the variable derivation rule base stores the standardized variable derivation rules.

[0049] The grammatical standards should take into account both logic and readability, for example:

[0050] Use "T" to connect RAW.CM.CMENDAT and RAW.CM.CMENTIM;

[0051] If RAW.PC.PCTPT="0h" or ARELTM>0, then ATPT=RAW.PC.PCTPT;

[0052] Get the date part of VSDTC and convert it to E8601DA10 format;

[0053] Get the part after ":" in RAW.DM.ETHNIC;

[0054] RAW.DS.DSSTDTC, when RAW.DS.DSDECOD="RANDOM".

[0055] S2: Building a variable derivation rule parsing model based on the variable derivation rule library, identifying categories of multi-dimensional features of the variable derivation rules, and establishing differentiated parsing rules for different feature categories;

[0056] The variable derivation rule parsing model is driven by natural language processing (NLP) logic and uses text processing functions or regular expression functions to identify categories of multi-dimensional features of variable derivation rules.

[0057] The categories of the multi-dimensional features of the variable derivation rules include grammatical structure, semantic logic and keywords;

[0058] Based on natural language processing (NLP) logic, regular expression matching and keyword recognition are performed on grammatical structures; and the nesting level of conditional statements is determined based on semantic logic.

[0059] S3: Generate the variable description file required for the user's clinical research project based on the variable derivation rule in the variable derivation rule library selected by the user, or write a new variable derivation rule based on the grammatical standard;

[0060] It's important to note that the variable description file describes the derivation rules for the variable. These rules include the attributes of the analysis dataset (name, structure, and key variables), the attributes of the variables within each dataset (name, type, length, and format), and the source of the variable (derivation method). For non-standard derivation rules that cannot be written using standard syntax, experienced personnel are required to iteratively analyze the model.

[0061] S4: extracting multi-dimensional features from the variable description file using the variable derivation rule parsing model, dynamically matching the parsing rules using a rule engine according to the categories of the multi-dimensional features, and calling a SAS statement library to generate a SAS statement for deriving variables;

[0062] For variables with simple derivation rules, the variable derivation rule parsing model can directly generate SAS statements based on the dynamically matched parsing rules.

[0063] For variables with complex derivation rules, a SAS statement library can be established and SAS statements can be generated by calling the SAS statement library.

[0064] The SAS statement library stores segmented SAS statements corresponding to the characteristics of the derivative method, which are written based on previous clinical research experience. The grammatical structure, semantic logic or keywords of the variable derivation rules in the variable description file are extracted through natural language processing (NLP). The grammatical structure, semantic logic or keywords are matched with the characteristics of the derivative method in the SAS statement library, and the variable derivation rules can be directly parsed into segmented SAS statements corresponding to the derivative method.

[0065] For example, for the BASE (baseline value) variable in the ADPE (physical examination analysis data set), its derivation method is "the last examination result before the trial treatment". The variable derivation rule parsing model can identify the analysis data set name "ADPE" and the variable derivation method "the last examination result before the trial treatment" and then generate the SAS statement for deriving the BASE variable as follows:

[0066] proc sort;by USUBJID PARAM PEDTC;run;

[0067] data base;

[0068] set ;

[0069] by USUBJID PARAM PEDTC;

[0070] where ^missing(AVALC) and PEDTC<:put(TR01SDTM,E8601DT19.);

[0071] if last.PARAM;

[0072] if ACLSIG='' then ACLSIG='';

[0073] if ACLSIGN=. then ACLSIGN=.;

[0074] BASE=AVAL;BASEC=AVALC;BPEDTC=PEDTC;BCLSIG=ACLSIG;BCLSIGN=ACLSIGN;

[0075] keep USUBJID PARAMBPEDTC BASE BASEC BCLSIG BCLSIGN;

[0076] run;

[0077] It should be noted that the above SAS statements are only examples.

[0078] S5: Using a topological sorting algorithm, processing the directed acyclic dependency relationship between the variables in the variable description file according to the in-degree decay principle, outputting the variables with an in-degree of 0 in sequence, and dynamically inferring the optimal output order of the SAS statement.

[0079] S51: Construct directed acyclic dependency relationships between variables based on the derivation logic between variables in the variable description file;

[0080] The dependency relationships between the variables can be classified into the following: "directly derived from the original data set variables and no longer used to derive new variables", "directly derived from the original data set variables and will be used to derive new variables", "new variables derived from a single original data set and no longer used to derive new variables", "new variables derived from a single original data set and will be used to derive new variables", "new variables derived from multiple original data sets and no longer used to derive new variables", and "new variables derived from multiple original data sets and will be used to derive new variables".

[0081] S52: Calculate the in-degree of each variable, where the in-degree of the variable directly derived from the original dataset is 0;

[0082] The in-degree refers to the number of other new variables that the new variable depends on in the directed acyclic dependency relationship.

[0083] For other variables, the PRXCHANGE function is used to delete the variables that have generated SAS statements; the PRXCHANGE function is used to identify the variables that have not generated SAS statements and are not repeated, and then the COUNT function is used to count the number of variables with dependent relationships, which is the in-degree.

[0084] S53: Based on the in-degree decay principle, the order of outputting variables with in-degree 0 is used as the optimal output order of the SAS statements. Specifically, the SAS statements corresponding to the variables with in-degree 0 are output first, and the dependencies of the outputted variables are released. The number of released dependencies is subtracted from the in-degrees of the remaining variables, and the SAS statements corresponding to the variables with in-degree 0 are output again. The above steps are repeated until all variables are output.

[0085] This example uses some variables to be generated in the analysis dataset ADSL (analysis dataset at the subject level) as an example to demonstrate the process of intelligently inferring the optimal output sequence of SAS statements using a dynamic topological sorting algorithm.

[0086] As shown in Table 1, the initial in-degree values of some variables in the analysis data set ADSL are:

[0087] Table 1 Some variables in ADSL

[0088]

[0089] It should be noted that variables in the original dataset are uniquely identified by the "original database name (RAW)", "original dataset name in the original database", and "variable name in the original dataset", connected by a period (."). In Table 1, DM represents the "demographic characteristics" variable in the original dataset, and VS represents the "vital signs" variable in the original dataset. For example, RAW.VS.VSORRES represents the VSORRES (vital sign measurement outcome) variable in the "vital signs dataset" in the "original database".

[0090] First, output the variables with an in-degree of 0, that is, the SAS statements corresponding to SUBJID, ARM, SEX, HEIGHTC, and WEIGHTC, and remove the dependencies of the above-output variables. Then, subtract the number of removed dependencies from the in-degree of the remaining variables to obtain Table 2:

[0091] Table 2. Remaining variables in ADSL after generating the variables with in-degree 0 in Table 1.

[0092]

[0093] Output the SAS statements corresponding to the variables with an in-degree of 0 again, and release the dependencies of the output variables; subtract the number of released dependencies from the in-degrees of the remaining variables to obtain Table 3, and finally output the variable BMI.

[0094] Table 3. Remaining variables in ADSL after generating the variables with in-degree 0 in Table 2.

[0095]

[0096] S6: obtaining a standardized program file of the analysis data set based on the optimal output sequence output;

[0097] According to the optimal output order obtained by the topological sorting algorithm, the code is reconstructed in an ordered execution sequence, and standardized program files of analysis data sets that comply with CDISC standards are output through standardized code packaging technology.

[0098] S7: calling the analysis data set standardization program file to directly generate a standardized analysis data set.

[0099] It should be noted that the programming language of the present invention is not limited to VBA and SAS, and is also applicable to programming languages such as R, Python, and Java.

[0100] Example 2:

[0101] The present invention further includes an electronic device, comprising:

[0102] a memory storing executable instructions;

[0103] A processor runs the executable instructions in the memory to implement the above-mentioned method for automating the standardization procedure of clinical research analysis data sets.

[0104] Example 3:

[0105] The present invention also includes a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the above-mentioned method for automating the standardization procedure of clinical research analysis data sets.

[0106] The above description is merely a preferred embodiment of the present invention. The scope of protection of the present invention is not limited to the above embodiment. All technical solutions based on the concept of the present invention are within the scope of protection of the present invention. It should be noted that for those skilled in the art, various improvements and modifications that do not depart from the principles of the present invention should also be considered within the scope of protection of the present invention.

Claims

1. A method for automating a standardization process for a clinical research analysis dataset, characterized in that: The following steps are involved: S1: Prepare grammatical standards for variable derivation methods, standardize variable derivation rules of previous clinical research data according to the grammatical standards, and build a variable derivation rule library; S2: Identify the multi-dimensional features of the variable derivation rules in the variable derivation rule library, set parsing rules according to the different multi-dimensional features, and establish a variable derivation rule parsing model; the variable derivation rule parsing model is driven by natural language processing logic and uses text processing functions or regular expression functions to identify the categories of the multi-dimensional features of the variable derivation rules; S3: Generate the variable description file required for the user's clinical research project based on the variable derivation rule in the variable derivation rule library selected by the user, or write a new variable derivation rule based on the grammatical standard; S4: extracting multi-dimensional features from the variable description file using the variable derivation rule parsing model, dynamically matching the parsing rules using the variable derivation rule parsing model according to the categories of the multi-dimensional features, and generating a SAS statement for deriving variables; S5: Using a topological sorting algorithm, processing the directed acyclic dependency relationship between the variables in the variable description file according to the in-degree decay principle, outputting the variables with in-degree 0 in sequence, and inferring the optimal output order of the SAS statement; S51: Construct directed acyclic dependency relationships between variables based on the derivation logic between variables in the variable description file; S52: Calculate the in-degree of each variable, where the in-degree of the variable directly derived from the original dataset is 0; For other variables, use the PRXCHANGE function to delete the variables that have generated SAS statements; use the PRXCHANGE function to identify the variables that have not generated SAS statements and are not repeated, and then use the COUNT function to count the number of variables with dependent relationships, which is the in-degree; S53: Based on the in-degree decay principle, the order of outputting variables with in-degree 0 is used as the optimal output order of the SAS statement; S6: obtaining a standardized program file of the analysis data set based on the optimal output sequence output; S7: calling the analysis data set standardization program file to generate a standardized analysis data set.

2. The method for automating the standardization process of clinical research analysis data sets according to claim 1, characterized in that: Specifically, step S1 comprises compiling a grammatical standard for the variable derivation method based on the variable derivation method of previous clinical research data and combining the experience of previous clinical research projects; standardizing the variable derivation rules of previous clinical research according to the grammatical standard, and constructing a variable derivation rule library based on the standardized variable derivation rules.

3. The method for automating the standardization process of clinical research analysis data sets according to claim 2, characterized in that: The variable derivation rule base is established using EXCEL software, and the variable derivation rule base stores the standardized variable derivation rules.

4. The method for automating the standardization process of clinical research analysis data sets according to claim 1, characterized in that: The categories of the multi-dimensional features include grammatical structure, semantic logic and keywords; Regular expression matching and keyword recognition of grammatical structures based on natural language processing logic; Determine the nesting level of conditional statements in semantic logic.

5. The method for automating the standardization process of clinical research analysis data sets according to claim 4, characterized in that: The text processing functions include SCAN, KSCAN, SUBSTR, LENGTH, KLENGTH, LENGTHN, LENGTHC, LENGTHM, UPCASE, LOWCASE, PROPCASE, COMPBL, COMPRESS, KCOMPRESS, STRIP, INDEX, FIND, COUNT and TRANWRD; The regular expression functions include PRXMATCH and PRXCHANGE.

6. The method for automating the standardization process of clinical research analysis data sets according to claim 1, characterized in that: The method for automating the standardization process of clinical research analysis data sets further includes establishing a SAS statement library; the SAS statement library stores segmented SAS statements corresponding to the characteristics of the derivative method written based on previous clinical research experience; The variable derivation rule parsing model can directly generate SAS statements according to the dynamically matched parsing rules, or generate SAS statements by calling a SAS statement library.

7. The method for automating the standardization process of clinical research analysis data sets according to claim 1, characterized in that: Specifically, step S53 first outputs the SAS statements corresponding to the variables with an in-degree of 0 and releases the dependencies of the output variables; subtracts the number of released dependencies from the in-degrees of the remaining variables and outputs the SAS statements corresponding to the variables with an in-degree of 0 again; and repeats the above steps until all variables are output.

8. The method for automating the standardization process of clinical research analysis data sets according to claim 1, characterized in that: Step S6 specifically involves reconstructing the code in an ordered execution sequence according to the optimal output order obtained by the topological sorting algorithm, and outputting a standardized program file of the analysis data set that meets the preset standards through standardized code encapsulation technology.

9. An electronic device, characterized in that: The electronic device comprises: a memory storing executable instructions; A processor, wherein the processor runs the executable instructions in the memory to implement the method for automating the standardization procedure of clinical research analysis data sets according to any one of claims 1 to 8.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, which, when executed by a processor, implements the method for automating the standardization procedure of a clinical research analysis data set according to any one of claims 1 to 8.

Images