Clinical research analysis data set standardization program automation method, equipment and medium

Abstract

The invention relates to the field of data management, in particular to a clinical research and analysis data set standardization program automation method and device and a medium. The method comprises the following steps: compiling a grammar standard of a variable derivation method, and constructing a variable derivation rule base; according to the method, multi-dimensional features of variable derivation rules in a variable description file of a user are recognized by constructing a variable derivation rule analysis model, analysis rules are dynamically matched, and then SAS statements used for deriving variables are automatically generated; the method comprises the following steps: judging a directed acyclic dependency relationship among variables on the basis of derivative logic of the variables in a variable description file of a user, deducing an optimal output sequence of SAS statements by adopting a dynamic topological sorting algorithm, outputting an analysis data set standardization program file, and calling the file to generate a standardized analysis data set. Compared with traditional manual writing, the method has the advantages that the analysis data set standardization program is automatically generated, the requirement for SAS programming experience is lowered, and working efficiency and quality are improved.

Description

Technical Field

[0001] The present invention relates to the field of data governance technology, and specifically to an automated method, device and medium for standardizing clinical research and analysis data sets. Background Art

[0002] SAS (Statistical Analysis System) is a widely used statistical analysis software and a programming language, mainly used in data processing, statistical analysis, business intelligence, data mining and other fields.

[0003] In the field of clinical research, including registered clinical research and non-registered clinical research, accurate and clearly structured analytical data sets are a prerequisite for accurately and efficiently outputting statistical analysis results. However, under normal circumstances, an experienced SAS programmer is required to write a set of analytical data set description files based on the data structure of the original data set, the information in the annotated case report form and the statistical analysis table template; and two SAS programmers with rich programming experience and proficient programming skills are required to manually write two sets of programs according to the requirements of this set of analytical data set description files, standardize the same set of original data sets, and then generate two completely consistent standardized analytical data sets.

[0004] The analysis data set description file is equivalent to a programming plan, which consists of the attributes of the analysis data set (name, structure, key variables, etc.), the attributes of the variables in each data set (name, type, length, format, etc.), and the source of the variables (derivation method) (this information together constitutes the variable derivation rules of the analysis data set). Usually, the first draft is completed before the program is written, and it needs to be continuously modified according to the actual data during the program writing process until it is perfect. This causes the two programmers to frequently modify their respective programs based on the modified analysis data set description file. At the same time, due to poor project communication and other reasons, the program written by the programmer may be inconsistent with the requirements of the analysis data set description file.

[0005] In summary, the existing analysis data set standardization process requires highly skilled programmers and takes a lot of time and effort, with high thresholds for writing and low efficiency. Therefore, there is an urgent need for an automated method, device and medium for the standardization of clinical research analysis data sets. Summary of the invention

[0006] In order to avoid the above-mentioned problems existing in the prior art, the purpose of the present invention is to provide a method, device and medium for automating the standardization procedure of clinical research analysis data sets, which can use SAS program to read and identify the natural language in the variable derivation rules, parse the variable derivation rules into SAS language; and automatically adjust the order of SAS statements according to the characteristics of the variable derivation rules, and finally output a SAS program that can generate an analysis data set.

[0007] To achieve the above object, the present invention provides the following technical solution: a method for automating the standardization procedure of clinical research analysis data sets, comprising the following steps: S1: Prepare the syntax standard of variable derivation method, standardize the variable derivation rules of previous clinical research data according to the syntax standard, and build a variable derivation rule library; S2: constructing a variable derivation rule parsing model based on the variable derivation rule library, identifying the categories of multi-dimensional features of the variable derivation rules, and establishing differentiated parsing rules for different feature categories; S3: Generate a variable description file required for the user's clinical research project according to the variable derivation rule in the variable derivation rule library selected by the user, or write a new variable derivation rule based on the grammatical standard; S4: extracting multi-dimensional features in the variable description file by the variable derivation rule parsing model, dynamically matching the parsing rules using a rule engine according to the categories of the multi-dimensional features, and calling a SAS statement library to generate a SAS statement for deriving variables; S5: using a topological sorting algorithm, processing the directed acyclic dependency relationship between the variables in the variable description file according to the in-degree decay principle, outputting the variables with in-degree 0 in sequence, and dynamically inferring the optimal output order of the SAS statement; S6: obtaining a standardized program file of the analysis data set based on the optimal output sequence output; S7: calling the analysis data set standardization program file to directly generate a standardized analysis data set.

[0008] The present invention is further configured such that step S1 specifically comprises: compiling a grammatical standard for the variable derivation method based on the variable derivation method of previous clinical research data and combining the experience of previous clinical research projects; standardizing the variable derivation rules of previous clinical research according to the grammatical standard, and constructing a variable derivation rule library based on the standardized variable derivation rules.

[0009] The present invention is further configured to use EXCEL software to establish the variable derivation rule base, and use EXCEL software to establish the variable derivation rule base, wherein the variable derivation rule base stores the standardized variable derivation rules.

[0010] The present invention is further configured such that, in step S2, the variable derivation rule parsing model is driven based on natural language processing (NLP) logic, and uses a text processing function or a regular expression function to identify the categories of the multi-dimensional features of the variable derivation rules.

[0011] The present invention is further configured such that the categories of the multi-dimensional features of the variable derivation rules include grammatical structure, semantic logic and keywords; Based on natural language processing (NLP) logic, regular expression matching and keyword recognition are performed on the grammatical structure; the nesting level of conditional statements is determined for the semantic logic.

[0012] The present invention is further configured such that the text processing functions include SCAN, KSCAN, SUBSTR, LENGTH, KLENGTH, LENGTHN, LENGTHC, LENGTHM, UPCASE, LOWCASE, PROPCASE, COMPBL, COMPRESS, KCOMPRESS, STRIP, INDEX, FIND, COUNT and TRANWRD; The regular expression functions include PRXMATCH and PRXCHANGE.

[0013] The present invention is further configured that the method for automating the standardization procedure of clinical research analysis data sets further includes establishing a SAS statement library, wherein the SAS statement library stores segmented SAS statements corresponding to the characteristics of the derivative method written based on previous clinical research experience; The variable derivation rule parsing model can directly generate SAS statements according to the dynamically matched parsing rules, or generate SAS statements by calling a SAS statement library.

[0014] The present invention is further configured that step S5 specifically includes the following steps: S51: constructing directed acyclic dependency relationships between variables according to the derivative logic between variables in the variable description file; S52: Calculate the in-degree of each variable, where the in-degree of the variable directly derived from the original data set is 0; For other variables, the PRXCHANGE function is used to delete the variables that have generated SAS statements; the PRXCHANGE function is used to identify the variables that have not generated SAS statements and are not repeated, and then the COUNT function is used to count the number of variables with dependent relationships, which is the in-degree; S53: Based on the in-degree decay principle, the order of outputting variables with in-degree 0 in sequence is taken as the optimal output order of the SAS statement.

[0015] The present invention is further configured such that step S53 specifically includes first outputting the SAS statements corresponding to the variables with an in-degree of 0, and releasing the dependencies of the outputted variables; subtracting the number of released dependencies from the in-degrees of the remaining variables, and outputting the SAS statements corresponding to the variables with an in-degree of 0 again; and repeating the above steps until all variables are outputted.

[0016] The present invention is further configured such that step S5 is specifically, and step S6 is specifically, according to the optimal output order obtained by the topological sorting algorithm, the ordered execution sequence is used for code reconstruction, and a standardized program file of the analysis data set that meets the preset standard is output through the standardized code encapsulation technology.

[0017] The present invention also includes an electronic device, the electronic device comprising: A memory storing executable instructions; A processor, wherein the processor runs the executable instructions in the memory to implement the above-mentioned method for automating the standardization procedure of clinical research analysis data sets.

[0018] The present invention also includes a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the above-mentioned method for automating the standardization procedure of clinical research analysis data sets.

[0019] The present invention collects and organizes the variable derivation rules described in the analysis data set description files used in previous clinical studies, and constructs a variable derivation rule library; uses natural language processing logic, adopts a SAS program to read and recognize the natural language in the variable derivation rules, and parses the variable derivation rules into SAS language; and automatically adjusts the order of SAS statements according to the characteristics of the variable derivation rules, and finally outputs a SAS program that can generate an analysis data set.

[0020] In summary, the beneficial effects of the technical solution of the present invention are as follows: (1) The present invention provides an automated method for standardizing clinical research analysis data sets, which lowers the threshold for writing analysis data set standardization programs and improves work efficiency. Users do not need to master SAS, R or other programming languages. They only need to select the variables and their corresponding derivative rules required for the clinical research project from the variable derivation rule library to export the analysis data set description file of the project; at the same time, they only need to fill in the special information of the new project in the analysis data set description file using appropriate and standardized natural language to quickly complete the analysis data set standardization program and the standardized analysis data set. The minimum work experience requirement for users is reduced from 2 years to 2 months; manual programming is changed to automatic programming, which improves programming efficiency by 90%. At the same time, new variable derivation rules written according to the grammatical standards of the present invention can be added to the variable derivation rule library for use in subsequent projects, thereby improving the marginal benefits of writing new variable derivation rules.

[0021] (2) The present invention implements automatic output of the program by parsing the analysis data set description file, avoiding errors that may be caused by manual writing, ensuring that the program meets the requirements of the analysis data set description file, and improving the work quality. At the same time, based on the determined parsing rules, the present invention avoids the risk of uncontrollable results brought about by the use of deep learning black box models, and can meet the high requirements for data quality in the field of clinical research.

[0022] (3) Wide scope of application. The present invention is not only applicable to generating SDTM and AdaM data sets and their programs specified in the CDISC standard, but also applicable to generating non-CDISC standard analysis data sets and their programs. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for describing the embodiments are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0024] Figure 1 This is a flow chart of the automated method for standardizing a clinical research analysis data set according to an embodiment of the present invention. DETAILED DESCRIPTION

[0025] In order to enable those skilled in the art to better understand the technical solution of the present invention, the technical solution of the present invention is clearly and completely described below in conjunction with the accompanying drawings of the present invention. Based on the embodiments of the present invention, other similar embodiments obtained by ordinary technicians in the field without making any creative work should all fall within the scope of protection of the present invention.

[0026] The present invention will be further described below in conjunction with the accompanying drawings and preferred embodiments.

[0027] Embodiment 1: like Figure 1 As shown in the figure, a preferred embodiment of the present invention is a method for automating the standardization procedure of clinical research analysis data sets, comprising the following steps: S1: Prepare the syntax standard of variable derivation method, standardize the variable derivation rules of previous clinical research data according to the syntax standard, and build a variable derivation rule library; Based on the variable derivation method of previous clinical research data and combined with the experience of previous clinical research projects, a grammatical standard for the variable derivation method is compiled; the variable derivation rules of previous clinical research are standardized according to the grammatical standard, and a variable derivation rule library is constructed based on the standardized variable derivation rules.

[0028] The variable derivation rule base is established using EXCEL software, and the variable derivation rule base stores the standardized variable derivation rules.

[0029] The grammatical standards should take into account both logic and readability, for example: Use "T" to connect RAW.CM.CMENDAT and RAW.CM.CMENTIM; If RAW.PC.PCTPT="0h" or ARELTM>0, then ATPT=RAW.PC.PCTPT; Get the date part of VSDTC and convert it to E8601DA10. format; Get the part after ":" in RAW.DM.ETHNIC; RAW.DS.DSSTDTC, when RAW.DS.DSDECOD="RANDOM".

[0030] S2: constructing a variable derivation rule parsing model based on the variable derivation rule library, identifying the categories of multi-dimensional features of the variable derivation rules, and establishing differentiated parsing rules for different feature categories; The variable derivation rule parsing model is driven by natural language processing (NLP) logic and uses text processing functions or regular expression functions to identify categories of multi-dimensional features of variable derivation rules.

[0031] The categories of the multi-dimensional features of the variable derivation rules include grammatical structure, semantic logic and keywords; Based on natural language processing (NLP) logic, regular expression matching and keyword recognition are performed on the grammatical structure; the nesting level of conditional statements is determined for the semantic logic.

[0032] S3: Generate a variable description file required for the user's clinical research project according to the variable derivation rule in the variable derivation rule library selected by the user, or write a new variable derivation rule based on the grammatical standard; It should be noted that the variable description file describes the derivation rules of the variables, which include the attributes of the analysis data set (name, structure, key variables, etc.), the attributes of the variables in each data set (name, type, length, format, etc.), and the source of the variables (derivation method). For non-standard derivation rules that cannot be written in a standard syntax, experienced personnel are required to iteratively parse the model.

[0033] S4: extracting multi-dimensional features in the variable description file by the variable derivation rule parsing model, dynamically matching the parsing rules using a rule engine according to the categories of the multi-dimensional features, and calling a SAS statement library to generate a SAS statement for deriving variables; For variables with simple derivation rules, the variable derivation rule parsing model can directly generate SAS statements based on the dynamically matched parsing rules.

[0034] For variables with complex derivation rules, a SAS statement library can be established to generate SAS statements by calling the SAS statement library.

[0035] The SAS statement library stores segmented SAS statements corresponding to the characteristics of the derivative method written based on previous clinical research experience. The grammatical structure, semantic logic or keywords of the variable derivation rules in the variable description file are extracted through natural language processing (NLP), and the grammatical structure, semantic logic or keywords are matched with the characteristics of the derivative method in the SAS statement library. The variable derivation rules can be directly parsed into segmented SAS statements corresponding to the derivative method.

[0036] For example, for the BASE (baseline value) variable in ADPE (physical examination analysis data set), its derivation method is "the last examination result before the trial treatment". The variable derivation rule parsing model can generate the SAS statement for deriving the BASE variable by identifying the analysis data set name "ADPE" and the variable derivation method "the last examination result before the trial treatment" as follows: proc sort;by USUBJID PARAM PEDTC;run; data base; set ; by USUBJID PARAM PEDTC; where ^missing(AVALC) and PEDTC<:put(TR01SDTM,E8601DT19.); if last.PARAM; if ACLSIG='' then ACLSIG=''; if ACLSIGN=. then ACLSIGN=.; BASE=AVAL;BASEC=AVALC;BPEDTC=PEDTC;BCLSIG=ACLSIG;BCLSIGN=ACLSIGN; keep USUBJID PARAMBPEDTC BASE BASEC BCLSIG BCLSIGN; run; It should be noted that the above SAS statements are only examples.

[0037] S5: A topological sorting algorithm is used to process the directed acyclic dependency relationship between the variables in the variable description file according to the in-degree decay principle, and the variables with an in-degree of 0 are output in sequence, and the optimal output order of the SAS statement is dynamically inferred.

[0038] S51: constructing directed acyclic dependency relationships between variables according to the derivative logic between variables in the variable description file; The dependency relationships between the variables can be classified into the following dependencies: "directly derived from the variables of the original data set and no longer used to derive new variables", "directly derived from the variables of the original data set and will be used to derive new variables", "new variables derived from a single original data set and no longer used to derive new variables", "new variables derived from a single original data set and will be used to derive new variables", "new variables derived from multiple original data sets and no longer used to derive new variables", and "new variables derived from multiple original data sets and will be used to derive new variables".

[0039] S52: Calculate the in-degree of each variable, where the in-degree of the variable directly derived from the original data set is 0; The in-degree refers to the number of other new variables that the new variable depends on in the directed acyclic dependency relationship.

[0040] For other variables, the PRXCHANGE function is used to delete the variables that have generated SAS statements; the PRXCHANGE function is used to identify the variables that have not generated SAS statements and are not repeated, and then the COUNT function is used to count the number of variables with dependent relationships, which is the in-degree.

[0041] S53: Based on the in-degree decay principle, the order of outputting variables with in-degree 0 is used as the optimal output order of SAS statements. That is, first output the SAS statements corresponding to the variables with in-degree 0, and release the dependencies of the output variables; subtract the number of released dependencies from the in-degrees of the remaining variables, and output the SAS statements corresponding to the variables with in-degree 0 again; repeat the above steps until all variables are output.

[0042] This embodiment takes some variables to be generated in the analysis data set ADSL (analysis data set at the subject level) as an example to demonstrate the process of intelligently inferring the optimal output sequence of SAS statements using a dynamic topological sorting algorithm.

[0043] As shown in Table 1, the initial in-degree values ​​of some variables in the analysis data set ADSL are: Table 1 Some variables in ADSL

[0044] It should be noted that the variables in the original dataset are uniquely identified by the "original database name (RAW)", "original dataset name in the original database" and "variable name in the original dataset", which are connected by a dot ".". In Table 1, DM represents the "demographic characteristics" of the original dataset, and VS represents the "vital signs" of the original dataset. For example, RAW.VS.VSORRES represents the VSORRES (vital signs measurement results) variable in the "vital signs dataset" in the "original database".

[0045] First, output the variables with an in-degree of 0, that is, the SAS statements corresponding to SUBJID, ARM, SEX, HEIGHTC, and WEIGHTC, and release the dependencies of the above-mentioned output variables. Then, subtract the number of released dependencies from the in-degree of the remaining variables to obtain Table 2: Table 2 The remaining variables in ADSL after generating the variables with in-degree 0 in Table 1

[0046] Output the SAS statements corresponding to the variables with an in-degree of 0 again, and release the dependencies of the output variables; subtract the number of released dependencies from the in-degrees of the remaining variables to obtain Table 3, and finally output the variable BMI.

[0047] Table 3 The remaining variables in ADSL after generating the variables with in-degree 0 in Table 2

[0048] S6: obtaining a standardized program file of the analysis data set based on the optimal output sequence output; According to the optimal output order obtained by the topological sorting algorithm, the ordered execution sequence is reconstructed for code, and standardized program files of analysis data sets that meet CDISC standards are output through standardized code packaging technology.

[0049] S7: calling the analysis data set standardization program file to directly generate a standardized analysis data set.

[0050] It should be noted that the programming language of the present invention is not limited to VBA and SAS, and is also applicable to programming languages ​​such as R, Python, and Java.

[0051] Embodiment 2: The present invention also includes an electronic device, the electronic device comprising: A memory storing executable instructions; A processor, wherein the processor runs the executable instructions in the memory to implement the above-mentioned method for automating the standardization procedure of clinical research analysis data sets.

[0052] Embodiment 3: The present invention also includes a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the above-mentioned method for automating the standardization procedure of clinical research analysis data sets.

[0053] The above is only a preferred embodiment of the present invention, and the protection scope of the present invention is not limited to the above embodiments. All technical solutions under the concept of the present invention belong to the protection scope of the present invention. It should be pointed out that for ordinary technicians in this technical field, some improvements and modifications without departing from the principle of the present invention should also be regarded as the protection scope of the present invention.

Claims

1. A method for automating a procedure for standardizing a clinical research analysis dataset, characterized in that: The following steps are involved: S1: Prepare the syntax standard of variable derivation method, standardize the variable derivation rules of previous clinical research data according to the syntax standard, and build a variable derivation rule library; S2: identifying the multi-dimensional features of the variable derivation rules in the variable derivation rule base, setting parsing rules according to the multi-dimensional features, and establishing a variable derivation rule parsing model; S3: Generate a variable description file required for the user's clinical research project according to the variable derivation rule in the variable derivation rule library selected by the user, or write a new variable derivation rule based on the grammatical standard; S4: extracting multi-dimensional features in the variable description file by the variable derivation rule parsing model, dynamically matching the parsing rules by using the variable derivation rule parsing model according to the categories of the multi-dimensional features, and generating a SAS statement for deriving variables; S5: using a topological sorting algorithm, processing the directed acyclic dependency relationship between the variables in the variable description file according to the in-degree decay principle, outputting the variables with in-degree 0 in sequence, and inferring the optimal output order of the SAS statement; S6: obtaining a standardized program file of the analysis data set based on the optimal output sequence output; S7: calling the analysis data set standardization program file to generate a standardized analysis data set.

2. The method for automating the standardization procedure of clinical research analysis data sets according to claim 1, characterized in that: Step S1 specifically includes: compiling a grammatical standard for the variable derivation method based on the variable derivation method of previous clinical research data and combining the experience of previous clinical research projects; standardizing the variable derivation rules of previous clinical research according to the grammatical standard, and constructing a variable derivation rule library based on the standardized variable derivation rules.

3. The method for automating the standardization procedure of clinical research analysis data sets according to claim 2, characterized in that: The variable derivation rule base is established using EXCEL software, and the variable derivation rule base stores the standardized variable derivation rules.

4. The method for automating the standardization procedure of clinical research analysis data sets according to claim 1, characterized in that: In step S2, the variable derivation rule parsing model is driven based on natural language processing logic and uses text processing functions or regular expression functions to identify the categories of multi-dimensional features of variable derivation rules.

5. The method for automating the standardization procedure of clinical research analysis data sets according to claim 4, characterized in that: The categories of the multi-dimensional features include grammatical structure, semantic logic and keywords; Regular expression matching and keyword recognition of grammatical structures based on natural language processing logic; Determine the nesting level of conditional statements in semantic logic.

6. The method for automating the standardization procedure of clinical research analysis data sets according to claim 5, characterized in that: The text processing functions include SCAN, KSCAN, SUBSTR, LENGTH, KLENGTH, LENGTHN, LENGTHC, LENGTHM, UPCASE, LOWCASE, PROPCASE, COMPBL, COMPRESS, KCOMPRESS, STRIP, INDEX, FIND, COUNT and TRANWRD; The regular expression functions include PRXMATCH and PRXCHANGE.

7. The method for automating the standardization procedure of clinical research analysis data sets according to claim 1, characterized in that: The method for automating the standardization procedure of clinical research analysis data sets also includes establishing a SAS statement library; the SAS statement library stores segmented SAS statements corresponding to the characteristics of the derivative method written based on previous clinical research experience; The variable derivation rule parsing model can directly generate SAS statements according to the dynamically matched parsing rules, or generate SAS statements by calling a SAS statement library.

8. The method for automating the standardization procedure of clinical research analysis data sets according to claim 1, characterized in that: Step S5 specifically includes the following steps: S51: constructing directed acyclic dependency relationships between variables according to the derivative logic between variables in the variable description file; S52: Calculate the in-degree of each variable, where the in-degree of the variable directly derived from the original data set is 0; Other variables use the PRXCHANGE function to delete the variables that have generated SAS statements; use the PRXCHANGE function to identify the variables that have not generated SAS statements and are not repeated, and then use the COUNT function to count the number of variables with dependent relationships, which is the in-degree; S53: Based on the in-degree decay principle, the order of outputting variables with in-degree 0 in sequence is taken as the optimal output order of the SAS statement.

9. The method for automating the standardization procedure of clinical research analysis data sets according to claim 8, characterized in that: Step S53 specifically includes: first outputting the SAS statements corresponding to the variables with an in-degree of 0, and releasing the dependencies of the outputted variables; subtracting the number of released dependencies from the in-degrees of the remaining variables, and outputting the SAS statements corresponding to the variables with an in-degree of 0 again; and repeating the above steps until all variables are outputted.

10. The method for automating the standardization procedure of clinical research analysis data sets according to claim 1, characterized in that: Step S6 specifically includes, according to the optimal output order obtained by the topological sorting algorithm, ordering the execution sequence to reconstruct the code, and outputting a standardized program file of the analysis data set that meets the preset standard through standardized code packaging technology.

11. An electronic device, characterized in that: The electronic device comprises: A memory storing executable instructions; A processor, wherein the processor runs the executable instructions in the memory to implement the method for automating the clinical research analysis data set standardization procedure according to any one of claims 1-10.

12. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, which, when executed by a processor, implements the method for automating the standardization procedure of a clinical research analysis data set according to any one of claims 1 to 10.

Images