Methods and systems for automatically generating logical checks when importing data verification plans.
By automatically parsing and matching data verification plan documents to generate logical verification, the high cost of manual data verification by database developers is solved, achieving efficient and accurate data verification.
Patent Information
- Application Number
- CN202211393636.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-11-08
- Publication Date
- 2025-11-14
- Estimated Expiration
- 2042-11-08
AI Technical Summary
In existing technologies, database developers need to manually implement data verification plans into the electronic data acquisition system, resulting in a large workload and high costs.
By reading the data verification plan file, parsing the table column headers, using string editing distance to match the table column headers, parsing the logical text into a multi-branch tree structure, determining the target location and calculation relationship, and generating logical verification, manual intervention is reduced.
It enables the automatic generation of logical verification plans for data verification, reducing the workload of database developers and improving the efficiency and accuracy of data verification.
Smart Images

Figure CN115641092B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of data quality assurance in clinical medical software, and specifically to a method and system for automatically generating logical checks by importing a Data Validation Plan (DVP). Background Technology
[0002] In clinical trials, on the Electronic Data Capture (EDC) system, the Database Developer (DBD) needs to implement data verification rules based on the Data Verification Plan (DVP) provided by the Data Quality Manager (DM). The DVP is typically described using tabular text. Data verification in the EDC system is usually implemented by writing code. Currently, the entire implementation of data verification from the DVP to the EDC is done manually by the DBD.
[0003] The existing method of manual DBD processing described above greatly increases the workload of DBD in implementing data verification plans, resulting in high processing costs. Summary of the Invention
[0004] The following provides a brief overview of one or more aspects to offer a basic understanding of them. This overview is not an exhaustive summary of all conceived aspects, nor is it intended to identify key or decisive elements of all aspects, nor to define the scope of any or all aspects. Its sole purpose is to present some concepts of one or more aspects in a simplified form to prepare for the more detailed descriptions that follow.
[0005] The purpose of this invention is to solve the above-mentioned problems and provide a method and system for automatically generating logical checks when importing data verification plans. This allows the system to directly understand the content in the data verification plan, thereby reducing the workload of DBD in implementing the data verification plan.
[0006] The technical solution of this invention is as follows: This invention discloses a method for automatically generating logical checks when importing data verification plans, the method comprising:
[0007] Step 1: Read the data verification plan file and parse the tables within it;
[0008] Step 2: Match table column headers based on name similarity;
[0009] Step 3: Based on the matched table column headers, parse each row in the data verification plan file to form structured information;
[0010] Step 4: Parse the case report table fields in the logical text;
[0011] Step 5: Parse the computational description in the logical text;
[0012] Step 6: Parse the object at the target location. By analyzing the location of the target data field and the location of the data field required in the calculation, determine whether the data location matches.
[0013] Step 7: Determine the confidence level and generate logical checks based on the confidence level.
[0014] According to an embodiment of the method for automatically generating logical checks by importing data verification plans according to the present invention, step 2 further includes:
[0015] By performing name matching between the table column headers parsed from the data verification plan file and the built-in system-recognizable table column headers, and using string edit distance as the approximation evaluation, the most similar column headers are matched.
[0016] According to an embodiment of the method for automatically generating logical checks by importing data verification plans according to the present invention, step 4 further includes:
[0017] Iterate through all data fields in the case report table. First, use the name as the representation of the data field and match the data field in the logical text. Second, perform fuzzy matching of the data field representation in the logical text. Finally, mark all matched substrings as the corresponding fields.
[0018] According to an embodiment of the method for automatically generating logical checks by importing data verification plans according to the present invention, step 5 further includes:
[0019] The logical text is parsed using the definition of a context-free grammar, transforming it into a forest structure of multi-branch trees. Trees that satisfy the grammar definition are retained, while those that do not are discarded. Finally, the tree structures that satisfy the grammar definition are collected, and a depth-first traversal is performed to convert the tree structures into operational expressions.
[0020] According to an embodiment of the method for automatically generating logical checks by importing data verification plans according to the present invention, step 6 further includes:
[0021] In the location analysis process, it is determined whether the calculation relationship between the target location object and the objects appearing in the logical text is satisfied. The determination rule is: whether the calculation of all objects corresponding to each target object matches the odd number of data points in the case report table.
[0022] According to an embodiment of the method for automatically generating logical checks by importing data verification plans according to the present invention, step 7 further includes:
[0023] After step 6 determines that the target location and the calculated field location satisfy the relationship, the confidence level is defined as the ratio of the abandoned tree size to the total tree size. When the confidence level is greater than the set threshold, the transformed expression is combined. The data field is a declarative logical rule or code, where the logical rule consists of a logical type, an expression, and a data point description. The logical type is determined in the form of the data verification plan. The expression and data point description are combined to generate a logical verification.
[0024] This invention also discloses a system for automatically generating logical checks when importing data verification plans. The system includes:
[0025] The reading module is used to read the data verification plan file and parse the tables within it;
[0026] The matching module is used to match table column headers based on name similarity;
[0027] The structured module is used to parse each row in the data verification plan file based on the matched table column headers to form structured information;
[0028] The field parsing module is used to parse the case report table fields in the logical text;
[0029] The computation description parsing module is used to parse computation descriptions in logical text.
[0030] The location relationship parsing module is used to parse objects at target locations. By analyzing the locations of target data fields and the locations of data fields required in the calculation, it determines whether the data locations match.
[0031] The logic check generation module is used to determine the confidence level and generate logic checks based on the confidence level.
[0032] According to an embodiment of the system for automatically generating logical checks by importing data verification plans according to the present invention, the matching module is further configured as follows:
[0033] By performing name matching between the table column headers parsed from the data verification plan file and the built-in system-recognizable table column headers, and using string edit distance as the approximation evaluation, the most similar column headers are matched.
[0034] According to an embodiment of the system for automatically generating logical checks when importing data verification plans according to the present invention, the field parsing module is further configured as follows:
[0035] Iterate through all data fields in the case report table. First, use the name as the representation of the data field and match the data field in the logical text. Second, perform fuzzy matching of the data field representation in the logical text. Finally, mark all matched substrings as the corresponding fields.
[0036] According to an embodiment of the system for automatically generating logical checks by importing data verification plans according to the present invention, the calculation description parsing module is further configured as follows:
[0037] The logical text is parsed using the definition of a context-free grammar, transforming it into a forest structure of multi-branch trees. Trees that satisfy the grammar definition are retained, while those that do not are discarded. Finally, the tree structures that satisfy the grammar definition are collected, and a depth-first traversal is performed to convert the tree structures into operational expressions.
[0038] According to an embodiment of the system for automatically generating logical checks by importing data verification plans according to the present invention, the positional relationship parsing module is further configured as follows:
[0039] In the location analysis process, it is determined whether the calculation relationship between the target location object and the objects appearing in the logical text is satisfied. The determination rule is: whether the calculation of all objects corresponding to each target object matches the odd number of data points in the case report table.
[0040] According to an embodiment of the system for automatically generating logical checks when importing data verification plans according to the present invention, the logical check generation module is further configured as follows:
[0041] Once the location relationship parsing module determines that the target location and the calculated field location satisfy the relationship, the confidence level is defined as the ratio of the abandoned tree size to the total tree size. When the confidence level is greater than the set threshold, the combined and transformed expression is used. The data field is a declarative logical rule or code, where the logical rule consists of a logical type, an expression, and a data point description. The logical type is determined in the form of the data verification plan. The expression and data point description are combined to generate a logical verification.
[0042] The present invention also discloses a computer-readable storage medium storing program instructions executable by a processor to implement the steps of the method for automatically generating logical checks by importing a data verification plan as described above.
[0043] The present invention also discloses a computer device comprising a memory, a processor, and program instructions stored in the memory that are executable by the processor, wherein the processor executes the program instructions to implement the steps of the method for automatically generating logical checks by importing a data verification plan as described above.
[0044] Compared with the prior art, the present invention has the following beneficial effects: the inventive points of the method of the present invention are as follows.
[0045] (1) Match the DVP table column headers based on the string edit distance approximation algorithm based on the table column headers identified by the built-in system, so as to avoid users manually configuring each column.
[0046] (2) Using the names of data points in CRF and SasName for precise and different degrees of fuzzy matching, data fields are matched in the calculation logic of user text description, thus solving the problem of matching data fields in free text.
[0047] (3) Computational expression matching based on context-free grammar solves the problem of confirming computational logic expressions in free text and makes the analysis results have a certain degree of credibility.
[0048] (4) By analyzing the location of the target data field and the location of the data field required in the calculation, the result of whether the data location matches can be obtained, thus solving some illegal or unparsable data verification expressions.
[0049] (5) By confirming the confidence level obtained through analysis of the computational logic text, logical checks are performed to avoid generating system analysis errors. Attached Figure Description
[0050] The above-described features and advantages of the present invention will be better understood after reading the following detailed description of embodiments of the present disclosure in conjunction with the accompanying drawings. In the drawings, components are not necessarily drawn to scale, and components having similar related characteristics or features may have the same or similar reference numerals.
[0051] Figure 1 A flowchart of an embodiment of the method for automatically generating logical checks by importing data verification plans according to the present invention is shown.
[0052] Figure 2 The diagram illustrates a schematic of an embodiment of the system for automatically generating logical checks when importing data verification plans according to the present invention. Detailed Implementation
[0053] The present invention will now be described in detail with reference to the accompanying drawings and specific embodiments. It should be noted that the aspects described below with reference to the accompanying drawings and specific embodiments are merely exemplary and should not be construed as limiting the scope of protection of the present invention in any way.
[0054] Figure 1 The flowchart illustrates an embodiment of the method for automatically generating logical checks when importing data verification plans according to the present invention. Please refer to... Figure 1 The implementation steps of the method in this embodiment are described in detail below.
[0055] Step 1: Read the data verification plan file and parse the tables within it.
[0056] Users upload the Data Verification Plan (DVP) file written by DM to the system, and the system stores and parses the file.
[0057] Step 2: Match table column headers based on name similarity.
[0058] The system includes built-in, recognizable table column headers, which are generated by the user for subsequent analysis. These headers may contain information such as target locations for data verification, challenge texts for failed data verification, and challenge logic describing verification calculations. The system matches the table column headers parsed from the data verification plan file with the built-in, system-recognizable headers by name, using string edit distance as the approximation criterion. The most similar header is then selected. Ideally, after automatic matching, the user can also manually adjust the matching process.
[0059] For example, each column header recognized by the built-in system has some commonly used names. If a column header means "text for opening a query," then its name might be "query text," "query message," etc. These commonly used names are used as candidates for column header names, and the most similar column header name is searched in the user's data verification plan file. The similarity here is based on the edit distance of the strings; the closer the edit distance, the more similar they are considered. If a column header has two candidates, and a column header in the user's file is 1 and 2 away from these two candidates, then the closest distance between this user's column header and this specified column header is 1. For each column header recognized by the system, the closest column header in the user's file is matched.
[0060] Step 3: Based on the matched table column headers, parse each row in the data verification plan file to form structured information.
[0061] Specifically, the table column header names match two aspects: visit name and field name. The text in these two columns is matched against the form structure defined by the CRF to determine the location of the target data field.
[0062] Step 4: Parse the CRF fields in the logical text. The logical text is a table header in the data verification plan document from Step 2. Each cell in the structured information under this header contains a text that describes the logic of a data calculation; this is called the logical text.
[0063] In step 4, all data fields in the Case Report Form (CRF) are traversed (Data Verification operates on the CRF; the Data Verification Plan (DVP) establishes rules to verify the accuracy of data in the CRF). First, the field name SasName (a standard character representation of data points in clinical data collection) is used as the field name, and the data fields in the logical text are matched. Next, the logical text is fuzzily matched to the representations of the data fields. Fuzzy matching here means that the field name SasName can differ from the substring in the logical text by one, two (or more) characters. Fuzzy matching stops at a difference of N characters. All matched substrings are marked as the corresponding fields for subsequent analysis.
[0064] In step 4, all relevant fields in the logical text are matched. Following the principle of getting as close as possible to the target object, the positions of these matched fields are determined.
[0065] For example, if the logical text is "Start date is less than end date", which mentions two fields "Start date" and "End date", and the target object is "Start date", and the view is "Filter", then the position of the "End date" field should be as close as possible to the target object, that is, under the "Filter" view, it should be the "End date" field in the same form as "Start date".
[0066] Step 5: Parse the computational description in the logical text.
[0067] The logical text is parsed using the definition of a context-free grammar, transforming it into a forest structure of multi-branch trees. The context-free grammar summarizes the sentence patterns in the logical text and defines the summation using a grammar; the definition itself is a recursive tree structure. Trees in the forest that conform to a specific structure (i.e., satisfy the grammar definition) are retained, while those that do not are discarded. Specifically, given a context-free grammar definition and a set of inputs, a sophisticated algorithm can be used to segment and combine the input into many tree structures. Tree structures that satisfy the grammar definition are retained, while others, such as those containing errors or not fully satisfying the grammar definition, are discarded. Finally, tree structures that satisfy the specific structure are collected and subjected to a depth-first traversal, converting the tree structures into operational expressions. The depth-first traversal process involves iterating through the child nodes of the current node from left to right, with each traversal resulting in a subexpression. Based on the properties of this node itself, these sub-expressions are concatenated into a larger expression and returned to the upper level (for example, concatenating all sub-expressions with an "OR" relationship). This process continues until the root node of the tree, after traversal, yields a complete expression.
[0068] Step 6: Parse the object at the target location. By analyzing the location of the target data field and the location of the data field required in the calculation, determine whether the data location matches.
[0069] The target location object and the target data field position describe the location of the target object and the object itself. Furthermore, the positions of the data fields required for the calculation are obtained in step 4. In the location analysis process, it is determined whether the calculation relationship between the target location object and the objects appearing in the logical text is satisfied. The determination rule is: whether all calculated objects corresponding to each target object match a single data point in the CRF.
[0070] Step 7: Determine the confidence level and generate logical checks based on the confidence level.
[0071] Specifically, after step 6 determines that the target location and the calculated field location satisfy a relationship, the confidence level is defined as the ratio of the abandoned tree size to the total tree size. When the confidence level is greater than a set threshold (e.g., 80%), the combined transformed expression, with the data field being a declarative logical rule or code, is stored and used in the system. The logical rule consists of a logical type, an expression, and a data point description. The logical type is defined in the DVP form. Combining the expression and data point description generates a logical check.
[0072] Figure 2 This illustrates the principle of an embodiment of the system for automatically generating logical checks when importing data verification plans according to the present invention. Please refer to... Figure 2 The system in this embodiment includes the following modules connected in sequence: reading module, matching module, structuring module, field parsing module, calculation description parsing module, positional relationship parsing module, and logical check generation module.
[0073] The read module is used to read the data verification plan file and parse the tables within it.
[0074] Users upload the Data Verification Plan (DVP) file written by DM to the system for storage, and then read and parse the file through the reading module.
[0075] The matching module is used to match table column headers based on name similarity.
[0076] The matching module is further configured as follows:
[0077] The system includes built-in, recognizable table column headers, which are generated by the user for subsequent analysis. These headers may contain information such as target locations for data verification, challenge texts for failed data verification, and challenge logic describing verification calculations. The system matches the table column headers parsed from the data verification plan file with the built-in, system-recognizable headers by name, using string edit distance as the approximation criterion. The most similar header is then selected. Ideally, after automatic matching, the user can also manually adjust the matching process.
[0078] For example, each column header recognized by the built-in system has some commonly used names. If a column header means "text for opening a query," then its name might be "query text," "query message," etc. These commonly used names are used as candidates for column header names, and the most similar column header name is searched in the user's data verification plan file. The similarity here is based on the edit distance of the strings; the closer the edit distance, the more similar they are considered. If a column header has two candidates, and a column header in the user's file is 1 and 2 away from these two candidates, then the closest distance between this user's column header and this specified column header is 1. For each column header recognized by the system, the closest column header in the user's file is matched.
[0079] The structured module is used to parse each row in the data verification plan file based on the matched table column headers, forming structured information.
[0080] Specifically, the table column header names match two aspects: visit name and field name. The text in these two columns is matched against the form structure defined by the CRF to determine the location of the target data field.
[0081] The field parsing module is used to parse the case report table fields in the logical text. The logical text matching module contains a table header in the data verification plan document. Within the structured information below this header, each cell contains text describing the logic of a data calculation; this is called logical text.
[0082] The field parsing module is further configured as follows.
[0083] The process iterates through all data fields in the Case Report Form (CRF) (Data verification operates on the CRF; the Data Verification Plan (DVP) establishes rules to verify the accuracy of data in the CRF). First, it uses the name SasName (a standard character representation of data points in clinical data collection) as the field name and matches it against the data field in the logical text. Next, it performs a fuzzy match between the logical text and the representation of the data field. Here, fuzzy matching means that the field name SasName can differ from the substring in the logical text by one, two (or more) characters. The fuzzy match stops at a difference of N characters. All matched substrings are marked as the corresponding fields for subsequent analysis.
[0084] In the field parsing module, all relevant fields in the logical text are matched. Following the principle of placing these matched fields as close as possible to the target object, their positions are determined. For example, if the logical text is "Start date is less than end date," mentioning two fields "Start date" and "End date," and the target object is "Start date," and the view is "Filter," then the position of the "End date" field should be as close as possible to the target object; that is, it should also be the "End date" field within the same form as "Start date" under the "Filter" view.
[0085] The computation description parsing module is used to parse computation descriptions in logical text.
[0086] The computation description parsing module is further configured as follows.
[0087] The logical text is parsed using the definition of a context-free grammar, transforming it into a forest structure of multi-branch trees. The context-free grammar summarizes the sentence patterns in the logical text and defines the summation using a grammar; the definition itself is a recursive tree structure. Trees in the forest that conform to a specific structure (i.e., satisfy the grammar definition) are retained, while those that do not are discarded. Specifically, given a context-free grammar definition and a set of inputs, a sophisticated algorithm can be used to segment and combine the input into many tree structures. Tree structures that satisfy the grammar definition are retained, while others, such as those containing errors or not fully satisfying the grammar definition, are discarded. Finally, tree structures that satisfy the specific structure are collected and subjected to a depth-first traversal, converting the tree structures into operational expressions. The depth-first traversal process involves iterating through the child nodes of the current node from left to right, with each traversal resulting in a subexpression. Based on the properties of this node itself, these sub-expressions are concatenated into a larger expression and returned to the upper level (for example, concatenating all sub-expressions with an "OR" relationship). This process continues until the root node of the tree, after traversal, yields a complete expression.
[0088] The location relationship parsing module is used to parse objects at target locations. By analyzing the locations of target data fields and the locations of data fields required in the calculation, it determines whether the data locations match. The object at the target location and the location of the target data fields describe the location of the target object and the object itself. Furthermore, the locations of the data fields required in the calculation are obtained in the field parsing module.
[0089] The positional relationship parsing module is further configured as follows.
[0090] In location analysis, it is determined whether the computational relationships between the target location object and the objects appearing in the logical text are satisfied. The determination rule is: whether all computational objects corresponding to each target object match a single data point in the CRF.
[0091] The logic check generation module is used to determine the confidence level and generate logic checks based on the confidence level.
[0092] The logic verification generation module is further configured as follows.
[0093] Once the location relationship parsing module determines that the target location and the calculated field location satisfy a relationship, the confidence level is defined as the ratio of the abandoned tree size to the total tree size. When the confidence level is greater than a set threshold (e.g., 80%), the combined and transformed expression, with the data fields being declarative logical rules or code, is stored and used in the system. The logical rule consists of a logical type, an expression, and a data point description. The logical type is defined in the DVP form. Combining the expression and data point description generates a logical check.
[0094] Furthermore, this invention discloses a computer-readable storage medium storing program instructions executable by a processor to implement the steps of the method for automatically generating logical checks through data verification plan import as described above. Specific steps have already been described. Figure 1 The embodiments shown are described in detail and will not be repeated here.
[0095] Furthermore, this invention discloses a computer device comprising a memory, a processor, and program instructions stored in the memory that are executable by the processor, wherein the processor executes the program instructions to implement the steps described above for implementing a method for automatically generating logical checks by importing a data verification plan. Specific steps have already been described. Figure 1 The embodiments shown are described in detail and will not be repeated here.
[0096] Although the methods described above are illustrated and depicted as a series of actions for the sake of simplicity, it should be understood and appreciated that these methods are not limited by the order of the actions, as some actions may occur in a different order and / or concurrently with other actions from the illustrations and descriptions herein or not illustrated and described herein but which may be understood by those skilled in the art, according to one or more embodiments.
[0097] Those skilled in the art will further appreciate that the various illustrative logic blocks, modules, circuits, and algorithm steps described in conjunction with the embodiments disclosed herein can be implemented as electronic hardware, computer software, or a combination of both. To clearly illustrate this interchangeability between hardware and software, the various illustrative components, blocks, modules, circuits, and steps are described above in a generalized manner in terms of their functionality. Whether such functionality is implemented as hardware or software depends on the specific application and the design constraints imposed on the overall system. Those skilled in the art may implement the described functionality in different ways for each specific application, but such implementation decisions should not be construed as departing from the scope of the invention.
[0098] The various illustrative logic blocks, modules, and circuits described in conjunction with the embodiments disclosed herein can be implemented or performed using a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or any combination thereof designed to perform the functions described herein. The general-purpose processor may be a microprocessor, but in alternatives, it may be any conventional processor, controller, microcontroller, or state machine. The processor may also be implemented as a combination of computing devices, such as a combination of a DSP and a microprocessor, multiple microprocessors, one or more microprocessors cooperating with a DSP core, or any other such configuration.
[0099] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein may be embodied directly in hardware, in a software module executed by a processor, or in a combination of both. The software module may reside in RAM memory, flash memory, ROM memory, EPROM memory, EEPROM memory, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art. An exemplary storage medium is coupled to a processor such that the processor can read and write information to / from the storage medium. In an alternative, the storage medium may be integrated into the processor. The processor and storage medium may reside in an ASIC. The ASIC may reside in a user terminal. In an alternative, the processor and storage medium may reside as discrete components in the user terminal.
[0100] In one or more exemplary embodiments, the described functionality may be implemented in hardware, software, firmware, or any combination thereof. If implemented in software as a computer program product, the functionality may be stored or transmitted as one or more instructions or code on or through a computer-readable medium. A computer-readable medium includes both computer storage media and communication media, encompassing any medium that facilitates the transfer of a computer program from one location to another. A storage medium may be any available medium accessible to a computer. By way of example and not limitation, such a computer-readable medium may include RAM, ROM, EEPROM, CD-ROM or other optical disc storage, disk storage or other magnetic storage devices, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and is accessible to a computer. Any connection is also legitimately referred to as a computer-readable medium. For example, if the software is transmitted from a website, server, or other remote source using coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL), or wireless technologies such as infrared, radio, and microwave, then the coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technologies such as infrared, radio, and microwave are included in the definition of a medium. As used in this article, disk and disc include compact discs (CDs), laser discs, optical discs, digital multi-purpose discs (DVDs), floppy disks, and Blu-ray discs. Disks typically reproduce data magnetically, while discs reproduce data optically using lasers. Combinations of these should also be included within the scope of computer-readable media.
[0101] The prior description of this disclosure is provided to enable any person skilled in the art to make or use this disclosure. Various modifications to this disclosure will be apparent to those skilled in the art, and the general principles defined herein may be applied to other variations without departing from the spirit or scope of this disclosure. Therefore, this disclosure is not intended to be limited to the examples and designs described herein, but should be accorded the widest scope consistent with the principles and novel features disclosed herein.
Claims
1. A method for automatically generating logical checks when importing data verification plans, characterized in that, The methods include: Step 1: Read the data verification plan file and parse the tables within it; Step 2: Match table column headers based on name similarity; Step 3: Based on the matched table column headers, parse each row in the data verification plan file to form structured information; Step 4: Parse the case report table fields in the logical text; Step 5: Parse the computational description in the logical text; Step 6: Parse the object at the target location. By analyzing the location of the target data field and the location of the data field required in the calculation, determine whether the data location matches. Step 7: Determine the confidence level and generate logical checks based on the confidence level; In step 2, the table column headers parsed from the data verification plan file are matched by name with the built-in system-recognizable table column headers, and the string edit distance is used as the approximation evaluation to match the most similar column headers. In step 3, the names of the table column headers are matched with two aspects: visit name and field name. The location of the target data field is determined by matching the text in these two columns with the form structure defined in the case report form. Step 4 further includes: traversing all data fields in the case report table, first using the name as the representation of the data field to match the data field in the logical text; second, performing fuzzy matching of the data field representation in the logical text; and finally, marking all matched substrings as the corresponding fields. Step 5 further includes: parsing the logical text, using the definition of context-free grammar, parsing the logical text into a forest structure composed of multi-branch trees; then retaining the trees in the forest that satisfy the grammar definition structure, and marking the trees that do not satisfy the grammar definition structure as discarded; finally, collecting the tree structures that satisfy the grammar definition structure, performing a depth-first traversal, and converting the tree structures into operational expressions.
2. The method for automatically generating logical checks by importing data verification plans according to claim 1, characterized in that, Step 7 further includes: After step 6 determines that the target location and the calculated field location satisfy the relationship, the confidence level is defined as the ratio of the abandoned tree size to the total tree size. When the confidence level is greater than the set threshold, the transformed expression is combined. The data field is a declarative logical rule or code, where the logical rule consists of a logical type, an expression, and a data point description. The logical type is determined in the form of the data verification plan. The expression and data point description are combined to generate a logical verification.
3. A system for automatically generating logical checks by importing data verification plans, characterized in that the system... include: The reading module is used to read the data verification plan file and parse the tables within it; The matching module is used to match table column headers based on name similarity; The structured module is used to parse each row in the data verification plan file based on the matched table column headers to form structured information; The field parsing module is used to parse the case report table fields in the logical text; The computation description parsing module is used to parse computation descriptions in logical text. The location relationship parsing module is used to parse objects at target locations. By analyzing the locations of target data fields and the locations of data fields required in the calculation, it determines whether the data locations match. The logic check generation module is used to determine the confidence level and generate logic checks based on the confidence level; In the matching module, the names of the table column headers parsed from the data verification plan file are matched with the built-in system-recognizable table column headers. The string edit distance is used as the approximation evaluation to match the most similar column headers. In the field parsing module, all data fields in the case report table are traversed. First, the name is used as the representation of the data field to match the data field in the logical text. Second, the logical text is fuzzily matched to the representation of the data field. Finally, all matched substrings are marked as the corresponding fields. In the structured module, the names of the table column headers match two aspects: visit name and field name. The text in these two columns is matched with the form structure defined in the case report form to determine the location of the target data field. The computation description parsing module is further configured as follows: it parses the logical text, uses the definition of context-free grammar to parse the logical text into a forest structure composed of multi-branch trees; then it retains the trees in the forest that satisfy the grammar definition structure and discards the trees that do not satisfy the grammar definition structure; finally, it collects the tree structures that satisfy the grammar definition structure, performs a depth-first traversal, and converts the tree structures into operational expressions. The location relationship parsing module is further configured to: in the location analysis process, determine whether the calculation relationship between the target location object and the objects appearing in the logical text is satisfied, wherein the judgment rule is: whether the calculation of all objects corresponding to each target object matches the odd number of data points in the case report table.
4. The system for automatically generating logical checks by importing data verification plans according to claim 3, characterized in that, The logic verification generation module is further configured as follows: Once the location relationship parsing module determines that the target location and the calculated field location satisfy the relationship, the confidence level is defined as the ratio of the abandoned tree size to the total tree size. When the confidence level is greater than the set threshold, the combined and transformed expression is used. The data field is a declarative logical rule or code, where the logical rule consists of a logical type, an expression, and a data point description. The logical type is determined in the form of the data verification plan. The expression and data point description are combined to generate a logical verification.
5. A computer-readable storage medium storing program instructions executable by a processor to implement the steps of the method for automatically generating logical checks by importing a data verification plan as described in any one of claims 1 to 2.
6. A computer device comprising a memory, a processor, and program instructions stored in the memory and executable by the processor, wherein the processor executes the program instructions to implement the steps of implementing a method for automatically generating logical checks by importing a data verification plan as claimed in any one of claims 1 to 2.
Citation Information
Patent Citations
Table data analysis method and device, computer equipment and storage medium
CN110502516A
Clinical electronic case report form construction method and system and electronic equipment
CN115132312A