Drug clinical test file quality control and risk early warning method based on artificial intelligence

Abstract

The invention relates to the technical field of artificial intelligence, and particularly discloses a drug clinical test file quality control and risk early warning method based on artificial intelligence, comprising the following steps: S1, collecting drug clinical test whole-process file data; s2, preprocessing the collected archive data; s3, feature extraction is carried out on the preprocessed data, and an optimal feature set is obtained through feature screening; s4, inputting the optimal feature set into a quality evaluation model for training, and then outputting an archive quality quantitative score; s5, inputting the quality quantitative score and the optimal feature set into a risk early warning model for training, and outputting a risk level; and S6, generating a quality assessment report, and pushing risk early warning information in real time. According to the drug clinical test file quality control and risk early warning method based on artificial intelligence, the AI model is combined with a dynamic monitoring mechanism, various risks occurring in a clinical test can be recognized in advance, and scientific decision support is provided for clinical test management.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and in particular to a method for quality control and risk warning of drug clinical trial records based on artificial intelligence. Background Technology

[0002] Drug clinical trials are a crucial step in verifying the safety and efficacy of drugs. Their records document core data from the entire clinical trial process, including protocol design, informed consent, case reporting, laboratory testing, ethical review, and adverse events. The quality of these records directly impacts the reliability and scientific validity of the clinical trial results; their completeness, accuracy, compliance, and logical coherence have a decisive influence on the success or failure of drug development and the safety of its clinical application.

[0003] Currently, the quality assessment of drug clinical trial records primarily relies on manual review, where professionals meticulously examine and score each page according to GCP guidelines, FDA standards, and other requirements. However, with the expansion of clinical trial scale, the number of records is growing exponentially. Manual review consumes significant time and manpower, and the varying professional backgrounds, experience levels, and judgment standards of different reviewers lead to inconsistent assessment results and a lack of objective and unified quantitative indicators. Furthermore, manual review is a post-hoc inspection, only identifying problems after all records have been completed, failing to identify potential quality issues in real time during the clinical trial process. This can exacerbate risks and even affect the compliance and effectiveness of the clinical trial.

[0004] To address the shortcomings of manual assessment, some studies have attempted to introduce information technology to assist in record management, developing clinical trial record management systems to achieve electronic storage and retrieval of records. However, existing systems can only complete basic record management and lack in-depth quality assessment and risk warning functions. Their core shortcomings are: first, they cannot achieve automated and intelligent quality judgment, still requiring manual participation in core review processes; second, the assessment dimensions are singular, focusing only on the completeness of the records while ignoring key dimensions such as data accuracy, logical consistency, and compliance, resulting in insufficient comprehensiveness of assessment results; third, they lack a dynamic risk warning mechanism, only able to passively respond to identified problems and unable to predict potential risks based on changes in record data, resulting in poor timeliness of warnings; fourth, the model design is simple, with existing machine learning models mostly using single algorithms, such as logistic regression and support vector machines, which are difficult to handle complex textual information and temporal features in record data, leading to low assessment accuracy and risk identification rates.

[0005] Therefore, there is an urgent need for an automated, precise, and comprehensive mechanism to assess the quality of drug clinical trial records and to provide real-time dynamic risk warnings. Summary of the Invention

[0006] The purpose of this invention is to provide an artificial intelligence-based method for quality control and risk warning of drug clinical trial records, enabling automated, precise, and comprehensive assessment of record quality, establishing a real-time dynamic risk warning mechanism, and solving the problems of low efficiency, strong subjectivity, delayed warning, and insufficient accuracy of existing technologies. This provides scientific, efficient, and reliable technical support for the management of drug clinical trial records.

[0007] To achieve the above objectives, this invention provides an artificial intelligence-based method for quality control and risk warning of drug clinical trial records, comprising the following steps: S1. Collect data from the entire process of drug clinical trials; S2. Preprocess the collected archival data; S3. Extract features from the preprocessed data and obtain the optimal feature set through feature filtering; S4. Input the optimal feature set into the quality assessment model for training and output a quantitative score of archive quality. S5. Input the quality quantification score and the optimal feature set into the risk warning model for training, and output the risk level; S6. Generate a quality assessment report and push risk warning information in real time.

[0008] Preferably, in S1, the files are divided into text files and numerical files, including files related to protocol design, informed consent, case reports, laboratory testing, ethical review, and adverse event records; Protocol design files include protocol approval documents, the final version of the clinical trial protocol, the revised version of the clinical trial protocol, the basis for sample size calculation, and the trial flowchart; Informed consent records include the original informed consent form, a revised version, the signed page of the informed consent form, and a record of the informed consent process. Case report files include Case Report Forms (CRF), Electronic Case Report Forms (eCRF), Data Question Forms, and Data Modification Records; Laboratory testing records include laboratory accreditation certificates, testing methodology validation reports, raw sample testing data, and result reports. Ethics review files include ethics review application forms, ethics review opinions, ethics approval documents, supplementary materials, and annual review reports; Adverse event record files include adverse event report forms, serious adverse event report forms, follow-up records, and risk control measures.

[0009] Preferably, in S1, a collection strategy combining real-time collection with timed incremental collection and distributed crawling with API interfaces is used to collect archive data, specifically as follows: Key milestone files are collected in real time and include protocol approval documents, final version of clinical trial protocol, sample size calculation basis, informed consent form signature page, informed consent process record, ethical review opinions, ethical approval documents and annual review report; Regular files are collected incrementally over a periodic period. Regular files include revised clinical trial protocols, trial flowcharts, case reports, laboratory test records, ethics review application forms, and supplementary materials. Structured data from various archives is collected in real time via API interface, while unstructured data is collected periodically using distributed crawlers. The collected archive data is stored in partitions according to archive type, clinical trial project, and time dimension.

[0010] Preferably, in S2, the preprocessing operations include data cleaning, standardization, normalization, missing value imputation, and outlier removal, specifically: S21. Use regular expressions to remove symbols, whitespace characters, and duplicate content from text files, remove duplicate case report records, and retain the latest version; for numerical files, remove data with incorrect formatting. S22. Convert all text files to UTF-8 encoding, segment them according to sentence level, and treat each sentence as a text sequence; for numerical files, use Z-score standardization to convert the values ​​into standard normal distribution data with a mean of 0 and a variance of 1. S23. Use the minimum-maximum normalization method to map the standardized data to the [0,1] interval; S24. An improved KNN algorithm is used to handle missing values, and neighbor samples are selected based on feature similarity. The Manhattan distance is used as the distance metric, and the calculation formula is as follows: ; in, , For the sample and samples eigenvectors, For the total number of features, For the sample The 1 eigenvalue, For the sample The One eigenvalue; S25. To remove outliers using the IQR method, first calculate the quartiles of the dataset. and Then calculate the interquartile range (IQR). Make the following judgment: like or If the data meets the judgment criteria, it will be identified as an outlier and deleted. This is the normalized data.

[0011] Preferably, S3 is as follows: The preprocessed text sequence is input into the BERT model, and the output of the last hidden layer is extracted as the text semantic feature vector. Simultaneously, the TF-IDF algorithm is used to calculate the weights of keywords in the text, and the top keywords with the highest TF-IDF values ​​are selected. Statistical feature vectors are constructed from keywords. The semantic feature vectors are then concatenated with the statistical feature vectors to obtain a subset of text features. ; Numerical features of word frequency were extracted from numerical archive data. These features included data integrity ratio, completion standardization, logical consistency score, and temporal continuity score. The data integrity ratio was the ratio of the number of completed fields to the total number of fields. Completion standardization was calculated by the proportion of fields whose completion format met requirements. The logical consistency score was obtained by verifying the logical relationships between related fields. The temporal continuity score was obtained by evaluating the continuity of key time points. The resulting subset of numerical features was denoted as... ; Process features are constructed based on the completion status of key milestones in clinical trials. The feature value for each key milestone is calculated based on its completion status and completion time. Completion status is categorized as completed, incomplete, and overdue. The completion time score is calculated by the deviation between the actual completion time and the planned completion time. The resulting subset of process features is denoted as... Each key node corresponds to two feature values: completion status and completion time score. Compliance features constructed in accordance with GCP regulations and FDA standards are denoted as follows: Subset of text features Numerical feature subsets Process feature subset Compliance feature subset Feature fusion is performed to obtain an initial feature set. .

[0012] Preferably, in S3, a method combining mutual information entropy and recursive feature elimination is used for feature selection, specifically as follows: S31. Calculate the initial feature set The mutual information entropy between each feature and the quality label measures the correlation between the feature and the quality assessment result. The formula for calculating the mutual information entropy is: ; in, The mutual information entropy value, For the initial feature set, For a set of quality labels, Features With tags The joint probability distribution, Features Marginal probability distribution, For tags The marginal probability distribution shows that the larger the mutual information entropy value, the stronger the correlation between the feature and the quality label. S32. Using mutual information entropy as the initial sorting criterion, a recursive feature elimination algorithm is used to iteratively filter features. In each iteration, a temporary model is trained, and the importance score of each feature is calculated. In each iteration, 5% of the features with the lowest importance scores are removed, and the top 30%-40% of the features with the highest importance scores are retained to form the optimal feature set. .

[0013] Preferably, in S4, the quality assessment model is a deep learning model based on the fusion of CNN and Transformer. The quality assessment model includes an input layer, a convolutional module, a Transformer encoder module, a fully connected module, and an output layer. The input layer receives the optimal feature set. The convolution module includes two convolutional layers and two pooling layers to capture the local dependencies of features. The Transformer encoder module includes Z coding layers, each of which includes a multi-head attention sublayer and a feedforward neural network sublayer. The output of each coding layer uses residual connections and layer normalization. The fully connected module consists of two fully connected layers. The first fully connected layer FC1 uses ReLU as the activation function, and the second fully connected layer FC2 uses Sigmoid as the activation function. The output layer multiplies the output of the second fully connected layer FC2 by 100 to obtain the archive quality quantification score. The scoring range is [0, 100], and the higher the score, the better the file quality.

[0014] Preferably, the first convolutional layer Conv1 of the quality assessment model has 64 kernels, a kernel size of 3×3, a stride of 1, a padding method of "same", and an activation function of ReLU; the first pooling layer Pool1 uses max pooling, with a pooling window size of 2×2 and a stride of 2. The second convolutional layer, Conv2, has 128 kernels, a kernel size of 3×3, a stride of 1, and uses the same padding method. The activation function is ReLU. The second pooling layer, Pool2, uses max pooling with a pooling window size of 2×2 and a stride of 2. The output of Pool2 is flattened and converted into a one-dimensional vector.

[0015] Preferably, in S5, the risk warning model adopts a deep learning model that integrates LSTM and attention mechanism. The risk warning model includes an input layer, an embedding layer, an LSTM layer, an attention layer, a fully connected layer, and an output layer. The input layer receives the quality score output by the quality assessment model. With the optimal feature set The embedding layer maps the input vector to a low-dimensional dense vector; The LSTM layer consists of 3 hidden layers, with a dropout rate of 0.3 and a recurrent dropout rate of 0.1 to prevent overfitting. The attention layer employs a self-attention mechanism, calculating the attention weights of the hidden states at each time step. The attention score is calculated using the following formula: ; in, For the first The query vector at the time step and the _th Attention scores between key vectors at each time step For the first Query vectors at each time step For the first The key vector at each time step Given a vector dimension, the attention weight calculation formula is as follows: ; in, The total length of the input sequence; Normalization using the Softmax function, attention output ,in For the first A vector of values ​​for each time step; The fully connected layer consists of two fully connected layers. The first fully connected layer FC3 uses ReLU as the activation function; the second fully connected layer FC4 uses Softmax as the activation function and outputs four prediction probability values, corresponding to four levels: low risk, medium risk, high risk, and very high risk. The output layer outputs the risk level with the highest probability value and calculates the risk score: ; in, , , , These are the predicted probabilities for low risk, medium risk, high risk, and extremely high risk levels, respectively. , , , The highest scores are for low risk, medium risk, high risk, and extremely high risk, respectively.

[0016] Preferably, in S6, the early warning information includes the risk level, the type of file involved, the scope of impact, and the emergency handling time limit. Early warning information is pushed according to the risk level. For extremely high risks, it is pushed in real time through three methods: SMS, email, and system pop-up window to ensure that relevant personnel receive it as soon as possible; for high risks, it is pushed through SMS and email within 24 hours; for medium risks, it is pushed through email within 48 hours. At the same time, the system records the reception status and processing progress of early warning information, and early warning information that is not processed in time will be reminded again.

[0017] Therefore, the present invention employs the above-mentioned artificial intelligence-based drug clinical trial record quality control and risk warning method, and the beneficial effects are as follows: (1) The present invention uses an artificial intelligence model to realize the automated evaluation of the quality of the archives. There is no need for manual page-by-page review. The evaluation time of a single archive is shortened from tens of minutes of manual review to seconds. The evaluation efficiency is also significantly improved. It can meet the rapid review needs of large-scale clinical trial archives and reduce labor and time costs.

[0018] (2) Based on multi-dimensional feature engineering and fusion deep learning model, this invention constructs an objective and unified quantitative evaluation standard to avoid the subjectivity and differences of manual evaluation. At the same time, it establishes a dynamic risk warning mechanism. Based on the deep learning model that integrates LSTM and attention mechanism, it can capture the temporal change trend of archive data, identify potential risks in advance, and ensure that relevant personnel take timely risk control measures to reduce the possibility of risk expansion.

[0019] (3) This invention constructs four feature subsets: text features, numerical features, process features, and compliance features, covering multiple key dimensions such as the integrity, accuracy, compliance, logic, and process standardization of archives. Compared with existing evaluation methods that only focus on a single dimension, the evaluation results are more comprehensive and can fully reflect the overall level of archive quality.

[0020] (4) The present invention can provide detailed evaluation reports and early warning information, and the evaluation reports can be exported in multiple formats and the early warning information can be pushed through multiple channels, which is convenient for promotion and application in actual clinical trial management.

[0021] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description

[0022] Figure 1 This is an overall flowchart of an embodiment of the artificial intelligence-based drug clinical trial record quality control and risk warning method of the present invention; Figure 2This is a flowchart of data acquisition and preprocessing in an embodiment of the artificial intelligence-based drug clinical trial record quality control and risk warning method of the present invention. Figure 3 This is a flowchart illustrating the feature extraction and screening process of an embodiment of the artificial intelligence-based drug clinical trial record quality control and risk warning method of the present invention. Figure 4 This is a schematic diagram illustrating the risk level output process of an embodiment of the artificial intelligence-based drug clinical trial record quality control and risk warning method of the present invention. Figure 5 This is a quality assessment report and risk warning diagram of an embodiment of the drug clinical trial record quality control and risk warning method based on artificial intelligence of the present invention. Detailed Implementation

[0023] The technical solution of the present invention will be further described below with reference to the accompanying drawings and embodiments.

[0024] Unless otherwise defined, the technical or scientific terms used in this invention shall have the ordinary meaning as understood by one of ordinary skill in the art to which this invention pertains.

[0025] like Figure 1 As shown, the method for quality control and risk warning of drug clinical trial records based on artificial intelligence is characterized by the following steps: S1. Collect data from the entire process of drug clinical trials, including records of protocol design, informed consent, case reports, laboratory testing, ethical review, and adverse event records.

[0026] Protocol design files include protocol approval documents, final version of the clinical trial protocol, revised version of the clinical trial protocol, sample size calculation basis, and trial flowchart; informed consent files include the original informed consent form, revised version, signed informed consent form page, and informed consent process record; case report files include case report forms (CRF), electronic case report forms (eCRF), data question forms, and data modification records.

[0027] Laboratory testing records include laboratory accreditation certificates, methodological validation reports, raw sample testing data, and result reports; ethics review records include ethics review application forms, ethics review opinions, ethics approval documents, supplementary materials, and annual review reports; adverse event record records include adverse event report forms, serious adverse event report forms, follow-up records, and risk control measures.

[0028] like Figure 2 As shown, this invention employs a data collection strategy that combines real-time acquisition with timed incremental acquisition, and distributed crawlers with API interfaces, specifically: Key milestone files are collected in real time and include protocol approval documents, the final version of the clinical trial protocol, the basis for sample size calculation, the informed consent form signature page, the informed consent process record, ethical review opinions, ethical approval documents, and annual review reports.

[0029] Regular records are collected incrementally at regular intervals. Regular records include revised clinical trial protocols, trial flowcharts, case reports, laboratory test records, ethics review application forms, and supplementary materials.

[0030] Structured data from various archives is collected in real time via API interface, while unstructured data is collected periodically using distributed crawlers. The collected archive data is partitioned and stored in a distributed database according to archive type, clinical trial project, and time dimension.

[0031] S2, such as Figure 3 As shown, the collected archival data undergoes preprocessing, including data cleaning, standardization, normalization, missing value imputation, and outlier removal. Specifically: S21. Use regular expressions to remove special symbols, whitespace characters, and duplicate content from text files, remove duplicate case report records, and retain the latest version; for numerical files, remove data with incorrect formatting.

[0032] S22. Convert the text archives to UTF-8 encoding and segment them according to the sentence level, with each sentence as a text sequence; for numerical archives, use Z-score standardization to convert the numerical values ​​into standard normal distribution data with a mean of 0 and a variance of 1.

[0033] S23. The min-max normalization method is used to map the standardized numerical data to the interval [0,1].

[0034] S24. An improved KNN algorithm is used to handle missing values. Neighbor samples are selected based on feature similarity, with the number of neighbors set to 15. The Manhattan distance metric is used, and the calculation formula is as follows: ; in, , For the sample ,sample eigenvectors, For the total number of features, For the sample The 1 eigenvalue, For the sample The Each feature value.

[0035] S25. To remove outliers using the IQR method, first calculate the quartiles of the dataset. and Then calculate the interquartile range (IQR) using the following formula: , The 25th percentile of the dataset. To determine the 75th percentile of the dataset, make the following judgment: like or If the data meets the judgment criteria, it will be identified as an outlier and deleted. This is the normalized data.

[0036] S3. Perform feature extraction on the preprocessed data, and obtain the optimal feature set through feature filtering; the extracted features include text features, numerical features, process features, and compliance features, specifically: like Figure 4 As shown, a combination of the BERT model and TF-IDF is used to extract text features. The BERT model consists of a 12-layer Transformer encoder with a hidden layer dimension of 768, 12 multi-head attention heads, and a text sequence length of 512. The preprocessed text sequence is input into the BERT model, and the output of the last hidden layer is extracted as the text semantic feature vector. Simultaneously, the TF-IDF algorithm is used to calculate the weights of keywords in the text, and the top keywords with the highest TF-IDF values ​​are selected. A statistical feature vector is constructed using 200 keywords. The semantic feature vector is then concatenated with the statistical feature vector to obtain a subset of text features. .

[0037] Numerical features of word frequency were extracted from numerical archive data. These features included data completeness ratio, completion standardization, logical consistency score, and temporal continuity score. Data completeness ratio was the ratio of completed fields to the total number of fields. Completion standardization was calculated by the proportion of fields whose format met requirements, such as date format and numerical units. Logical consistency score was obtained by verifying the logical relationships between related fields, such as the inclusion time being less than the exclusion time or the dosage being within a reasonable range. One point was deducted for each logical inconsistency, with a maximum score of 100. Temporal continuity score was obtained by assessing the continuity of key time points, such as the follow-up interval meeting protocol requirements, with a maximum score of 100. The resulting subset of numerical features was denoted as... The dimension is 4, and all feature values ​​are normalized to the interval [0,1].

[0038] Based on the completion status of key nodes in clinical trials, process features are constructed. Key nodes selected include protocol approval documents, the final version of the clinical trial protocol, sample size calculation basis, informed consent form signature pages, informed consent process records, ethical review opinions, ethical approval documents, and annual review reports. The feature value for each key node is calculated based on its completion status and completion time. Completion status is categorized as completed, incomplete, and overdue, assigned values ​​of 1.0, 0.0, and 0.5 respectively. The completion time score is calculated based on the deviation between the actual completion time and the planned completion time. No deviation earns 1.0 points, and each additional day of deviation deducts 0.01 points, with a minimum score of 0. The resulting subset of process features is denoted as […]. The dimension is 20, and each key node corresponds to two feature values: completion status and completion time score.

[0039] Building compliance features based on GCP guidelines and FDA standards Subset of text features Numerical feature subsets Process feature subset Compliance feature subset Feature fusion is performed to obtain an initial feature set. During feature fusion, feature standardization is used to map all feature values ​​to the [0,1] interval.

[0040] Then, a method combining mutual information entropy and recursive feature elimination is used for feature selection, specifically: S31. Calculate the initial feature set The mutual information entropy between each feature and the quality label measures the correlation between the feature and the quality assessment result. The formula for calculating the mutual information entropy is: ; in, The mutual information entropy value, For the initial feature set, For a set of quality labels, Features With tags The joint probability distribution, Features Marginal probability distribution, For tags The marginal probability distribution shows that the larger the mutual information entropy value, the stronger the correlation between the feature and the quality label.

[0041] S32. Using mutual information entropy as the initial sorting criterion, a recursive feature elimination algorithm is used to iteratively filter features. In each iteration, a temporary model is trained, and the importance score of each feature is calculated. In each iteration, the 5% of features with the lowest importance scores are removed. The temporary model uses a random forest algorithm with 100 decision trees and a maximum depth of 10. After iteration, the top 30%-40% of features by importance score are retained to construct the optimal feature set. After screening, the optimal feature set dimension is approximately 302-403, and the specific dimension can be adjusted according to the actual data.

[0042] S4. After training the quality assessment model by inputting the optimal feature set, the output is a quantitative score of file quality. The quality assessment model of this invention uses a deep learning model based on the fusion of CNN and Transformer. The quality assessment model includes an input layer, a convolutional module, a Transformer encoder module, a fully connected module, and an output layer. The input layer receives the optimal feature set. The feature dimension is D, and the input vector shape is (batch_size, D), where batch_size is the batch size, which is set to 32.

[0043] The convolutional module consists of two convolutional layers and two pooling layers, used to capture local dependencies of features. The first convolutional layer, Conv1, has 64 kernels, a kernel size of 3×3, a stride of 1, and uses the same padding method. The activation function is ReLU, and the output shape is (batch_size, D, 64). The first pooling layer, Pool1, uses max pooling, with a pooling window size of 2×2, a stride of 2, and an output shape of (batch_size, D / 2, 64).

[0044] The second convolutional layer, Conv2, has 128 kernels, a kernel size of 3×3, a stride of 1, and uses the same padding method (same as above). The activation function is ReLU. The second pooling layer, Pool2, uses max pooling with a pooling window size of 2×2 and a stride of 2. The output shape is (batch_size, D / 4, 128). The output of Pool2 is flattened and converted into a one-dimensional vector with an output shape of (batch_size, D / 4). 128).

[0045] The Transformer encoder module consists of Z=6 encoding layers, each including a multi-head attention sublayer and a feedforward neural network sublayer. The output of each encoding layer uses residual connections and layer normalization. The fully connected module consists of two fully connected layers. The first fully connected layer FC1 has an input dimension of 512 and an output dimension of 256, using ReLU activation. The second fully connected layer FC2 has an input dimension of 256 and an output dimension of 1, using Sigmoid activation, with the output value mapped to the [0,1] interval. The output layer multiplies the output of the second fully connected layer FC2 by 100 to obtain the archive quality quantification score. The scoring range is [0, 100], and the higher the score, the better the file quality.

[0046] S5. Input the quality quantification score and the optimal feature set into the risk warning model for training, and output the risk level. The risk warning model of this invention adopts a deep learning model that integrates LSTM and attention mechanisms. The risk warning model includes an input layer, an embedding layer, an LSTM layer, an attention layer, a fully connected layer, and an output layer. The input layer receives the quality score output by the quality assessment model. With the optimal feature set The input vector is concatenated to form the early warning model input vector. The input dimension is D+1, where D is the dimension of the optimal feature set. The shape of the input vector is (batch_size, T, D+1), where T is the time step, which is set to 30, meaning that 30 consecutive days of archival data are used for risk prediction.

[0047] The embedding layer maps the input vector to a low-dimensional dense vector. The output dimension of the embedding layer is 256, and the output shape is (batch_size, T, 256). The LSTM layer includes three hidden layers, each with 256 neurons. The dropout rate is set to 0.3, and the recurrent dropout rate is set to 0.1 to prevent overfitting. The attention layer uses a self-attention mechanism to calculate the attention weights of the hidden states at each time step, highlighting key risk features. The attention score is calculated as follows: ; in, For the first The query vector at the time step and the _th Attention scores between key vectors at each time step For the first Query vectors at each time step For the first The key vector at each time step Given a vector dimension, the attention weight calculation formula is as follows: ; in, The total length of the input sequence.

[0048] Normalization using the Softmax function, attention output ,in For the first The value vectors at each time step are given, and the attention output has a shape of (batch_size, 256).

[0049] The fully connected layer consists of two layers. The first fully connected layer, FC3, has an input dimension of 256 and an output dimension of 128, using ReLU as the activation function. The second fully connected layer, FC4, has an input dimension of 128 and an output dimension of 4, using Softmax as the activation function. It outputs four predicted probability values, corresponding to low risk, medium risk, high risk, and very high risk levels. The output layer outputs the risk level with the highest probability value and calculates the risk score using the following formula: ; in, The risk score ranges from [0, 100]. , , , These are the predicted probabilities for low risk, medium risk, high risk, and extremely high risk levels, respectively. , , , These represent the highest scores for low risk, medium risk, high risk, and extremely high risk, with low risk scores ranging from [0,25), medium risk scores ranging from [25,50), high risk scores ranging from [50,75], and extremely high risk scores ranging from [75,100].

[0050] Score the quality of the archives With the optimal feature set The data is concatenated, input into the trained risk warning model, and the model outputs the risk level and risk score. When the risk level is medium or above, an early warning signal is triggered. The early warning signal includes the risk level, the type of file involved, the location of the defect, the scope of impact, and the emergency handling time limit. The push priority is set according to the risk level, and the push methods include SMS, email, and system pop-up.

[0051] S6. Generate a quality assessment report and push risk warning information in real time.

[0052] like Figure 5As shown, the quality assessment report generated by this invention includes six core modules: basic file information, detailed quality score, risk point identification, historical data comparison, and a list of improvement suggestions. Basic file information includes the clinical trial project name, file type, collection time, and data volume. Detailed quality score includes overall quality score, text quality score, numerical quality score, process quality score, compliance quality score, and the percentage of each dimension's score. Risk point identification clearly identifies the specific location of high-risk defects, such as an error in a field of a case report form or a missing element in an informed consent form. Historical data comparison displays the current score compared to scores from similar projects in the same period, analyzing quality trends. The list of improvement suggestions provides recommendations for addressing identified risk points, including supplementing missing fields, correcting logical errors, and strengthening process control.

[0053] The warning information includes the risk level, the type of files involved, the scope of impact, and the emergency handling time limit. The warning information is pushed out according to the risk level. For extremely high risk, it is pushed out simultaneously through SMS, email, and system pop-up window to ensure that relevant personnel receive it as soon as possible; for high risk, it is pushed out through SMS and email; for medium risk, it is pushed out through email; and for low risk, there is no push out.

[0054] The emergency response time limit is set according to the risk level: 24 hours for extremely high risk, 72 hours for high risk, and 7 days for medium risk. At the same time, the system records the reception status and processing progress of the early warning information, and early warning information that is not processed in time will be reminded a second time.

[0055] Example 1 This embodiment uses a Phase III clinical trial project of a certain drug as an application scenario. The project includes 10 research centers, 500 enrolled patients, and a total of more than 12,000 files of various types, covering six core types of files such as protocol design, informed consent, and case reports. The method of this invention is used to conduct file quality assessment and risk warning.

[0056] The hardware environment is set up using a distributed server cluster, which includes one master server and ten slave servers. The master server is equipped with an Intel Xeon Gold 6248 processor, 128GB of memory, a 2TB SSD, and an NVIDIA Tesla V100 GPU; the slave servers are equipped with an Intel Xeon Silver 4210 processor, 64GB of memory, a 1TB SSD, and an NVIDIA Tesla P40 GPU.

[0057] Software environment: The operating system is Ubuntu 20.04 LTS, the database is MySQL 8.0, the distributed storage is Hadoop 3.2, the deep learning framework is PyTorch 1.10, the Python version is 3.8, the data processing libraries include Pandas 1.4, NumPy 1.21, and Scikit-learn 1.0, and the natural language processing libraries include NLTK 3.7 and HuggingFace Transformers 4.18.

[0058] A total of 12,368 files were collected from the entire clinical trial project for this drug, including 245 protocol design files, 1,056 informed consent files, 8,763 case reports, 1,248 laboratory testing files, 326 ethical review files, and 730 adverse event records. Data formats included 8,234 PDF files, 2,156 DOCX files, 987 XML files, and 991 CSV files. Simultaneously, five experts with over 10 years of experience reviewing clinical trial files were invited to manually annotate 2,000 randomly selected files according to GCP guidelines and FDA standards, assigning quality scores (0-100 points) and risk levels (low, medium, high, very high) as real-world labels for model training.

[0059] S1. Collect archival data using a combination of distributed crawlers and API interfaces: For structured data in electronic medical record systems and laboratory information systems, collect data in real time through API interfaces; for unstructured data such as scanned copies of paper archives and externally submitted PDF documents, collect data periodically using distributed crawlers, with the collection cycle set to 24 hours.

[0060] Text information in scanned documents was extracted using OCR technology (Tesseract OCR engine, with a recognition accuracy of 98.5%). The collected data was stored in partitions according to document type, clinical trial project, and time dimension for easy retrieval and access. After collection, a total of 12,368 documents were completed, with a data integrity rate of 99.8%, and no missing core documents.

[0061] S2, Data Preprocessing: S21. Use regular expressions to remove special characters, whitespace characters, and duplicate content (such as deleting "###") from text files. Remove irrelevant symbols (such as ", "), remove duplicate case report records, and keep the latest version. For numerical files, remove data with incorrect format (such as converting non-standard values ​​such as "12a" and "thirty" to standard number format, and marking those that cannot be converted as missing values).

[0062] S22. Convert all text files to UTF-8 encoding and segment them according to sentence level, with each sentence as a text sequence; for numerical files (such as laboratory test results, dosage data, etc.), use the Z-score standardization method to process them, and the mean of the standardized data is 0 and the variance is 1.

[0063] S23. Using the minimum-maximum normalization method, the standardized numerical data is mapped to the [0,1] interval.

[0064] S24. An improved KNN algorithm was used to handle missing values, with the number of neighbors set to 15, and Manhattan distance used as the distance metric. Missing values ​​in this dataset were mainly concentrated in some non-core fields of the case report table, with a missing value rate of 3.2%. After imputation, the data integrity reached 100%. For example, for the missing dosage data in the case report table, using patient age, gender, and disease stage as features, the mean of the neighbor samples was calculated as the imputation value, and the Manhattan distance was calculated as follows: ; in, , Let the ages of patient a and patient b be... , For patient a and patient b, their genders are: , For the disease staging of patients a and b, for missing fields in text-based files, fill in the "no relevant record" flag to ensure data integrity.

[0065] S25. Use the IQR method to identify outliers and calculate... , The threshold for outlier detection is , Data in the numerical archive that exceeded this range was identified as outliers. A total of 128 outliers were identified, accounting for 0.8%. After manual verification, they were confirmed to be data entry errors and were removed. After removal, the reliability of the data was significantly improved.

[0066] S3. Text features are extracted using a combination of the BERT-Base model and TF-IDF. The preprocessed text sequence is input into the BERT model with a sequence length of 512. The output of the last hidden layer is extracted as a semantic feature vector with a dimension of 768. The TF-IDF algorithm is used to calculate keyword weights. The first 200 keywords are selected to construct a statistical feature vector with a dimension of 200. These vectors are then concatenated to obtain a subset of the text features. The dimension is 968.

[0067] Four types of numerical features were constructed: data integrity ratio, completion standardization, logical consistency score, and time continuity score. Calculations showed that the average data integrity ratio for this project archive was 0.96, the average completion standardization score was 0.92, the average logical consistency score was 95.3, and the average time continuity score was 93.7. After normalization, a subset of numerical features was obtained. The dimension is 4.

[0068] Ten key nodes in the core process were selected, and each key node corresponds to two features: completion status and completion time score, to construct a subset of process features. The dimension is 20. According to statistics, the project plan approval and ethics review were completed on time, while the enrollment and trial summary report were completed late, with completion time scores of 0.8 and 0.7 respectively.

[0069] Based on GCP guidelines and FDA standards, 15 compliance check items were constructed (such as "completeness of informed consent elements" and "timely reporting of adverse events"). Each check item corresponds to a feature value, thus constructing a subset of compliance features. The dimensions are 15. Preliminary assessment indicates that 13 inspection items meet the standards, and 2 inspection items partially meet the standards, with compliance characteristic values ​​of 1.0 and 0.5 respectively.

[0070] By concatenating the above features, we obtain the initial feature set. The dimension is 968+4+20+15=1007, and all feature values ​​are normalized to the [0,1] interval to ensure consistent dimensions.

[0071] A method combining mutual information entropy and recursive feature elimination was used to select features. First, the mutual information entropy between each feature and the quality label was calculated, with a mean value of 0.78, indicating a strong correlation between the features and the quality label. Then, 50 iterations of recursive feature elimination were performed, removing 5% of weakly correlated features in each iteration, ultimately retaining 386 features to construct the optimal feature set. The dimension is 386.

[0072] S4. Model Training: The 2000 labeled archive data sets were divided into a 7:2:1 ratio: 1400 training sets, 400 validation sets, and 200 test sets. The model structure followed the design described in the invention, with consistent parameter settings for convolutional layers, Transformer encoders, and fully connected layers. The AdamW optimizer was used, with a batch size of 32 and 100 iterations. During training, the training set loss gradually decreased from an initial 0.85 to 0.03, and the validation set loss decreased from 0.78 to 0.05. After 56 iterations, the validation set loss stopped decreasing, triggering an early stopping strategy to save the optimal model parameters.

[0073] The model performance was evaluated on the test set. The mean squared error between the model's predicted scores and the actual scores was 0.04, the mean absolute error was 0.12, and the evaluation accuracy reached 95.2%. The correlation coefficient between the predicted scores and the actual scores was [not specified]. This indicates that the model's prediction performance is good and can accurately reflect the quality level of archives.

[0074] The optimal feature set of the 12,368 files in this project Input the trained quality assessment model and output a quality score for each document. According to statistics, the overall quality score of the project files was 89.7 points, with the average score for program design files being 94.3 points, informed consent files being 92.6 points, case report files being 88.5 points, laboratory testing files being 91.2 points, ethics review files being 95.7 points, and adverse event record files being 87.3 points.

[0075] S5. 3680 archival data points from 30 consecutive days of the project were selected and divided into a training set of 2576 data points, a validation set of 736 data points, and a test set of 368 data points, using a 7:2:1 ratio. The model structure followed the design described in the invention, with consistent parameter settings for the embedding layer, LSTM layer, attention layer, and fully connected layer. The Adam optimizer was used, with a batch size of 32 and 80 iterations. During training, the cross-entropy loss on the training set decreased from an initial 1.85 to 0.12, and the loss on the validation set decreased from 1.76 to 0.15. After 62 iterations, the loss on the validation set stopped decreasing, triggering an early stopping strategy to save the optimal model parameters.

[0076] The model performance was evaluated on the test set using accuracy, precision, recall, and F1 score as evaluation metrics. The evaluation results are shown in Table 1. The results show that the overall accuracy of the model reached 93.5%, with 94.2% precision and 92.8% recall for low-risk identification; 93.7% precision and 91.5% recall for medium-risk identification; 92.6% precision and 93.3% recall for high-risk identification; and 95.1% precision and 94.7% recall for very high-risk identification, indicating that the model can accurately identify different levels of risk.

[0077] Table 1 Multidimensional Evaluation Table of Model Performance

[0078] The archive quality score for 30 consecutive days The data is concatenated with the optimal feature set, input into the trained risk warning model, and outputs the risk level and risk score. The project identified 186 medium-risk cases, 42 high-risk cases, and 8 very high-risk cases, with no low-risk cases. Very high-risk cases mainly involved issues such as logical inconsistencies in case report forms and untimely reporting of adverse events; high-risk cases mainly involved issues such as incomplete informed consent forms and non-standard laboratory test data recording; medium-risk cases mainly involved issues such as errors in file completion and minor delays in process steps. The early warning response time was 2.8 seconds, meeting the design requirements.

[0079] S6. Generate the project's archive quality assessment report, including the basic information module displaying the project name, 12,368 archives, collection time, etc.; the quality score details module displays the overall score of 89.7 points, with scores for each dimension as follows: text quality 91.2 points, numerical quality 90.5 points, process quality 87.3 points, compliance quality 92.6 points, and the percentage of each dimension's score; the defects in the adverse event record archives are mainly concentrated in the reporting time field.

[0080] The risk point identification clearly points out the specific defects in 8 extremely high-risk files, such as logical contradictions in the dosage data of 3 case report forms in Center 5 and overdue submission of 2 adverse event reports in Center 8.

[0081] Historical data comparison shows that the project score is 4.4 points higher than the average score of 85.3 points for similar projects, placing it in the upper range of quality. The improvement suggestion list provides specific recommendations for addressing the identified deficiencies, such as supplementing the informed consent form, correcting data errors in the case report form, and strengthening the management of adverse event reporting. The evaluation report was exported in PDF and Excel formats and submitted to the project management team.

[0082] Warning information was pushed out according to risk level: 8 extremely high-risk cases were pushed out via SMS, email, and system pop-ups, with an emergency handling time limit of 24 hours; 42 high-risk cases were pushed out via SMS and email, with an emergency handling time limit of 72 hours; and 186 medium-risk cases were pushed out via email, with an emergency handling time limit of 7 days. Upon receiving the warning information, the project management team promptly organized relevant research centers to make rectifications. The rectification of the 8 extremely high-risk cases was completed within 24 hours, the 42 high-risk cases within 72 hours, and the 186 medium-risk cases within 7 days, effectively controlling the risks of clinical trials.

[0083] In summary, this embodiment, by applying the method of the present invention, achieves automated and precise assessment of the quality of clinical trial project files for this anti-tumor drug, as well as real-time risk warning. After implementation, the file assessment cycle was shortened from 30 days of manual review to 2 hours, an efficiency improvement of 360 times; the consistency of assessment results reached 92%, an improvement of 22.7% compared to manual assessment; the average advance risk identification time reached 14 days, an improvement of 72% compared to traditional post-review; the rectification completion rate reached 100%, and the compliance of clinical trials was significantly improved.

[0084] Therefore, the present invention adopts the above-mentioned artificial intelligence-based drug clinical trial record quality control and risk warning method, which effectively solves the problems of low efficiency, strong subjectivity and delayed warning in the existing technology, and provides scientific, efficient and reliable technical support for clinical trial record management, and has important practical application value.

[0085] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit them. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the technical solutions of the present invention, and these modifications or equivalent substitutions cannot cause the modified technical solutions to deviate from the spirit and scope of the technical solutions of the present invention.

Claims

1. A method for quality control and risk warning of drug clinical trial records based on artificial intelligence, characterized in that, Includes the following steps: S1. Collect data from the entire process of drug clinical trials; S2. Preprocess the collected archival data; S3. Extract features from the preprocessed data and obtain the optimal feature set through feature filtering; S4. Input the optimal feature set into the quality assessment model for training and output a quantitative score of archive quality. S5. Input the quality quantification score and the optimal feature set into the risk warning model for training, and output the risk level; S6. Generate a quality assessment report and push risk warning information in real time.

2. The method for quality control and risk warning of drug clinical trial records based on artificial intelligence according to claim 1, characterized in that: In S1, the files are divided into text files and numerical files, including files related to program design, informed consent, case reports, laboratory testing, ethical review, and adverse event records. Protocol design files include protocol approval documents, the final version of the clinical trial protocol, the revised version of the clinical trial protocol, the basis for sample size calculation, and the trial flowchart; Informed consent records include the original informed consent form, a revised version, the signed page of the informed consent form, and a record of the informed consent process. Case report files include Case Report Forms (CRF), Electronic Case Report Forms (eCRF), Data Question Forms, and Data Modification Records; Laboratory testing records include laboratory accreditation certificates, testing methodology validation reports, raw sample testing data, and result reports. Ethics review files include ethics review application forms, ethics review opinions, ethics approval documents, supplementary materials, and annual review reports; Adverse event record files include adverse event report forms, serious adverse event report forms, follow-up records, and risk control measures.

3. The method for quality control and risk warning of drug clinical trial records based on artificial intelligence according to claim 2, characterized in that: In S1, a data collection strategy combining real-time collection with timed incremental collection, and combining distributed crawlers with API interfaces, is adopted for archive data collection. Specifically: Key milestone files are collected in real time and include protocol approval documents, final version of clinical trial protocol, sample size calculation basis, informed consent form signature page, informed consent process record, ethical review opinions, ethical approval documents and annual review report; Regular files are collected incrementally over a periodic period. Regular files include revised clinical trial protocols, trial flowcharts, case reports, laboratory test records, ethics review application forms, and supplementary materials. Structured data from various archives is collected in real time via API interface, while unstructured data is collected periodically using distributed crawlers. The collected archive data is stored in partitions according to archive type, clinical trial project, and time dimension.

4. The method for quality control and risk warning of drug clinical trial records based on artificial intelligence according to claim 1, characterized in that: In S2, preprocessing operations include data cleaning, standardization, normalization, missing value imputation, and outlier removal, specifically: S21. Use regular expressions to remove symbols, whitespace characters, and duplicate content from text files, remove duplicate case report records, and retain the latest version; for numerical files, remove data with incorrect formatting. S22. Convert all text files to UTF-8 encoding, segment them according to sentence level, and treat each sentence as a text sequence; for numerical files, use Z-score standardization to convert the values ​​into standard normal distribution data with a mean of 0 and a variance of 1. S23. Use the minimum-maximum normalization method to map the standardized data to the [0,1] interval; S24. An improved KNN algorithm is used to handle missing values, and neighbor samples are selected based on feature similarity. The Manhattan distance is used as the distance metric, and the calculation formula is as follows: ； in, , For the sample ,sample eigenvectors, For the total number of features, For the sample The 1 eigenvalue, For the sample The One eigenvalue; S25. To remove outliers using the IQR method, first calculate the quartiles of the dataset. and Then calculate the interquartile range (IQR) and make the following judgment: like or If the data meets the judgment criteria, it will be identified as an outlier and deleted. This is the normalized data.

5. The method for quality control and risk warning of drug clinical trial records based on artificial intelligence according to claim 1, characterized in that: S3 specifically refers to: The preprocessed text sequence is input into the BERT model, and the output of the last hidden layer is extracted as the text semantic feature vector. Simultaneously, the TF-IDF algorithm is used to calculate the weights of keywords in the text, and the top keywords with the highest TF-IDF values ​​are selected. Statistical feature vectors are constructed from keywords. The semantic feature vectors are then concatenated with the statistical feature vectors to obtain a subset of text features. ; Numerical features of word frequency were extracted from numerical archive data. These features included data integrity ratio, completion standardization, logical consistency score, and temporal continuity score. The data integrity ratio was the ratio of the number of completed fields to the total number of fields. Completion standardization was calculated by the proportion of fields whose completion format met requirements. The logical consistency score was obtained by verifying the logical relationships between related fields. The temporal continuity score was obtained by evaluating the continuity of key time points. The resulting subset of numerical features was denoted as... ; Process features are constructed based on the completion status of key milestones in clinical trials. The feature value for each key milestone is calculated based on its completion status and completion time. Completion status is categorized as completed, incomplete, and overdue. The completion time score is calculated by the deviation between the actual completion time and the planned completion time. The resulting subset of process features is denoted as... Each key node corresponds to two feature values: completion status and completion time score. Compliance features constructed in accordance with GCP regulations and FDA standards are denoted as follows: Subset of text features Numerical feature subsets Process feature subset Compliance feature subset Feature fusion is performed to obtain an initial feature set. .

6. The method for quality control and risk warning of drug clinical trial records based on artificial intelligence according to claim 5, characterized in that, In S3, a method combining mutual information entropy and recursive feature elimination is used for feature selection, specifically: S31. Calculate the initial feature set The mutual information entropy between each feature and the quality label measures the correlation between the feature and the quality assessment result. The formula for calculating the mutual information entropy is: ； in, The mutual information entropy value, For the initial feature set, For a set of quality labels, Features With tags The joint probability distribution, Features Marginal probability distribution, For tags The marginal probability distribution shows that the larger the mutual information entropy value, the stronger the correlation between the feature and the quality label. S32. Using mutual information entropy as the initial sorting criterion, a recursive feature elimination algorithm is used to iteratively filter features. In each iteration, a temporary model is trained, and the importance score of each feature is calculated. In each iteration, 5% of the features with the lowest importance scores are removed, and the top 30%-40% of the features with the highest importance scores are retained to form the optimal feature set. .

7. The method for quality control and risk warning of drug clinical trial records based on artificial intelligence according to claim 1, characterized in that, In S4, the quality assessment model uses a deep learning model based on the fusion of CNN and Transformer. The quality assessment model includes an input layer, a convolutional module, a Transformer encoder module, a fully connected module, and an output layer. The input layer receives the optimal feature set. The convolution module includes two convolutional layers and two pooling layers to capture the local dependencies of features. The Transformer encoder module includes Z coding layers, each of which includes a multi-head attention sublayer and a feedforward neural network sublayer. The output of each coding layer uses residual connections and layer normalization. The fully connected module consists of two fully connected layers. The first fully connected layer FC1 uses ReLU as the activation function, and the second fully connected layer FC2 uses Sigmoid as the activation function. The output layer multiplies the output of the second fully connected layer FC2 by 100 to obtain the archive quality quantification score. The scoring range is [0, 100], and the higher the score, the better the file quality.

8. The method for quality control and risk warning of drug clinical trial records based on artificial intelligence according to claim 7, characterized in that, The first convolutional layer Conv1 of the quality assessment model has 64 convolutional kernels, a kernel size of 3×3, a stride of 1, and uses the same padding method. The activation function is ReLU. The first pooling layer Pool1 uses max pooling, with a pooling window size of 2×2 and a stride of 2. The second convolutional layer, Conv2, has 128 kernels, a kernel size of 3×3, a stride of 1, and uses the same padding method. The activation function is ReLU. The second pooling layer, Pool2, uses max pooling with a pooling window size of 2×2 and a stride of 2. The output of Pool2 is flattened and converted into a one-dimensional vector.

9. The method for quality control and risk warning of drug clinical trial records based on artificial intelligence according to claim 1, characterized in that, In S5, the risk warning model adopts a deep learning model that integrates LSTM and attention mechanism. The risk warning model includes an input layer, an embedding layer, an LSTM layer, an attention layer, a fully connected layer, and an output layer. The input layer receives the quality score output by the quality assessment model. With the optimal feature set The embedding layer maps the input vector to a low-dimensional dense vector; The LSTM layer consists of 3 hidden layers, with a dropout rate of 0.3 and a recurrent dropout rate of 0.1 to prevent overfitting. The attention layer employs a self-attention mechanism, calculating the attention weights of the hidden states at each time step. The attention score is calculated using the following formula: ； in, For the first The query vector at the time step and the _th Attention score between key vectors at each time step For the first Query vector at each time step For the first The key vector at each time step Given a vector dimension, the attention weights are calculated using the following formula: ； in, The total length of the input sequence; Normalization using the Softmax function, attention output ,in For the first A vector of values ​​for each time step; The fully connected layer consists of two fully connected layers. The first fully connected layer FC3 uses ReLU as the activation function; the second fully connected layer FC4 uses Softmax as the activation function and outputs four prediction probability values, corresponding to four levels: low risk, medium risk, high risk, and very high risk. The output layer outputs the risk level with the highest probability value and calculates the risk score: ； in, , , , These are the predicted probabilities for low risk, medium risk, high risk, and extremely high risk levels, respectively. , , , The highest scores are for low risk, medium risk, high risk, and extremely high risk, respectively.

10. The method for quality control and risk warning of drug clinical trial records based on artificial intelligence according to claim 1, characterized in that, In S6, early warning information includes risk level, type of file involved, scope of impact, and emergency handling time limit. Early warning information is pushed out according to risk level. Extremely high risk is pushed in real time through SMS, email, and system pop-up window to ensure that relevant personnel receive it as soon as possible; high risk is pushed through SMS and email within 24 hours; medium risk is pushed through email within 48 hours. At the same time, the system records the reception status and processing progress of early warning information, and early warning information that is not processed in time will be reminded again.

Images