A risk assessment-based intelligent supervision method and system for clinical trial institutions

Abstract

This invention discloses an intelligent regulatory method and system for clinical trial institutions based on risk assessment, relating to the field of drug clinical trial regulatory technology. The method includes: collecting registration information, project information, and historical inspection records of clinical trial institutions; intelligently mapping problem texts to form a structured dataset; categorizing discovered problems into five quality risk dimensions according to the responsible party and calculating risk indices; generating a five-dimensional risk radar chart for visualization; formulating differentiated regulatory plans based on comprehensive risk scores; intelligently generating regulatory schemes and allocating inspection tasks; and implementing closed-loop tracking management of inspection problems. This invention solves the technical problems of resource mismatch, lack of scientific basis for plans, and low efficiency in problem classification in existing regulatory models by constructing a five-dimensional quality risk model, introducing time decay factors and project complexity coefficients, and using the Hungarian algorithm to optimize task allocation, thus achieving risk-oriented and intelligent regulation of clinical trial institutions.

Description

Technical Field

[0001] This invention relates to the field of drug clinical trial supervision technology, specifically to an intelligent supervision method and system for clinical trial institutions based on risk assessment. Background Technology

[0002] Drug clinical trials are systematic studies conducted on human subjects to determine the efficacy and safety of drugs. They are an important step in drug development and a crucial link in drug registration and market launch. The system for accrediting drug clinical trial institutions has been changed to a registration system, with regulatory power delegated to provincial drug administrations, which now conduct routine supervision and inspections of these institutions.

[0003] However, the existing regulatory model has the following technical problems:

[0004] There is a serious mismatch between regulatory resources and workload: the provincial drug administration lacks sufficient full-time regulatory personnel with GCP expertise, while the province has a large number of clinical trial institutions and projects, making it difficult to effectively cover daily supervision.

[0005] The formulation of regulatory plans lacks scientific basis: Traditional regulatory plans rely on human experience and subjective judgment, making it difficult to allocate regulatory resources according to the risk levels of different institutions, resulting in insufficient regulation of high-risk institutions and excessive regulation of low-risk institutions.

[0006] The efficiency of problem classification and risk identification is low: the problems found during on-site inspections are recorded and classified manually, which is time-consuming and labor-intensive. It is also difficult to systematically associate and map the problem text with the key points of regulatory verification, and it is impossible to achieve structured and digital processing of problems.

[0007] Lack of continuous risk tracking and visualization capabilities: For institutions with historical problems, it is difficult to quickly assess the rectification of previous issues when changing inspection experts; the quality risks of institutions lack intuitive visualization methods.

[0008] The lack of an intelligent matching mechanism for expert task allocation: When experts are selected for on-site inspections, it is impossible to accurately match them based on their professional expertise and the risk characteristics of the inspection content, which affects the quality and efficiency of the inspection.

[0009] Therefore, developing a method and system that enables intelligent and risk-oriented supervision of clinical trial institutions has significant practical and technological value. Summary of the Invention

[0010] This invention provides a method and system for intelligent supervision of clinical trial institutions based on risk assessment. By constructing a five-dimensional quality risk model, calculating a comprehensive risk score, intelligently allocating inspection tasks, and implementing closed-loop tracking management, it solves problems such as resource mismatch, lack of scientific basis for plans, and low efficiency in problem classification in the existing supervision model.

[0011] To achieve the above objectives, the present invention provides the following technical solution: a method for intelligent supervision of clinical trial institutions based on risk assessment, comprising the following steps:

[0012] Step S1: Collect the registration information, project information, and historical supervision and inspection records of clinical trial institutions, and intelligently classify and map the text of problems found during the inspection according to the preset verification point classification system to form a structured problem dataset;

[0013] Step S2: The problems found in the on-site inspections in the structured problem dataset are categorized into five quality risk dimensions according to the responsible parties. The five quality risk dimensions include: institutional process management risk, ethics committee management risk, professional group process management risk, drug management risk, and researcher management risk.

[0014] Step S3: Using a semi-quantitative analysis method, calculate the risk index for each risk dimension. The formula for calculating the risk index is as follows:

[0015]

[0016] in, For the first Risk index in multiple dimensions For the first The total number of verification points involved in each dimension For the first Weight coefficients for class problems For the first Dimension 1 Frequency of occurrence of this type of problem For the first Dimension 1 The severity coefficient of the problem type;

[0017] Step S4: Using the calculated five risk dimension indices as coordinate values, generate a radar chart of the quality risk of clinical trial institutions to visualize the risk status;

[0018] Step S5: Based on the comprehensive risk scores of each clinical trial institution, formulate a differentiated intelligent supervision plan. The comprehensive risk score is calculated by weighting the risk index of each dimension, the time decay factor and the project complexity coefficient.

[0019] Step S6: Based on the intelligent supervision plan, generate a supervision scheme and assign inspection tasks;

[0020] Step S7: Intelligent processing and closed-loop tracking management of the problems found during the inspection.

[0021] Preferably, the intelligent point mapping in step S1 adopts a text classification algorithm based on the BERT pre-trained model, including: encoding the question text, performing semantic similarity matching with the standard expressions in the verification point library, and outputting the most matching verification point number and confidence score; when the confidence score is lower than a preset threshold, the question is marked as pending manual review.

[0022] Preferably, in step S2, the problems found during on-site inspections are categorized into five quality risk dimensions according to the responsible party, as follows:

[0023] Institutional process management risks correspond to the quality management system, institutional qualifications and conditions, overall management of investigational drugs, data and archive management, and organizational management requirements of clinical trial institutions.

[0024] The management of risks by the ethics committee corresponds to the requirements related to the independence, compliance, completeness, and oversight responsibilities of ethical review;

[0025] The risks associated with the professional group's process management correspond to the matching degree of the principal investigator's qualifications, the construction of the professional group's quality control system, personnel training management, and the relevant requirements for the implementation of the clinical trial projects undertaken.

[0026] Drug management risks correspond to the requirements for the entire process of receiving, storing, distributing, recycling, and destroying investigational drugs;

[0027] Project execution management risks correspond to the requirements related to the researcher's fulfillment of responsibilities, informed consent process, subject screening and enrollment, trial protocol execution, safety event recording, and trial data recording and reporting.

[0028] Preferably, the weighting coefficient in step S3 The determination of severity is achieved using the analytic hierarchy process (AHP), which includes: constructing a judgment matrix, calculating the weight values ​​of indicators at each level, and calculating the consistency ratio (CR). When CR < 0.1, the weight allocation is considered consistent. The severity coefficients for the issues are set as follows: high-risk factors correspond to a severity coefficient of 3, medium-risk factors correspond to a severity coefficient of 2, and low-risk factors correspond to a severity coefficient of 1.

[0029] Preferably, the radar chart in step S4 is generated as follows: using the five-dimensional quality risk index as the radius value, five coordinate axes are drawn on the radar chart at equal angular intervals, representing institutional process management risk, ethics committee management risk, professional group process management risk, drug management risk, and researcher management risk, respectively. The index values ​​of each dimension are marked on the corresponding coordinate axes and then connected sequentially to form a risk contour polygon. The larger the area of ​​the risk contour polygon, the more prominent the quality risk of the institution. The more irregular the shape of the risk contour polygon, the more uneven the risk distribution of the institution.

[0030] Preferably, the formula for calculating the comprehensive risk score in step S5 is:

[0031]

[0032] in, For comprehensive risk scoring; For the first The weighting coefficients for each risk dimension satisfy... ; For the first Risk index in multiple dimensions; The time decay factor is calculated using the following formula: ,in The time interval since the last inspection. This is the attenuation coefficient, with a value ranging from 0.05 to 0.15. The project complexity coefficient is calculated using the following formula: ,in The total number of projects undertaken by the organization. The proportion of innovative drug projects; , These are the corresponding weighting coefficients;

[0033] Based on the comprehensive risk score The numerical range of S categorizes clinical trial institutions into three regulatory levels: when S ≥ S high At that time, it was a high-risk level, when S low ≤S high At that time, the risk level was medium, when S low The time is a low-risk level; the S high and S low The preset threshold for hierarchical division.

[0034] ​​Preferably, the differentiated intelligent supervision plan in step S5 includes: setting differentiated supervision frequencies for institutions with different risk levels, setting high-risk institutions to be inspected at least once every six months, medium-risk institutions to be inspected once a year, and low-risk institutions to be inspected once every two years; when a specific risk dimension index of an institution exceeds a preset single warning threshold, a special supervision plan is generated in addition to the regular supervision plan.

[0035] Preferably, step S6 further includes the following sub-steps:

[0036] S6.1: Based on the scope of the inspection in the regulatory plan, the inspection content shall be divided into three categories according to document type: security documents, validity documents, and general documents;

[0037] S6.2: Based on the risk assessment standards for each inspection item, mark high-risk factors, medium-risk factors, and low-risk factors;

[0038] S6.3: Retrieve the expert's professional direction tags from the expert database, calculate the matching degree between the expert's expertise and the risk characteristics of the inspection content, and use the matching degree maximization algorithm to solve the optimal task allocation scheme;

[0039] S6.4: Automatically estimate the required inspection time based on the level of the assigned risk factors and the workload.

[0040] Preferably, the mathematical model for optimal task allocation in step S6.3 is:

[0041] Objective function:

[0042] Constraints:

[0043]

[0044]

[0045] ∈{0,1}, ,

[0046] in, To check the number of task items, Number of available experts; For experts For the task Match score; For binary decision variables, it represents whether to perform the task. Assigned to experts ;

[0047] The matching score The calculation formula is:

[0048]

[0049] in, The similarity between the expert's professional expertise and the task content. This represents the current level of busyness of the experts. Rate the expert's examination experience. , , The corresponding weight coefficients and + + =1.

[0050] Preferably, step S7 further includes the following sub-steps:

[0051] S7.1: During on-site inspections, problems discovered are recorded via mobile terminals. The system automatically recommends matching verification point numbers based on semantic matching. After the inspectors confirm the information, the problems are archived.

[0052] S7.2: Establish a problem lifecycle tracking mechanism to record the problem's discovery time, responsible party, rectification deadline, rectification status, and review results;

[0053] S7.3: After the inspected organization submits a response to the problem rectification, the system automatically associates the response content with the original problem and conducts an intelligent preliminary review based on the preset rectification and acceptance rules;

[0054] S7.4: Automatically issue warnings for problems that are not rectified on time or are not rectified to a satisfactory standard, and implement escalation procedures based on the risk level of the problem;

[0055] S7.5: For the same type of problem that recurs in different inspection cycles, the system automatically marks it as a systemic defect and increases the weight coefficient of its corresponding risk dimension.

[0056] Preferably, a risk assessment-based intelligent regulatory system for clinical trial institutions includes:

[0057] The project management module is used to connect to the national drug or medical device filing platform, automatically synchronize clinical trial project filing information, and establish the relationship between the project and the clinical trial institution.

[0058] The institution management module is used to establish a database of clinical trial institutions based on data from the filing platform, and to link all clinical trial projects undertaken by each institution, forming a three-level association between institution, project, and issue.

[0059] The risk assessment engine module has a built-in problem classification model based on the key points of the verification, which is used to automatically calculate the five-dimensional quality risk index and generate a risk radar chart.

[0060] The intelligent regulatory plan module is used to automatically generate differentiated regulatory plans based on the comprehensive risk score;

[0061] The intelligent supervision solution and task allocation module is used to classify the inspection content and automatically allocate inspection tasks based on expert professional tags.

[0062] The issue tracking and closed-loop management module is used to establish issue lifecycle tracking records;

[0063] The regulatory work summary and data visualization module is used to generate multi-dimensional regulatory data statistical reports;

[0064] The data layer module includes an institutional database, a project database, an expert database, a problem knowledge base, and a legal and standard database.

[0065] Preferably, a computer-readable storage medium stores a computer program thereon, which, when executed by a processor, implements a risk assessment-based intelligent monitoring method for clinical trial institutions.

[0066] Preferably, an electronic device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement a risk assessment-based intelligent regulatory method for clinical trial institutions.

[0067] The beneficial effects of this invention are as follows:

[0068] This invention constructs a five-dimensional quality risk assessment model, comprehensively calculating risk indices across five dimensions: institutional process management risk, ethics committee management risk, professional group process management risk, drug management risk, and investigator management risk. Furthermore, it introduces a time decay factor and project complexity coefficient to form a comprehensive risk score, achieving a quantitative assessment of the risk level of clinical trial institutions. Based on the comprehensive risk score, institutions are divided into high, medium, and low regulatory levels, with differentiated regulatory frequencies. This ensures that high-risk institutions receive increased supervision, while low-risk institutions receive fewer unnecessary inspections. Compared to the traditional model of inspecting all institutions once a year, this saves approximately 60% of regulatory manpower, significantly improving the efficiency of regulatory resource utilization.

[0069] This invention generates a risk radar chart using a five-dimensional quality risk index as coordinate values. The area of ​​the risk outline polygon represents the overall risk level, and the degree of irregularity of the shape represents the balance of risk distribution. It can intuitively show the performance differences of various institutions in different risk dimensions, making it easier for regulators and institutional managers to quickly identify risk shortcomings and providing visual decision support for the formulation of targeted regulatory plans.

[0070] This invention employs a text classification algorithm based on a BERT pre-trained model to automatically map unstructured inspection question text to a pre-defined verification point classification system, outputting confidence scores and implementing a manual review fallback mechanism for low-confidence questions. Practical verification shows that the automated classification rate can reach over 90%, improving efficiency by approximately 15 times compared to traditional manual classification methods. Simultaneously, it achieves structured storage of question data, laying a data foundation for subsequent risk analysis and data mining.

[0071] This invention introduces a time decay factor, causing the contribution of historical issues to the current risk score to decrease over time, thus enabling dynamic updates to the risk score and avoiding the unreasonable phenomenon of "permanently labeling" institutions with historical issues. Simultaneously, it establishes a closed-loop tracking mechanism covering the entire lifecycle from issue discovery, rectification submission, intelligent preliminary review, overdue warnings, to systemic defect identification, ensuring that every issue is effectively rectified and managed in a closed-loop manner. For recurring similar issues, the system automatically marks them as systemic defects and increases the weight coefficient of the corresponding risk dimension, forming a continuous improvement regulatory loop.

[0072] This invention constructs a matching score model based on experts' professional expertise, workload, and experience, and uses the Hungarian algorithm to solve for the optimal task allocation scheme, achieving precise matching between inspection tasks and experts' professional strengths. This method ensures that high-risk inspection content is handled by experts with the corresponding professional background, solving the problems of arbitrary expert selection and low professional matching in the traditional model, and effectively improving the quality and efficiency of on-site inspections.

[0073] This invention constructs a complete and scalable intelligent regulatory technology solution by establishing a structured classification system for key verification points, a standardized five-dimensional risk model, a quantitative risk index calculation method, and a reusable expert matching algorithm. This solution is not only applicable to the supervision and inspection of clinical trial institutions by provincial drug regulatory authorities, but can also be extended to related fields such as the supervision of medical device clinical trials and the inspection of Good Manufacturing Practices (GMP) for pharmaceuticals, demonstrating good technology transferability and promising industrial application prospects. Attached Figure Description

[0074] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0075] Figure 1 This is a diagram of the overall architecture of the present invention;

[0076] Figure 2 This is a schematic diagram illustrating the structure of the five-dimensional quality risk assessment model of this invention;

[0077] Figure 3 This is an overall flowchart of the intelligent monitoring plan formulation and task allocation of the present invention;

[0078] Figure 4 This is a flowchart illustrating the intelligent problem handling and closed-loop tracking management of the present invention.

[0079] Figure 5 This is a detailed flowchart of the intelligent problem point mapping in step S1 of the present invention;

[0080] Figure 6 A flowchart illustrating the algorithm for intelligent task allocation in this invention;

[0081] Figure 7 This is a state transition diagram for the lifecycle tracking of the problem in this invention;

[0082] Figure 8 This is a diagram showing the system deployment and data interaction relationship of the present invention. Detailed Implementation

[0083] The technical solution of the present invention will now be clearly and completely described with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0084] Example 1

[0085] This embodiment uses the example of a provincial drug regulatory authority (hereinafter referred to as the provincial authority) implementing intelligent supervision over 32 drug clinical trial institutions (hereinafter referred to as GCP institutions) within its jurisdiction to illustrate the technical solution of the present invention in detail.

[0086] I. System deployment and data initialization, such as Figure 1 , Figure 8 As shown.

[0087] First, the intelligent regulatory system for clinical trial institutions based on risk assessment, as described in this invention, was deployed at the provincial level. The system connects to the national drug / medical device registration platform through a project management module, automatically synchronizing the registration information of 32 GCP institutions in the province, including institution name, registration number, specialty, and registration date.

[0088] Meanwhile, the system imported historical supervision and inspection records from each institution over the past three years (January 2021 to December 2023) through the institution management module, totaling 342 texts of issues discovered during on-site inspections. Each issue record includes information such as inspection time, issue description, responsible party, and rectification status.

[0089] The system stores the above data in the organization database and project database of the data layer module, forming a basic dataset with a three-level association between organization, project, and issue.

[0090] II. Intelligent problem mapping, i.e., step S1.

[0091] like Figure 5 As shown, the system activates the risk assessment engine module to perform intelligent point mapping on 342 historical question texts. The system uses a text classification algorithm based on a BERT pre-trained model to encode each question text and then perform semantic similarity matching with the preset verification point standard expressions in the question knowledge base. The preset verification point classification system is constructed according to the "Key Points and Judgment Principles for Supervision and Inspection of Drug Clinical Trial Institutions (Trial Implementation)" and contains 87 verification point numbers and corresponding standard expressions.

[0092] For example, consider the following question text: Researchers failed to record subjects' concomitant medication use as required by the protocol. The system encodes this information and matches it against the checkpoint database, outputting the most matching checkpoint number as 3.3.4 with a confidence score of 0.92. Since this confidence score is higher than a preset threshold (0.85 in this embodiment), the system automatically categorizes the issue into the researcher management risk dimension.

[0093] For issues with a confidence level below 0.85, the system marks them as requiring manual review, and the issues are then manually confirmed by supervisors to complete the categorization. In this embodiment, 31 out of 342 issues require manual review, while the remaining 311 issues are automatically categorized by the system, achieving an automation rate of 90.9%.

[0094] After the aggregation is completed, the system generates a structured problem dataset, such as Figure 5 As shown, each issue record contains the following fields: issue ID, issue text, referral number, occurrence time, responsible organization, rectification status, and confidence score.

[0095] III. Construction of the five-dimensional quality risk model and calculation of the risk index, namely steps S2 and S3.

[0096] like Figure 2 As shown, according to step S2, the system categorizes the problems found during on-site inspections into five quality risk dimensions based on the responsible party. In this embodiment, the specific correspondence between the five dimensions is as follows:

[0097] Institutional process management risks: These correspond to the quality management system, qualifications and conditions, overall management of investigational drugs, data and archive management, and organizational management requirements of clinical trial institutions. For example, issues such as whether the institution has the qualifications to conduct clinical trials (1.1), whether the institution has established a quality management system (1.5), and whether the investigational drug management documents are complete (4.2) fall under this dimension.

[0098] Ethics Committee Management Risks: This corresponds to the requirements related to the independence, compliance, completeness, and oversight responsibilities of ethics reviews. For example, issues such as whether the ethics committee is registered (2.1) and whether the ethics review documents are complete (2.2) are included in this dimension.

[0099] Professional group process management risks: These correspond to the professional group's quality control system construction, personnel training management, and clinical trial process execution requirements. For example, issues such as whether the professional group has the corresponding conditions (check point number 1.6) and whether researchers have been trained and authorized (check point number 3.1.2) fall under this dimension.

[0100] Drug management risks correspond to the requirements related to the entire process of receiving, storing, distributing, recycling, and destroying investigational drugs. For example, issues such as whether the receiving records of investigational drugs are complete (check point number 4.1) and whether the storage conditions of investigational drugs meet the requirements (check point number 4.3) fall under this dimension.

[0101] Researcher risk management corresponds to the requirements related to the researcher's fulfillment of responsibilities, informed consent process, subject screening and enrollment, trial protocol execution, safety event recording, and trial data recording and reporting. For example, issues such as whether the informed consent form is signed correctly (check point number 3.2.2) and whether the adverse event records are complete (check point number 3.3.2) fall under this dimension.

[0102] The system calculates a five-dimensional risk index for each institution. Taking institution A as an example, 23 issues were found in the past three years of supervision and inspection, distributed across the five dimensions mentioned above.

[0103] The system executes step S3, using a semi-quantitative analysis method to calculate the risk index for each dimension. The calculation formula is:

[0104]

[0105] in, The total number of verification points involved in this dimension is automatically calculated by the system based on the issue knowledge base. For the first Weight coefficients for class problems For the first Dimension 1 Frequency of occurrence of this type of problem For the first Dimension 1 Severity coefficient of the problem

[0106] In this embodiment, the weighting coefficient The determination was made using the Analytic Hierarchy Process (AHP). The specific steps are as follows:

[0107] The first step is to construct a judgment matrix. Three GCP inspection experts compare the importance of each checkpoint pairwise using a 1-9 scale, where 1 represents equal importance and 9 represents extreme importance. The geometric mean of the scores is then used to obtain the judgment matrix.

[0108] The second step is to calculate the weight vector. The judgment matrix is ​​normalized, and the weight coefficients for each verification point are calculated. .

[0109] The third step is consistency verification. The consistency ratio CR of the judgment matrix is ​​calculated. In this embodiment, CR = 0.06 < 0.1, indicating that the weight allocation is consistent and the weight coefficients are valid.

[0110] Severity coefficient The risk level of the problem is set as follows: high-risk factors, such as falsification and failure to report serious adverse events, correspond to 3; medium-risk factors, such as non-standard records and missing documents, correspond to 2; and low-risk factors, such as format errors and missing signatures, correspond to 1.

[0111] Frequency of occurrence This is calculated by dividing the number of times this type of problem occurs under this dimension by the total number of inspections conducted by the organization. The calculated five-dimensional risk indices for organization A are as follows:

[0112] Risk Dimensions Risk Index Ri Institutional process management risks 0.32 Ethics Committee manages risks 0 Professional team process management risks 0.5 Drug management risks 0.25 Researchers manage risks 0.68

[0113] IV. Risk radar chart generation, i.e., step S4.

[0114] The system executes step S4, using the calculated five risk dimension indices as coordinate values ​​to generate a quality risk radar chart for organization A.

[0115] The specific generation method is as follows: using a five-dimensional quality risk index as the radius value, with equal angular intervals and an angle of 72° between adjacent coordinate axes, five coordinate axes are plotted on the radar chart to represent institutional process management risk, ethics committee management risk, professional group process management risk, drug management risk, and researcher management risk, respectively. The index values ​​of each dimension are marked on the corresponding coordinate axes, and then the marked points are connected sequentially to form a closed risk contour polygon.

[0116] In this embodiment, the radar chart of Institution A shows that the markers on the two coordinate axes of researcher management risk 0.68 and professional group process management risk 0.50 are significantly prominent, and the risk outline polygon is convex in this direction; the marker on the coordinate axis of ethics committee management risk 0.00 is at its minimum value, and the polygon is concave in this direction.

[0117] This radar chart visually demonstrates to regulators and institutional managers that Institution A's core risks lie in two areas: researcher compliance and the establishment of a professional group quality control system. These should be the focus of subsequent monitoring and inspection.

[0118] V. Comprehensive risk assessment and intelligent supervision plan development, i.e., step S5

[0119] The system executes step S5, first calculating the comprehensive risk score for each institution. The formula for calculating the comprehensive risk score is:

[0120]

[0121] In this embodiment, the values ​​of each parameter are as follows:

[0122] The weighting coefficients for the five risk dimensions. Determined using the analytic hierarchy process (AHP) in this embodiment. The risk levels were set according to their importance as follows: Investigator management risk 0.30, Professional group process management risk 0.25, Institutional process management risk 0.20, Drug management risk 0.15, Ethics Committee management risk 0.10, meeting the requirements. .

[0123] The risk indices for each dimension are calculated in step S3.

[0124] Time decay factor. The calculation formula is as follows: ,in The time interval since the last inspection, in months. This is the attenuation coefficient. In this embodiment... A value of 0.10 indicates that the longer the time since the last inspection, the smaller the contribution of historical issues to the current risk score. For example, if an organization's last inspection was 12 months ago, then... .

[0125] Project complexity coefficient. The calculation formula is as follows: ,in The total number of projects undertaken by the organization. This represents the percentage of innovative drug projects. In this example, it represents the total number of projects undertaken by an institution. The proportion of innovative drug projects ,but .

[0126] : The weighting coefficient of the time decay factor, which is 0.15 in this embodiment.

[0127] : The weighting coefficient of the project complexity coefficient, which is 0.10 in this example.

[0128] like Figure 3 As shown, the comprehensive risk scores of the 32 institutions were calculated. The values ​​range from 0.15 to 0.82. Based on the preset hierarchical division threshold, in this embodiment... , The 32 institutions were divided into three regulatory tiers:

[0129] High risk level 6 institutions;

[0130] Medium risk level 14 institutions;

[0131] Low risk level 12 institutions.

[0132] The system automatically generates an annual intelligent regulatory plan based on the hierarchical division and the deviation of each institution from its specific risk dimensions.

[0133] High-risk institutions: inspected at least once every six months, for a total of 12 inspections per year;

[0134] Medium-risk level institutions: inspected once a year, with a total of 14 inspection tasks throughout the year;

[0135] Low-risk institutions: Inspections will be conducted every two years, with 6 institutions scheduled for inspection this year.

[0136] A total of 32 inspections were conducted throughout the year. Compared to the traditional regulatory model of inspecting all institutions once a year, this plan saves approximately 60% of regulatory manpower while achieving doubled supervision of high-risk institutions.

[0137] Furthermore, for Institution C, with a comprehensive risk score of 0.55, classifying it as a medium-risk institution, its drug management risk dimension index is 0.72, exceeding the preset single-item warning threshold of 0.60. In addition to the regular annual inspection plan, the system automatically generates a special regulatory plan for the management of investigational drugs, such as... Figure 3 As shown.

[0138] VI. Intelligent generation of regulatory plans and allocation of inspection tasks, i.e., step S6

[0139] The system executes step S6, taking a special inspection of a high-risk institution A as an example, to explain in detail the intelligent generation of the regulatory plan and the task allocation process.

[0140] S6.1: Category of Inspection Content

[0141] Based on the scope of the regulatory plan, the system categorizes the inspection content into three types according to document type:

[0142] Safety documents include adverse event records, serious adverse event reports, program deviation records, safety data summary tables, etc., totaling 35 items in this embodiment.

[0143] Validity documents include efficacy assessment records, endpoint event determinations, case report forms, etc., totaling 28 items in this embodiment.

[0144] General documents include informed consent forms, researcher resumes, training records, equipment maintenance records, etc., totaling 42 items in this embodiment.

[0145] S6.2: Risk Factor Labeling

[0146] The system automatically labels the risk factor level based on the risk assessment criteria for each inspection item:

[0147] There are 12 high-risk factors, including the standardization of informed consent signing, underreporting of adverse events, deviation from protocol implementation, management of investigational drugs, and verification of data authenticity.

[0148] There are 28 medium-risk factors, including record integrity, document archiving standardization, equipment calibration, training records, etc.

[0149] There are 65 low-risk factors, including file format and signature integrity.

[0150] S6.3: Intelligent task allocation, such as... Figure 6 As shown.

[0151] The system retrieves available GCP (Good Clinical Practice) examination experts from the expert database. In this embodiment, the expert database contains 25 experts, each with professional category tags such as oncology, cardiovascular, neurology, drug analysis, medical ethics, etc., as well as attributes such as workload coefficient and examination experience score.

[0152] The matching score between the expert's expertise in system calculation and the risk characteristics of the inspection content. The calculation formula is:

[0153]

[0154] In this embodiment, the weighting coefficient is set to , , This indicates that the degree of professional matching is the most important consideration.

[0155] For the task of verifying the compliance of informed consent signing, which is a high-risk factor, i.e., task T1, the system calculates the matching score for each expert:

[0156] Expert A, whose professional focus is medical ethics, has a busyness score of 0.30, an experience score of 0.80, a Sim score of 0.95, and a M score of 0.60×0.95+0.20×0.70+0.20×0.80=0.57+0.14+0.16=0.87.

[0157] Expert B, specializing in oncology, has a workload of 0.50, an experience score of 0.90, a Sim score of 0.60, and a M score of 0.60×0.60+0.20×0.50+0.20×0.90=0.36+0.10+0.18=0.64.

[0158] Expert C, specializing in drug analysis, has a workload of 0.20, an experience score of 0.70, a Sim score of 0.30, and a M score of 0.60×0.30+0.20×0.80+0.20×0.70=0.18+0.16+0.14=0.48.

[0159] The system uses the Hungarian algorithm to solve the matching degree matrix above to obtain the optimal task allocation scheme, ensuring that each task is assigned to the expert with the highest matching degree, while the workload of each expert is relatively balanced.

[0160] For the inspection of Institution A, the system ultimately assigned 3 experts: Expert A was responsible for the informed consent and ethics-related inspections, Expert D was responsible for the efficacy evaluation and protocol implementation inspections, and Expert E was responsible for the overall quality control inspections.

[0161] S6.4: Inspection Time Estimation

[0162] The system automatically estimates the required inspection time based on the risk factor level and workload assigned to the expert.

[0163] High-risk factors: 30 minutes for each estimate;

[0164] Medium-risk factors: 15 minutes for each estimate;

[0165] Low-risk factors: 5 minutes for each estimate.

[0166] Taking into account both the organization's size and the project's complexity, an adjustment factor is applied. Since organization A in this example is relatively large, the adjustment factor is 1.2. The estimated total time for this inspection is 2.5 working days, and the system synchronizes this information to the task allocation notification.

[0167] VII. Problem closed-loop tracking management, i.e., step S7

[0168] The system executes step S7 to perform full lifecycle tracking and management of the problems discovered during the inspection, such as... Figure 4 As shown.

[0169] S7.1: On-site problem record.

[0170] Inspection experts record issues they discover on-site using mobile devices. For example, expert A identifies an issue by inputting a description such as, "The informed consent form for subject S-023 was signed after the initial screening date." The system automatically recommends verification point number 3.2.2 based on semantic matching, with a confidence score of 0.94. After expert confirmation, the issue is archived.

[0171] A total of 23 issues were identified during this inspection, which were automatically categorized by the responsible party: 2 issues related to institutional process management, 6 issues related to professional group process management, 13 issues related to researcher management, 2 issues related to drug management, and 0 issues related to ethics committee management.

[0172] S7.2: Establish problem lifecycle tracking, such as Figure 7 As shown.

[0173] The system establishes a lifecycle tracking record for each issue, including the following fields: issue ID, issue discovery time, responsible party, rectification deadline, rectification status, and review result.

[0174] S7.3: Intelligent preliminary review of rectification responses.

[0175] The inspected institution submitted its rectification response through the system within the rectification period. Taking question 3.2.2 as an example, the institution's response included: an explanation of the rectification measures, such as retraining the researchers, re-signing informed consent forms for subject S-023, and supporting materials, such as uploading scanned copies of the re-signed informed consent forms and training records.

[0176] The system invokes the rectification and acceptance rule engine for intelligent preliminary review:

[0177] Integrity check: The system detected that the correspondence between the rectification measures and the discovered problems is complete, and the check passed.

[0178] Standardization check: The system checks the supporting materials in PDF format and they must contain the necessary elements, such as the signing date, researcher's signature, and subject's signature. If they pass the check, the system will be deemed to have passed.

[0179] Timeliness test: The system's rectification response submission time is 12 working days. If it is less than 15 working days, it is considered passed.

[0180] All three tests passed, and the system updated the issue status to "Approved".

[0181] Regarding another issue where an adverse event record was missing, the organization's response was only a textual statement that the record had been supplemented but no supporting documentation was uploaded. The system detected the missing supporting documentation, marked the issue as "Review Failed," and automatically returned the application with the reason for rejection: "Please supplement the supporting documentation for the adverse event record."

[0182] S7.4: Automatic Early Warning and Upgrade Processing

[0183] The system automatically issues warnings for problems that are not rectified on time: 3 days before the deadline for rectification, the system sends a reminder notice to the person in charge of the organization; if no response is submitted by the deadline, the system sends a supervision notice to the person in charge of the organization; if the problem is not rectified within 15 days, the problem is automatically escalated to the department that is using the system.

[0184] S7.5: Systemic Defect Identification and Weight Adjustment

[0185] The system identifies recurring problem types for the same institution across different inspection cycles. In this example, Institution A's failure to promptly record adverse events, discovered during this inspection, shares the same categorization number as a similar problem found in the 2022 inspection. The system automatically marks this problem as a systemic defect and, in the next risk index calculation, incorporates the weighting coefficient of this type of problem under the researcher management risk dimension. Increase by 20%.

[0186] VIII. Summary of Regulatory Work

[0187] The system generates the following statistical reports through the regulatory work summary and data visualization module:

[0188] Risk ranking and trend analysis of 32 GCP institutions in the province;

[0189] Statistics on the frequency of various problems and a map of hot issues;

[0190] Comparative display of risk radar charts for various institutions;

[0191] Statistics on the completion of regulatory tasks and the rate of rectification of problems.

[0192] Example 2

[0193] This embodiment illustrates a variant application scenario of the radar chart in this invention.

[0194] For Institution B, the risk level is medium, with a comprehensive risk score of 0.48. Its five-dimensional risk indices are as follows: institutional process management risk 0.28, ethics committee management risk 0.15, professional group process management risk 0.32, drug management risk 0.55, and researcher management risk 0.35.

[0195] The system generated a risk radar chart using the same method, showing that the institution's risk score in the drug management dimension was 0.55, significantly higher than other dimensions, with the risk contour polygon showing a significant bulge in this direction. This visualization result suggests to regulators that Institution B's core risk is concentrated in the management of investigational drugs, and the regulatory plan should focus on the entire process of drug receipt, storage, distribution, and recall.

[0196] Compared to the multi-dimensional uneven distribution of Institution A in Example 1, its researcher management risk is 0.68, professional group process management risk is 0.50, and other dimensions are relatively low. Institution B has a more regular polygonal shape in terms of risk profile, but the single dimension of drug management risk highlights the direction that needs to be focused on.

[0197] Example 3

[0198] This embodiment illustrates the time decay factor in this invention. Dynamic effects.

[0199] Consider two inspection scenarios for the same organization D:

[0200] Scenario 1: Institution D underwent an inspection in January 2023, receiving a comprehensive risk score of 0.55, classifying it as a medium-risk institution. The institution thoroughly rectified the identified issues, and no similar problems have recurred since. Following traditional regulatory practices, the institution was scheduled for another inspection at the same level in 2024.

[0201] Scenario 2: Application of this invention: When calculating the comprehensive risk score of agency D in January 2024, the system introduces a time decay factor. Since Agency D did not encounter any new major issues within 12 months, the time decay factor reduced the contribution of historical problems, and the overall risk score dropped to 0.38. Although still classified as medium risk, the decreased risk score led the system to lower the agency's inspection priority when developing the 2024 regulatory plan.

[0202] This mechanism achieves dynamic risk adjustment and precise regulatory adaptation, avoids the unreasonable phenomenon of permanently labeling institutions with historical issues, and incentivizes institutions to continuously improve their quality systems.

[0203] Example 4

[0204] This embodiment illustrates the project complexity coefficient in this invention. Its regulatory effect.

[0205] Consider two institutions, E and F:

[0206] Institution E: Total number of projects undertaken M=3, all of which are generic drug projects; innovative drug projects account for Q=0. Therefore... .

[0207] Organization F: Total number of projects undertaken M=8, of which 4 are innovative drug projects, representing Q=0.5%. Therefore... .

[0208] The two institutions have similar risk dimension indices, but Institution F has a higher overall risk score because it undertakes more complex projects. This reflects the objective impact of project complexity on regulatory risk, making the formulation of regulatory plans more aligned with reality.

[0209] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A method for intelligent supervision of clinical trial institutions based on risk assessment, characterized in that, Includes the following steps: Step S1: Collect the registration information, project information, and historical supervision and inspection records of clinical trial institutions, and intelligently classify and map the text of problems found during the inspection according to the preset verification point classification system to form a structured problem dataset; Step S2: The problems found in the on-site inspections in the structured problem dataset are categorized into five quality risk dimensions according to the responsible parties. The five quality risk dimensions include: institutional process management risk, ethics committee management risk, professional group process management risk, drug management risk, and researcher management risk. Step S3: Using a semi-quantitative analysis method, calculate the risk index for each risk dimension. The formula for calculating the risk index is as follows: in, For the first Risk index in multiple dimensions For the first The total number of verification points involved in each dimension For the first Weight coefficients for class problems For the first Dimension 1 Frequency of occurrence of this type of problem For the first Dimension 1 The severity coefficient of the problem type; Step S4: Using the calculated five risk dimension indices as coordinate values, generate a radar chart of the quality risk of clinical trial institutions to visualize the risk status; Step S5: Based on the comprehensive risk scores of each clinical trial institution, formulate a differentiated intelligent supervision plan. The comprehensive risk score is calculated by weighting the risk index of each dimension, the time decay factor and the project complexity coefficient. Step S6: Based on the intelligent supervision plan, generate a supervision scheme and assign inspection tasks; Step S7: Intelligent processing and closed-loop tracking management of the problems found during the inspection.

2. The method according to claim 1, characterized in that, In step S1, the intelligent point mapping adopts a text classification algorithm based on the BERT pre-trained model, including: encoding the question text, performing semantic similarity matching with the standard expressions in the verification point library, and outputting the most matching verification point number and confidence score; when the confidence score is lower than a preset threshold, the question is marked as pending manual review.

3. The method according to claim 1, characterized in that, In step S2, the problems found during the on-site inspection are categorized into five quality risk dimensions according to the responsible party, as follows: Institutional process management risks correspond to the quality management system, institutional qualifications and conditions, overall management of investigational drugs, data and archive management, and organizational management requirements of clinical trial institutions. The management of risks by the ethics committee corresponds to the requirements related to the independence, compliance, completeness, and oversight responsibilities of ethical review; The risks associated with the professional group's process management correspond to the matching degree of the principal investigator's qualifications, the construction of the professional group's quality control system, personnel training management, and the relevant requirements for the implementation of the clinical trial projects undertaken. Drug management risks correspond to the requirements for the entire process of receiving, storing, distributing, recycling, and destroying investigational drugs; Project execution management risks correspond to the requirements related to the researcher's fulfillment of responsibilities, informed consent process, subject screening and enrollment, trial protocol execution, safety event recording, and trial data recording and reporting.

4. The method according to claim 1, characterized in that, The weighting coefficient in step S3 The determination of severity is achieved using the analytic hierarchy process (AHP), which includes: constructing a judgment matrix, calculating the weight values ​​of indicators at each level, and calculating the consistency ratio (CR). When CR < 0.1, the weight allocation is considered consistent. The severity coefficients for the issues are set as follows: high-risk factors correspond to a severity coefficient of 3, medium-risk factors correspond to a severity coefficient of 2, and low-risk factors correspond to a severity coefficient of 1.

5. The method according to claim 1, characterized in that, The radar chart in step S4 is generated as follows: using the five-dimensional quality risk index as the radius value, five coordinate axes are drawn on the radar chart at equal angular intervals, representing institutional process management risk, ethics committee management risk, professional group process management risk, drug management risk, and researcher management risk, respectively. The index values ​​of each dimension are marked on the corresponding coordinate axes and then connected sequentially to form a risk contour polygon. The larger the area of ​​the risk contour polygon, the more prominent the quality risk of the institution. The more irregular the shape of the risk contour polygon, the more uneven the risk distribution of the institution.

6. The method according to claim 1, characterized in that, The formula for calculating the comprehensive risk score in step S5 is as follows: in, For comprehensive risk scoring; For the first The weighting coefficients for each risk dimension satisfy... ; For the first Risk index in multiple dimensions; The time decay factor is calculated using the following formula: ,in The time interval since the last inspection. This is the attenuation coefficient, with a value ranging from 0.05 to 0.

15. The project complexity coefficient is calculated using the following formula: ,in The total number of projects undertaken by the organization. The proportion of innovative drug projects; , These are the corresponding weighting coefficients; Based on the comprehensive risk score The numerical range of S categorizes clinical trial institutions into three regulatory levels: when S ≥ S high At that time, it was a high-risk level, when S low ≤S high At that time, the risk level was medium, when S low The time is a low-risk level; the S high and S low The preset threshold for hierarchical division;​​ The differentiated intelligent supervision plan formulated in step S5 includes: setting differentiated supervision frequencies for institutions with different risk levels, setting high-risk institutions to be inspected at least once every six months, medium-risk institutions to be inspected once a year, and low-risk institutions to be inspected once every two years; when a specific risk dimension index of an institution exceeds the preset single warning threshold, a special supervision plan is generated in addition to the regular supervision plan.

7. The method according to claim 1, characterized in that, Step S6 further includes the following sub-steps: S6.1: Based on the scope of the inspection in the regulatory plan, the inspection content shall be divided into three categories according to document type: security documents, validity documents, and general documents; S6.2: Based on the risk assessment standards for each inspection item, mark high-risk factors, medium-risk factors, and low-risk factors; S6.3: Retrieve the expert's professional direction tags from the expert database, calculate the matching degree between the expert's expertise and the risk characteristics of the inspection content, and use the matching degree maximization algorithm to solve the optimal task allocation scheme; S6.4: Automatically estimate the required inspection time based on the level of the assigned risk factors and the workload.

8. The method according to claim 7, characterized in that, The mathematical model for optimal task allocation in step S6.3 is as follows: Objective function: Constraints: ∈{0,1}, , in, To check the number of task items, Number of available experts; For experts For the task Match score; For binary decision variables, it represents whether to perform the task. Assigned to experts ; The matching score The calculation formula is: in, The similarity between the expert's professional expertise and the task content. This represents the current level of busyness of the experts. Rate the expert's examination experience. , , The corresponding weight coefficients and + + =1.

9. The method according to claim 1, characterized in that, Step S7 further includes the following sub-steps: S7.1: During on-site inspections, problems discovered are recorded via mobile terminals. The system automatically recommends matching verification point numbers based on semantic matching. After the inspectors confirm the information, the problems are archived. S7.2: Establish a problem lifecycle tracking mechanism to record the problem's discovery time, responsible party, rectification deadline, rectification status, and review results; S7.3: After the inspected organization submits a response to the problem rectification, the system automatically associates the response content with the original problem and conducts an intelligent preliminary review based on the preset rectification and acceptance rules; S7.4: Automatically issue warnings for problems that are not rectified on time or are not rectified to a satisfactory standard, and implement escalation procedures based on the risk level of the problem; S7.5: For the same type of problem that recurs in different inspection cycles, the system automatically marks it as a systemic defect and increases the weight coefficient of its corresponding risk dimension.

10. A risk-assessment-based intelligent monitoring system for clinical trial institutions, characterized in that, include: The project management module is used to connect to the national drug or medical device filing platform, automatically synchronize clinical trial project filing information, and establish the relationship between the project and the clinical trial institution. The institution management module is used to establish a database of clinical trial institutions based on data from the filing platform, and to link all clinical trial projects undertaken by each institution, forming a three-level association between institution, project, and issue. The risk assessment engine module has a built-in problem classification model based on the key points of the verification, which is used to automatically calculate the five-dimensional quality risk index and generate a risk radar chart. The intelligent regulatory plan module is used to automatically generate differentiated regulatory plans based on the comprehensive risk score; The intelligent supervision solution and task allocation module is used to classify the inspection content and automatically allocate inspection tasks based on expert professional tags. The issue tracking and closed-loop management module is used to establish issue lifecycle tracking records; The regulatory work summary and data visualization module is used to generate multi-dimensional regulatory data statistical reports; The data layer module includes an institutional database, a project database, an expert database, a problem knowledge base, and a legal and standard database.

Images