Clinical test data analysis method and system based on AI big language model

Abstract

The invention discloses a clinical test data analysis method and system based on an AI large language model, and relates to the technical field of clinical data processing, and the method comprises the steps: obtaining support data of an AI large language model analysis result, and collecting data source data of the support data, analyzing according to the data source data of the support data to obtain a support data accuracy evaluation value; time characteristic data of the support data are collected, and a support data reliability evaluation value is obtained through comprehensive analysis according to the time characteristic data of the support data and the support data accuracy evaluation value; and according to the support data accuracy evaluation value and the support data reliability evaluation value, carrying out comprehensive analysis to obtain an AI large language model analysis result reliability index, and according to the AI large language model analysis result reliability index, evaluating and feeding back the AI large language model analysis result. According to the method, the AI analysis result is comprehensively evaluated, and the accuracy and credibility of the AI in clinical test application are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of clinical data processing, and specifically to a clinical trial data analysis method and system based on an AI large language model. Background Art

[0002] Clinical trial data comes from a wide range of sources and in huge amounts, especially in large-scale, multi-center clinical trials, where the management and analysis of clinical trial data have high requirements. On the other hand, AI big language models have been pre-trained with a large amount of text data and have strong natural language processing capabilities. They can understand the semantic and grammatical relationships of texts and perform operations such as text classification, summarization, and translation.

[0003] For example, the invention patent with announcement number CN112786126B is a method, device, electronic device and medium for time series analysis of clinical trial data, which involves the field of data analysis. Among them, the time series analysis method of clinical trial data includes: extracting the stage information of different medical stages in the case report form, constructing a medical data dictionary based on the stage information, and the case report form is used to store the clinical trial data; determining the medical stage to be bound that matches the target case report form based on different analysis dimensions; binding the medical data dictionary with the target case report form based on the medical stage to be bound, and configuring the binding result as a multidimensional information table based on the time series.

[0004] For example, the invention patent with publication number CN118116536A is a large language model system for clinical trial data analysis and interpretation. The system includes a user interface, a large language model, a natural language processing module, a data storage and management module, a data preprocessing and feature extraction module, a data cleaning and conversion module, a result interpretation and visualization module, a data analysis and modeling module, and a reporting and inference module; the large language model is used to deeply understand and generate natural language text and process complex text data in clinical trials; the data analysis and modeling module is used to analyze and model data using machine learning or statistical methods, extract information, discover patterns, and predict results; the reporting and inference module is used to automatically generate high-quality clinical trial reports and summaries to improve the readability and comprehensibility of research results.

[0005] However, in the process of implementing the technical solution of the invention in the embodiments of the present application, the present application discovered that the above technology has at least the following technical problems: the large language model is a complex neural network model, which usually cannot provide an explanation of causal relationships, thereby affecting the clinical researchers' trust in the analysis results of the large language model, and the accuracy of the clinical trial analysis results cannot be improved. Summary of the invention

[0006] In view of the deficiencies in the prior art, the present invention provides a clinical trial data analysis method and system based on an AI large language model, which can effectively solve the problems involved in the above-mentioned background technology.

[0007] To achieve the above objectives, the present invention is implemented through the following technical solutions: The first aspect of the present invention provides a clinical trial data analysis method based on an AI large language model, comprising: obtaining supporting data for the analysis results of the AI ​​large language model, and collecting data source data for the supporting data, and obtaining an accuracy assessment value of the supporting data based on the data source data of the supporting data.

[0008] The time characteristic data of the supporting data is collected, and the reliability assessment value of the supporting data is obtained through comprehensive analysis based on the time characteristic data of the supporting data and the accuracy assessment value of the supporting data.

[0009] Based on the supporting data accuracy evaluation value and the supporting data reliability evaluation value, a comprehensive analysis is conducted to obtain the reliability index of the AI ​​large language model analysis results. Based on the AI ​​large language model analysis result reliability index, the AI ​​large language model analysis results are evaluated and feedback is provided.

[0010] As a further method, the supporting data accuracy assessment value is obtained based on the data source data analysis of the supporting data. The specific analysis process is: the data source data of the supporting data includes each working status indicator of each clinical trial data acquisition device, the number of required supporting data items and the number of actually collected supporting data items.

[0011] According to the working status indicators of each clinical trial data acquisition device, the accuracy index of the acquisition device is obtained after processing.

[0012] Based on the accuracy index of the collection equipment, the number of required support data items and the actual number of collected support data items, a comprehensive analysis is conducted to obtain the accuracy assessment value of the support data.

[0013] As a further method, the processing obtains the acquisition equipment accuracy index, and the specific process is: the working status indicators include the electromagnetic signal interference intensity of each clinical trial data acquisition device and the working voltage of each clinical trial data acquisition device; the critical electromagnetic signal interference intensity, the reference working voltage and the allowable deviation working voltage of each clinical trial data acquisition device are extracted from the clinical trial database, and the acquisition equipment accuracy index is obtained through comprehensive analysis.

[0014] As a further method, the time characteristic data of the supporting data include the data collection period of each clinical trial data, the length of the adjacent collection interval of each clinical trial data and the value of each clinical trial data collected in history.

[0015] As a further method, the comprehensive analysis obtains the supporting data reliability assessment value, and the specific analysis process is: obtaining the numerical value of each clinical trial data recorded at the current time point, and extracting the allowable deviation value of each clinical trial data from the clinical trial database; based on the time characteristic data of the supporting data and the supporting data accuracy assessment value, comprehensive analysis is performed to obtain the supporting data reliability assessment value.

[0016] As a further method, the comprehensive analysis obtains the reliability index of the AI ​​large language model analysis result. The specific analysis process is: according to the supporting data accuracy evaluation value and the supporting data reliability evaluation value, the AI ​​large language model analysis result reliability index is obtained by comprehensive analysis. The AI ​​large language model analysis result reliability index is a quantitative index obtained by analyzing the supporting data of the analysis result, and is used to quantitatively evaluate the reliability of the AI ​​large language model analysis result.

[0017] As a further method, the AI ​​large language model analysis results are evaluated and fed back according to the AI ​​large language model analysis result reliability index, specifically including: extracting the AI ​​large language model analysis result reliability index threshold from the clinical trial database, comparing the AI ​​large language model analysis result reliability index with the AI ​​large language model analysis result reliability index threshold, if the AI ​​large language model analysis result reliability index is greater than or equal to the AI ​​large language model analysis result reliability index threshold, then evaluating the AI ​​large language model analysis result as a high reliability result, providing an analysis report to researchers, and storing the AI ​​large language model analysis results in the clinical trial database.

[0018] As a further method, the evaluating and feeding back the AI ​​large language model analysis results according to the AI ​​large language model analysis result reliability index also includes: extracting the lower limit value of the AI ​​large language model analysis result reliability index from the clinical trial database, comparing the AI ​​large language model analysis result reliability index with the lower limit value of the AI ​​large language model analysis result reliability index; if the AI ​​large language model analysis result reliability index is greater than or equal to the lower limit value of the AI ​​large language model analysis result reliability index, and the AI ​​large language model analysis result reliability index is less than the AI ​​large language model analysis result reliability index threshold, then the AI ​​large language model analysis result is evaluated as a medium reliability result, and the data collection step is verified.

[0019] If the reliability index of the AI ​​large language model analysis result is less than the lower limit of the reliability index of the AI ​​large language model analysis result, the AI ​​large language model analysis result will be evaluated as a low reliability result, and a warning will be immediately issued to the researchers, temporarily suspending the clinical trial or retraining the AI ​​large language model.

[0020] The second aspect of the present invention provides a clinical trial data analysis system based on an AI large language model, including: a data acquisition module, used to obtain supporting data for the analysis results of the AI ​​large language model, and to collect data source data of the supporting data, and to obtain an accuracy assessment value of the supporting data based on the data source data of the supporting data.

[0021] The data reliability assessment module is used to collect the time characteristic data of the supporting data, and obtain the supporting data reliability assessment value through comprehensive analysis based on the time characteristic data of the supporting data and the supporting data accuracy assessment value.

[0022] The model analysis result evaluation module is used to obtain the AI ​​large language model analysis result reliability index based on the supporting data accuracy evaluation value and the supporting data reliability evaluation value through comprehensive analysis, and evaluate and provide feedback on the AI ​​large language model analysis results based on the AI ​​large language model analysis result reliability index.

[0023] Compared with the prior art, the embodiments of the present invention have at least the following beneficial effects: (1) The present invention provides a clinical trial data analysis method and system based on an AI large language model, which can accurately evaluate the accuracy of data collection in clinical trials, provide a reliable basis for AI analysis results, and further comprehensively evaluate the reliability by considering time feature data, and comprehensively evaluate AI analysis results to achieve accurate feedback. This is conducive to improving the accuracy and credibility of AI applications in clinical trials, timely discovering and improving data problems, providing strong support for decision-making, and ensuring the scientificity and reliability of the results.

[0024] (2) By monitoring the electromagnetic signal interference intensity and the working voltage of the acquisition equipment, the present invention can fully understand the impact of the equipment working environment and its own status on data acquisition, avoid the impact of electromagnetic interference and unstable working voltage on equipment performance, and reduce data deviation. Comprehensive analysis of the electromagnetic signal interference intensity and the working voltage of the equipment can provide a quantitative basis for evaluating the applicability of the equipment in clinical trials, which helps to improve the accuracy and reliability of data acquisition.

[0025] (3) By correlating the acquisition equipment accuracy index with the actual number of acquisition support data items, the present invention can accurately evaluate the quality and reliability of data acquisition, ensure the accuracy of clinical trial data, help to timely discover equipment performance problems and insufficient data acquisition problems, and provide a basis for equipment optimization and data improvement.

[0026] Of course, any product implementing the present invention does not necessarily need to achieve all of the advantages described above at the same time. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 It is a schematic diagram of the method flow of the present invention.

[0028] Figure 2 It is a schematic diagram of system module connection of the present invention.

[0029] Figure 3 It is a schematic diagram of the functional relationship between the support data reliability evaluation value and the reliability index of the AI ​​large language model analysis result involved in the embodiment of the present invention. DETAILED DESCRIPTION

[0030] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0031] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device.

[0032] Reference Figure 1 As shown, the first aspect of the present invention provides a clinical trial data analysis method based on an AI large language model, comprising: obtaining supporting data for the analysis results of the AI ​​large language model, and collecting data source data of the supporting data, and obtaining an accuracy assessment value of the supporting data based on the data source data of the supporting data.

[0033] It should be understood that the supporting data in this embodiment refers to data that can provide a basis, evidence or explanation for the analysis results obtained by the AI ​​large language model, including clinical trial data, such as blood pressure, blood sugar and medication frequency. It is understandable that the collection frequency and collection time of clinical trial data vary.

[0034] The time characteristic data of the supporting data is collected, and the reliability assessment value of the supporting data is obtained through comprehensive analysis based on the time characteristic data of the supporting data and the accuracy assessment value of the supporting data.

[0035] Based on the supporting data accuracy evaluation value and the supporting data reliability evaluation value, a comprehensive analysis is conducted to obtain the reliability index of the AI ​​large language model analysis results. Based on the AI ​​large language model analysis result reliability index, the AI ​​large language model analysis results are evaluated and feedback is provided.

[0036] Specifically, the support data accuracy assessment value is obtained based on the data source data analysis of the support data, and the specific analysis process is: the data source data of the support data includes each working status indicator of each clinical trial data acquisition device, the number of required support data items and the number of actual collected support data items.

[0037] It should be understood that the number of required support data items in this embodiment refers to the number of various data items that are theoretically required to support the analysis results in the clinical trial data analysis process based on the AI ​​large language model, according to the requirements and objectives of the analysis task. It is based on an ideal state and considers the sum of data items required to accurately obtain analysis results, conduct effective data verification and interpretation, etc. The number of data items actually collected to support the analysis results refers to the number of various data items that are actually obtained through various data collection devices, means and processes during the clinical trial process. It reflects the amount of data that can truly provide support for data analysis in actual operations.

[0038] According to the working status indicators of each clinical trial data acquisition device, the accuracy index of the acquisition device is obtained after processing.

[0039] Based on the accuracy index of the collection equipment, the number of required support data items and the actual number of collected support data items, a comprehensive analysis is conducted to obtain the accuracy assessment value of the support data.

[0040] Specifically, the method for obtaining the data accuracy evaluation value is as follows: ; In the formula, Indicates the support data accuracy assessment value, Indicates the accuracy index of the acquisition equipment. Indicates the number of data items actually collected. Indicates the number of data items required to be supported. Indicates the data accuracy assessment impact factor corresponding to the preset acquisition equipment accuracy index, Indicates the data accuracy assessment impact factor corresponding to the preset number of collection support data items.

[0041] In a specific embodiment, a supporting data accuracy assessment threshold is obtained from a clinical trial database, and the supporting data accuracy assessment value is compared with the supporting data accuracy assessment threshold. If the supporting data accuracy assessment value is greater than or equal to the supporting data accuracy assessment threshold, further analysis is performed to obtain a supporting data reliability assessment value. If the supporting data accuracy assessment value is less than the supporting data accuracy assessment threshold, maintenance personnel are notified to check the working status of the equipment and to supplement the collected required supporting data that has not been collected.

[0042] It should be understood that in this embodiment and The data accuracy assessment impact factors corresponding to the acquisition device accuracy index and the number of acquisition support data items preset in the clinical trial database respectively represent the numerical values ​​of the degree of influence of the acquisition device accuracy index and the number of acquisition support data items on the support data accuracy assessment value, and can be directly obtained from the clinical trial database when used. For example, the acquisition device accuracy index or the number of acquisition support data items and the data accuracy assessment impact factors corresponding to the acquisition device accuracy index or the number of acquisition support data items preset in the clinical trial database form a mapping set, and the acquisition device accuracy index or the number of acquisition support data items input into the mapping set is used to obtain the data accuracy assessment impact factors corresponding to the acquisition device accuracy index or the number of acquisition support data items, wherein the mapping relationship is a many-to-one or one-to-one relationship, and the value range of the weight factor in this embodiment is between 0 and 1.

[0043] In this embodiment, the support data accuracy evaluation value is used to quantitatively evaluate the accuracy of the support data collected in the clinical trial. The larger the acquisition equipment accuracy index, or the larger the number of actual collected support data items, the larger the corresponding support data accuracy evaluation value, indicating that the support data has a higher accuracy.

[0044] The algorithm of this embodiment combines the acquisition equipment precision index, the number of required support data items and the actual number of support data items collected, and comprehensively analyzes to obtain the support data accuracy evaluation value. In this formula, there is a correlation between the acquisition equipment precision index and the actual number of support data items collected. The acquisition equipment precision index reflects the accuracy of the data collected by the device under a specific working state, and the actual number of support data items collected is the amount of available data actually obtained in actual operation. The larger the acquisition equipment precision index, the more accurate and comprehensive the device can collect data. Under the same conditions, the actual support data items collected may be larger. By correlating the acquisition equipment precision index with the actual number of support data items collected, the quality and reliability of data collection can be accurately evaluated, the accuracy of clinical trial data can be guaranteed, and it is helpful to promptly discover equipment performance problems and insufficient data collection, providing a basis for equipment optimization and data improvement.

[0045] Specifically, the processing obtains the acquisition equipment accuracy index, and the specific process is: the working status indicators include the electromagnetic signal interference intensity of each clinical trial data acquisition device and the working voltage of each clinical trial data acquisition device.

[0046] It should be understood that the electromagnetic signal interference intensity in this embodiment refers to the electromagnetic signal intensity of the working environment of the clinical trial data acquisition device, which can be monitored by a spectrum analyzer. The working voltage of each clinical trial data acquisition device is collected by a voltage sensor.

[0047] The critical electromagnetic signal interference intensity, the reference operating voltage and the allowable deviation operating voltage of each clinical trial data acquisition device were extracted from the clinical trial database, and the acquisition equipment accuracy index was obtained through comprehensive analysis.

[0048] In a specific embodiment, the acquisition method of the acquisition device accuracy index is as follows: ; In the formula, represents the precision index of the acquisition equipment, e represents the natural constant, represents the electromagnetic signal interference intensity of the jth clinical trial data acquisition device, Indicates the critical electromagnetic signal interference intensity, represents the operating voltage of the jth clinical trial data acquisition device, represents the reference operating voltage of the jth clinical trial data acquisition device, Indicates the allowable deviation working voltage, Indicates the device accuracy impact factor corresponding to the preset electromagnetic signal interference intensity. represents the equipment accuracy influencing factor corresponding to the preset working voltage, j represents the serial number of each clinical trial data acquisition device, j=1,2,3,...,m, and m represents the total number of clinical trial data acquisition devices.

[0049] It should be understood that in this embodiment and They are the equipment precision influencing factors corresponding to the electromagnetic signal interference intensity and the working voltage preset in the clinical trial database, respectively, and represent the numerical values ​​of the influence degree of the electromagnetic signal interference intensity and the working voltage on the precision index of the acquisition equipment, respectively, and can be directly obtained from the clinical trial database when used. For example, a mapping set is formed by combining the electromagnetic signal interference intensity or the working voltage with the equipment precision influencing factors corresponding to the electromagnetic signal interference intensity or the working voltage preset in the clinical trial database, and the equipment precision influencing factors corresponding to the electromagnetic signal interference intensity or the working voltage are obtained according to the electromagnetic signal interference intensity or the working voltage input mapping set, wherein the mapping relationship is a many-to-one or one-to-one relationship, and the value range of the weight factor in this embodiment is between 0 and 1.

[0050] In a specific embodiment, let m=1, for a certain clinical trial data acquisition device, =-60dBm, =0.5V, = =0.5, the reference working voltage is 12V, and the data example of the acquisition equipment accuracy index is shown in Table 1. The acquisition equipment accuracy index is used to quantitatively evaluate the working performance of the clinical trial data acquisition equipment. When the value of the electromagnetic signal interference intensity is smaller, or the working voltage of the clinical trial data acquisition equipment deviates from the reference value, the corresponding acquisition equipment accuracy index is smaller, indicating that the working performance of the clinical trial data acquisition equipment is lower, and the accuracy of the collected clinical trial data is lower.

[0051] It should be understood that the unit of the critical electromagnetic signal interference intensity in this embodiment is decibel milliwatt. The smaller the value, the greater the interference intensity. Here, it means that when the interference intensity reaches -60dBm and above, it may have a significant impact on the device.

[0052] Table 1. Data example of acquisition equipment accuracy index It should be understood that the algorithm of this embodiment combines the electromagnetic signal interference intensity and the working voltage, and comprehensively analyzes to obtain the acquisition equipment accuracy index. In this formula, the electromagnetic signal interference intensity and the working voltage directly affect the performance of the clinical trial data acquisition equipment. The working voltage is the basis for the normal operation of the equipment. If the working voltage deviates from the reference working voltage, the various components of the equipment may not work in the optimal state. The electromagnetic signal interference intensity will interfere with the transmission and processing of electronic signals inside the equipment. When the electromagnetic signal interference intensity is large, additional voltage or current may be induced in the circuit of the equipment, which is similar to superimposing an interference signal on the normal working voltage. This interference signal will interact with the working voltage and affect the processing accuracy of the equipment for the data acquisition signal.

[0053] In the signal processing circuit of the device, the operating voltage determines the benchmark for operations such as signal amplification and filtering. The presence of electromagnetic signal interference intensity will change the characteristics of the input signal, such as the amplitude and frequency of the signal. When the operating voltage deviates from the reference value, the ability of the device to cope with electromagnetic signal interference will also change. For example, when the operating voltage is low, the signal amplifier inside the device may not provide sufficient amplification to overcome the signal attenuation caused by electromagnetic interference, thereby affecting the accuracy of data collection.

[0054] This embodiment can fully understand the impact of the equipment working environment and its own status on data collection by monitoring the electromagnetic signal interference intensity and the working voltage of the acquisition equipment, which helps to timely discover potential problems, avoid the impact of electromagnetic interference and unstable working voltage on equipment performance, and reduce data deviation. Comprehensive analysis can evaluate the accuracy of equipment data collection, provide a quantitative basis for evaluating the applicability of equipment in clinical trials, facilitate targeted measures to improve the working environment, improve the accuracy and reliability of data collection, and ensure the effectiveness of clinical trial results.

[0055] Specifically, the time characteristic data of the supported data include the data collection period of each clinical trial data, the length of the adjacent collection interval of each clinical trial data, and the numerical value of each clinical trial data collected in history.

[0056] It should be understood that, in this embodiment, the data collection cycle refers to the preset interval between each data collection, and the adjacent collection interval refers to the time interval between the time point of the adjacent collection of a certain data and the current time point.

[0057] Specifically, comprehensive analysis is performed to obtain the reliability assessment value of the supporting data. The specific analysis process is: obtaining the numerical value of each clinical trial data recorded at the current time point, and extracting the allowable deviation value of each clinical trial data from the clinical trial database.

[0058] Based on the time characteristic data of the supporting data and the accuracy assessment value of the supporting data, a comprehensive analysis is conducted to obtain the reliability assessment value of the supporting data.

[0059] In a specific embodiment, the method of obtaining the support data reliability evaluation value is as follows: ; In the formula, Indicates support for the data reliability assessment value, Indicates the support data accuracy assessment value, represents the length of the adjacent collection interval of the i-th type of clinical trial data, represents the data collection period of the i-th clinical trial data, Indicates the value of the i-th clinical trial data recorded at the current time point, represents the value of the i-th clinical trial data collected at the t-th time, represents the allowable deviation value of the i-th category clinical trial data, Indicates the data reliability assessment impact factor corresponding to the preset collection interval. Indicates the data reliability assessment impact factor corresponding to the preset clinical trial data value, It represents the data reliability assessment influencing factor corresponding to the preset supporting data accuracy assessment value, i represents the number of each clinical trial data type, i=1,2,3,...,n, n represents the total number of clinical trial data types, t represents the number of each historical collection, t=1,2,3,...,s, s represents the total number of historical collections.

[0060] It should be understood that in this embodiment , and The data reliability assessment influencing factors corresponding to the collection interval time, clinical trial data value and supporting data accuracy assessment value preset in the clinical trial database respectively represent the values ​​of the influence degree of the collection interval time, clinical trial data value and supporting data accuracy assessment value on the supporting data reliability assessment value respectively, and can be directly obtained from the clinical trial database when used. For example, the collection interval time, clinical trial data value or supporting data accuracy assessment value and the data reliability assessment influencing factors corresponding to the collection interval time, clinical trial data value or supporting data accuracy assessment value preset in the clinical trial database are formed into a mapping set, and the data reliability assessment influencing factors corresponding to the collection interval time, clinical trial data value or supporting data accuracy assessment value are obtained according to the collection interval time, clinical trial data value or supporting data accuracy assessment value input into the mapping set, wherein the mapping relationship is a many-to-one or one-to-one relationship, and the value range of the weight factor in this embodiment is between 0 and 1.

[0061] In this embodiment, the support data reliability evaluation value is used to quantitatively evaluate the reliability of the support data collected in the clinical trial. The longer the adjacent collection interval is, or the more the currently recorded clinical trial data value deviates from the average value of the historical collection value, or the smaller the support data accuracy evaluation value is, the smaller the corresponding support data reliability evaluation value is, indicating that the support data is more reliable.

[0062] The algorithm of this embodiment combines the length of the adjacent collection interval, the deviation value between the currently recorded clinical trial data value and the historically collected value, and the support data accuracy evaluation value, and comprehensively analyzes to obtain the support data reliability evaluation value. In this formula, there is a correlation between the length of the adjacent collection interval, the deviation value between the currently recorded clinical trial data value and the historically collected value, and the support data accuracy evaluation value. The length of the adjacent collection interval will affect the acceptability of the deviation value between the currently recorded clinical trial data value and the historically collected value. When the length of the adjacent collection interval is constant, if the deviation value is larger, it may mean that the more serious the abnormal situation that occurs during the data collection process. The deviation value directly affects the support data accuracy evaluation value. The larger the deviation value, the lower the accuracy of the support data. At the same time, the reduction of the adjacent collection interval can detect data changes and abnormal situations more promptly, which helps to improve the accuracy of the support data accuracy evaluation value.

[0063] This embodiment accurately evaluates the reliability of supporting data by comprehensively considering the data in terms of time, accuracy, and relationship with historical data. This helps to timely discover abnormal data fluctuations, judge the reliability of equipment and collection processes, provide a basis for equipment maintenance and collection method improvement, and ensure the quality of clinical trial data.

[0064] Specifically, a reliability index of the AI ​​large language model analysis result is obtained through comprehensive analysis. The specific analysis process is: according to the supporting data accuracy evaluation value and the supporting data reliability evaluation value, a reliability index of the AI ​​large language model analysis result is obtained through comprehensive analysis. The reliability index of the AI ​​large language model analysis result is a quantitative index obtained by analyzing the supporting data of the analysis result, and is used to quantitatively evaluate the reliability of the AI ​​large language model analysis result.

[0065] In a specific embodiment, the method of obtaining the reliability index of the AI ​​large language model analysis result is as follows: ; In the formula, Indicates the reliability index of the AI ​​large language model analysis results. Indicates support for the data reliability assessment value, Indicates the support data accuracy assessment value, It indicates the reliability impact factor of the analysis result corresponding to the preset support data reliability assessment value. Indicates the reliability impact factor of the analysis result corresponding to the preset support data accuracy assessment value.

[0066] It should be understood that in this embodiment and The reliability influencing factors of the analysis results corresponding to the support data reliability evaluation values ​​and the support data accuracy evaluation values ​​preset in the clinical trial database respectively represent the numerical values ​​of the degree of influence of the support data reliability evaluation values ​​and the support data accuracy evaluation values ​​on the reliability index of the analysis results of the AI ​​large language model respectively, and can be directly obtained from the clinical trial database when used. For example, a mapping set is formed by combining the support data reliability evaluation value or the support data accuracy evaluation value with the reliability influencing factors of the analysis results corresponding to the support data reliability evaluation value or the support data accuracy evaluation value preset in the clinical trial database, and the reliability influencing factors of the analysis results corresponding to the support data reliability evaluation value or the support data accuracy evaluation value are obtained by inputting the mapping set according to the support data reliability evaluation value or the support data accuracy evaluation value, wherein the mapping relationship is a many-to-one or one-to-one relationship, and the value range of the weight factor in this embodiment is between 0 and 1.

[0067] In a specific embodiment, Figure 3 As shown, = = 0.5, when =0.5, the functional relationship between the reliability evaluation value of the supported data and the reliability index of the AI ​​large language model analysis result is shown in curve a; when =0.8, the functional relationship between the reliability evaluation value of the supported data and the reliability index of the AI ​​large language model analysis result is shown in curve b; when =1.2, the functional relationship between the support data reliability evaluation value and the reliability index of the AI ​​large language model analysis result is shown in curve c. The reliability index of the AI ​​large language model analysis result is jointly determined by the support data reliability evaluation value and the support data accuracy evaluation value. The larger the support data reliability evaluation value and the support data accuracy evaluation value, the larger the corresponding AI large language model analysis result reliability index, indicating that the AI ​​large language model analysis result is more reliable.

[0068] The algorithm of this embodiment combines the support data reliability evaluation value and the support data accuracy evaluation value, and comprehensively analyzes to obtain the reliability index of the AI ​​large language model analysis result. In this formula, there is a correlation between the support data reliability evaluation value and the support data accuracy evaluation value. The larger the support data accuracy evaluation value, the larger the support data reliability evaluation value. By considering the reliability and accuracy evaluation values ​​at the same time, we can have a more comprehensive understanding of the quality of the data and provide a basis for judging the credibility of the AI ​​analysis results.

[0069] Specifically, the AI ​​large language model analysis results are evaluated and fed back according to the AI ​​large language model analysis result reliability index, including: extracting the AI ​​large language model analysis result reliability index threshold from the clinical trial database, comparing the AI ​​large language model analysis result reliability index with the AI ​​large language model analysis result reliability index threshold; if the AI ​​large language model analysis result reliability index is greater than or equal to the AI ​​large language model analysis result reliability index threshold, the AI ​​large language model analysis result is evaluated as a high reliability result, and an analysis report is provided to researchers, and the AI ​​large language model analysis results are stored in the clinical trial database.

[0070] Specifically, the AI ​​large language model analysis results are evaluated and fed back according to the AI ​​large language model analysis result reliability index, and also include: extracting the lower limit value of the AI ​​large language model analysis result reliability index from the clinical trial database, comparing the AI ​​large language model analysis result reliability index with the AI ​​large language model analysis result reliability index lower limit value; if the AI ​​large language model analysis result reliability index is greater than or equal to the AI ​​large language model analysis result reliability index lower limit value, and the AI ​​large language model analysis result reliability index is less than the AI ​​large language model analysis result reliability index threshold value, then the AI ​​large language model analysis result is evaluated as a medium reliability result, and the data collection step is verified.

[0071] If the reliability index of the AI ​​large language model analysis result is less than the lower limit of the reliability index of the AI ​​large language model analysis result, the AI ​​large language model analysis result will be evaluated as a low reliability result, and a warning will be immediately issued to the researchers, temporarily suspending the clinical trial or retraining the AI ​​large language model.

[0072] Reference Figure 2 As shown, the second aspect of the present invention provides a clinical trial data analysis system based on an AI large language model, comprising: a data acquisition module, a data reliability assessment module, a model analysis result assessment module and a clinical trial database.

[0073] The data collection module is used to obtain supporting data for the analysis results of the AI ​​large language model and to collect data source data of the supporting data, and to obtain an accuracy assessment value of the supporting data based on data source data analysis of the supporting data.

[0074] The data reliability assessment module is used to collect the time characteristic data of the supporting data, and obtain the supporting data reliability assessment value through comprehensive analysis based on the time characteristic data of the supporting data and the supporting data accuracy assessment value.

[0075] The model analysis result evaluation module is used to obtain the AI ​​large language model analysis result reliability index based on the supporting data accuracy evaluation value and the supporting data reliability evaluation value through comprehensive analysis, and evaluate and provide feedback on the AI ​​large language model analysis results based on the AI ​​large language model analysis result reliability index.

[0076] The clinical trial database is used to store data related to clinical trial data analysis, including indicators such as critical electromagnetic signal interference intensity, reference operating voltage and allowable deviation operating voltage of each clinical trial data acquisition device. The data in the clinical trial database can be obtained by collecting historical clinical trial data or from public databases related to clinical trials.

[0077] In a specific embodiment, the present invention provides a clinical trial data analysis method and system based on an AI large language model, which can accurately evaluate the accuracy of data collection in clinical trials, provide a reliable basis for AI analysis results, and further comprehensively evaluate the reliability by considering time feature data, and comprehensively evaluate AI analysis results to achieve accurate feedback. This is conducive to improving the accuracy and credibility of AI applications in clinical trials, timely discovering and improving data problems, providing strong support for decision-making, and ensuring the scientificity and reliability of the results.

[0078] The preferred embodiments of the present invention disclosed above are only used to help explain the present invention. The preferred embodiments do not describe all the details in detail, nor do they limit the invention to the specific implementation methods described. Obviously, many modifications and changes can be made according to the content of this specification. This specification selects and specifically describes these embodiments in order to better explain the principles and practical applications of the present invention, so that technicians in the relevant technical field can understand and use the present invention well. As long as they do not deviate from the structure of the present invention or exceed the scope defined by the present invention, they should all belong to the protection scope of the present invention.

Claims

1. A clinical trial data analysis method based on an AI large language model, characterized by: include: Obtain supporting data for the AI ​​large language model analysis results, collect data source data for the supporting data, and obtain an accuracy assessment value for the supporting data based on data source data analysis of the supporting data; Collect time characteristic data of the supporting data, and obtain a reliability evaluation value of the supporting data through comprehensive analysis based on the time characteristic data of the supporting data and the accuracy evaluation value of the supporting data; Based on the supporting data accuracy evaluation value and the supporting data reliability evaluation value, a comprehensive analysis is conducted to obtain the reliability index of the AI ​​large language model analysis results. Based on the AI ​​large language model analysis result reliability index, the AI ​​large language model analysis results are evaluated and feedback is provided.

2. The clinical trial data analysis method based on the AI ​​large language model according to claim 1 is characterized by: The supporting data accuracy evaluation value is obtained by analyzing the data source data of the supporting data. The specific analysis process is as follows: The data source data of the supporting data include each working status indicator of each clinical trial data collection device, the number of required supporting data items and the number of actually collected supporting data items; According to the working status indicators of each clinical trial data collection device, the accuracy index of the collection device is obtained after processing; Based on the accuracy index of the collection equipment, the number of required support data items and the actual number of collected support data items, a comprehensive analysis is conducted to obtain the accuracy assessment value of the support data.

3. The clinical trial data analysis method based on the AI ​​large language model according to claim 2 is characterized by: The method for obtaining the accuracy evaluation value of the supporting data is as follows: ； In the formula, Indicates the support data accuracy assessment value, Indicates the accuracy index of the acquisition equipment. Indicates the number of data items actually collected. Indicates the number of data items required to be supported. Indicates the data accuracy assessment impact factor corresponding to the preset acquisition equipment accuracy index, Indicates the data accuracy assessment impact factor corresponding to the preset number of collection support data items.

4. The clinical trial data analysis method based on the AI ​​large language model according to claim 2 is characterized by: The processing obtains the acquisition device accuracy index, and the specific process is: The working status indicators include the electromagnetic signal interference intensity of each clinical trial data acquisition device and the working voltage of each clinical trial data acquisition device; The critical electromagnetic signal interference intensity, the reference operating voltage and the allowable deviation operating voltage of each clinical trial data acquisition device were extracted from the clinical trial database, and the acquisition equipment accuracy index was obtained through comprehensive analysis.

5. The clinical trial data analysis method based on the AI ​​large language model according to claim 1 is characterized by: The time characteristic data of the supporting data include the data collection period of each clinical trial data, the length of the adjacent collection interval of each clinical trial data and the value of each clinical trial data collected in history.

6. The clinical trial data analysis method based on the AI ​​large language model according to claim 5 is characterized by: The comprehensive analysis obtained the support data reliability assessment value, and the specific analysis process is as follows: Obtain the values ​​of each clinical trial data recorded at the current time point, and extract the allowable deviation values ​​of each clinical trial data from the clinical trial database; Based on the time characteristic data of the supporting data and the accuracy assessment value of the supporting data, a comprehensive analysis is conducted to obtain the reliability assessment value of the supporting data.

7. The clinical trial data analysis method based on the AI ​​large language model according to claim 1 is characterized by: The comprehensive analysis obtains the reliability index of the AI ​​large language model analysis result. The specific analysis process is as follows: According to the supporting data accuracy evaluation value and the supporting data reliability evaluation value, a comprehensive analysis is performed to obtain the reliability index of the AI ​​large language model analysis result. The AI ​​large language model analysis result reliability index is a quantitative index obtained by analyzing the supporting data of the analysis result, and is used to quantitatively evaluate the reliability of the AI ​​large language model analysis result.

8. The clinical trial data analysis method based on the AI ​​large language model according to claim 1 is characterized by: The evaluation and feedback of the AI ​​large language model analysis results according to the AI ​​large language model analysis result reliability index specifically includes: Extract the reliability index threshold of the AI ​​large language model analysis result from the clinical trial database, compare the reliability index of the AI ​​large language model analysis result with the reliability index threshold of the AI ​​large language model analysis result, and if the reliability index of the AI ​​large language model analysis result is greater than or equal to the reliability index threshold of the AI ​​large language model analysis result, evaluate the AI ​​large language model analysis result as a high reliability result, provide an analysis report to the researchers, and store the AI ​​large language model analysis result in the clinical trial database.

9. The clinical trial data analysis method based on the AI ​​large language model according to claim 8 is characterized by: The evaluating and feeding back the AI ​​large language model analysis results according to the AI ​​large language model analysis result reliability index also includes: Extract the lower limit value of the reliability index of the AI ​​large language model analysis result from the clinical trial database, compare the reliability index of the AI ​​large language model analysis result with the lower limit value of the reliability index of the AI ​​large language model analysis result, if the reliability index of the AI ​​large language model analysis result is greater than or equal to the lower limit value of the reliability index of the AI ​​large language model analysis result, and the reliability index of the AI ​​large language model analysis result is less than the reliability index threshold of the AI ​​large language model analysis result, then evaluate the AI ​​large language model analysis result as a medium reliability result, and verify the data collection step; If the reliability index of the AI ​​large language model analysis result is less than the lower limit of the reliability index of the AI ​​large language model analysis result, the AI ​​large language model analysis result will be evaluated as a low reliability result, and a warning will be immediately issued to the researchers, temporarily suspending the clinical trial or retraining the AI ​​large language model.

10. A system using the clinical trial data analysis method based on the AI ​​large language model as described in any one of claims 1 to 9, characterized in that: include: The data collection module is used to obtain supporting data for the analysis results of the AI ​​large language model and to collect data source data for the supporting data, and to obtain an accuracy assessment value of the supporting data based on data source data analysis of the supporting data; A data reliability assessment module is used to collect time characteristic data of supporting data, and obtain a reliability assessment value of supporting data through comprehensive analysis based on the time characteristic data of supporting data and the accuracy assessment value of supporting data; The model analysis result evaluation module is used to obtain the AI ​​large language model analysis result reliability index based on the supporting data accuracy evaluation value and the supporting data reliability evaluation value through comprehensive analysis, and evaluate and provide feedback on the AI ​​large language model analysis results based on the AI ​​large language model analysis result reliability index.

Images