HIV-1 virus load detection performance evaluation method and system

Abstract

The invention discloses an HIV-1 virus load detection performance evaluation method and system. For each pair of data points (xi, yi) in the data set D, the slope a and the intercept b of a straight line between all possible two points are calculated; the estimated value of the slope a is the median of all the slopes. The estimation of the intercept b is obtained by calculating the residual error of each data point and finding the median of the residual errors; the invention relates to the technical field of virus load detection. According to the invention, through multi-dimensional analysis including correlation, consistency, classification accuracy, chi-square test, Bland-Altman consistency analysis, Kappa coefficient and the like, a comprehensive performance evaluation framework is provided for the virus load detection kit. The comprehensive evaluation is helpful for ensuring that the kit can provide reliable results in various application scenarios. Through strict experimental design and data analysis, possible problems of the kit can be found and corrected in time, so that the detection accuracy and reliability of the kit are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of viral load detection, and in particular to a method and system for evaluating HIV-1 viral load detection performance. Background Art

[0002] In the management of human immunodeficiency virus (HIV) infection, the concept of "treatment as prevention" has become a core strategy. Practice has shown that highly effective antiretroviral therapy is the most effective means of preventing and treating HIV infection. This treatment can not only inhibit the replication of the virus in the infected person, but also significantly reduce the risk of virus transmission.

[0003] HIV viral load monitoring is a key indicator for evaluating the activity and dynamic changes of viral replication. It not only plays a vital role in guiding clinicians to use drugs rationally, but is also an indispensable tool for evaluating treatment effects. In particular, HIV-1 viral load testing plays a pivotal role in the discovery, diagnosis and treatment of AIDS.

[0004] In recent years, with the continuous growth of the demand for HIV viral load testing, especially in the field of rapid AIDS diagnosis, HIV-1 nucleic acid point-of-care testing (POCT) technology has emerged. However, the advent of new technologies is inevitably accompanied by the need for strict verification of their performance to ensure their accuracy and reliability, so as to evaluate their application value in clinical practice.

[0005] In order to comprehensively evaluate the performance of viral load test kits, it is necessary to use fixed-value quality control products and clinical samples to make a comprehensive judgment by comparing the test results of different test reagents. This method can not only help us understand the detection limit, accuracy and repeatability of the reagents, but also provide valuable data support and reference for clinical laboratories in actual testing work. Through such an evaluation, we can ensure that the selected test reagents can truly reflect the patient's viral load, thereby guiding clinical decision-making and optimizing treatment effects.

[0006] To this end, the present invention proposes a method and system for evaluating HIV-1 viral load detection performance. Summary of the invention

[0007] In view of this, the embodiments of the present invention hope to provide a method and system for evaluating the performance of HIV-1 viral load detection, so as to solve or alleviate the technical problems existing in the prior art, namely, how to comprehensively evaluate the performance of the viral load detection kit, and at least provide a beneficial option for this; the technical solution of the present invention is implemented as follows:

[0008] First, HIV-1 viral load test performance evaluation method:

[0009] 1. Overview:

[0010] The present invention aims to solve the above technical problems by using Passing-Bablok regression to evaluate the linear relationship between two methods, which does not require the data to satisfy the normal distribution and is particularly suitable for method comparison and consistency analysis. Passing-Bablok regression can give a regression line, a confidence interval, and a prediction interval, thereby helping researchers to more fully understand the relationship between the test results of the two reagents.

[0011] (II) Technical solution:

[0012] In order to achieve the above technical objectives, the following steps S1 to S6 need to be performed.

[0013] 2.1 Step S1, data preparation and organization:

[0014] Ensure that there are two sets of corresponding viral load test results, each from two different reagents. And remove or process outliers and missing values ​​to form data set D.

[0015] 2.1.1 Step S100, data correspondence:

[0016] Make sure that every pair of data points (x i ,y i ), x i and i They represent the viral load test results of the same sample using reagent A and reagent B respectively.

[0017] 2.1.2 Step S101, outlier processing:

[0018] When using the IQR rule, values ​​lower than (Q1-1.5*IQR) or higher than (Q3+1.5*IQR) are defined as outliers, where Q1 is the first quartile, Q3 is the third quartile, and (IQR=Q3-Q1) is the interquartile range.

[0019] 2.1.3 Missing value processing:

[0020] For missing values, methods such as deleting observations with missing values, filling with mean / median / mode, or estimating using interpolation / prediction models are adopted.

[0021] 2.1.4 Dataset Construction:

[0022] After the above processing, we get a clean dataset D:

[0023] D=[(x1,y1),(x2,y2),…,(x n ,y n )];

[0024] Where n is the number of samples, x i and i are the test results of reagent A and reagent B for the i-th sample.

[0025] 2.2 Step S2, Passing-Bablok regression analysis:

[0026] For each pair of data points (x i ,y i ), calculate the slope a and intercept b of all possible lines between two points. The estimate of the slope a is the median of all these slopes. The estimate of the intercept b is obtained by calculating the residual for each data point and finding the median of these residuals.

[0027] 2.2.1 Step S200, slope calculation:

[0028] For each pair of different data points (x i ,y i ) and (x j ,y j )(where i≠j), calculate the slope a of the line between two points ij :

[0029]

[0030] After calculating the slopes of all possible two-point combinations, we sort these slope values ​​and take the median as the slope estimate a for Passing-Bablok regression.

[0031] 2.2.2 Step S201, intercept calculation:

[0032] After determining the slope a, calculate the residual for each data point i (Also called vertical offset):

[0033] residual i =y i -(a*x i );

[0034] Again, we take the median of these residuals as our estimate of the intercept, b.

[0035] 2.2.3 Step S202, output result:

[0036] The output of Passing-Bablok regression includes the regression line equation (y=ax+b), slope a, intercept b, and their confidence intervals. The confidence intervals can be calculated using the bootstrap statistical method to estimate the variability of the parameters.

[0037] 2.3 Step S3, correlation analysis:

[0038] Each pair of data points (x i ,y i ) and its intercept b, calculated the Pearson correlation coefficient, and analyzed the regression results to evaluate the degree of linear correlation between the test results of the two reagents.

[0039] The Pearson correlation coefficient r is an indicator that measures the strength and direction of the linear relationship between two variables. Its value range is between -1 and 1, where 1 indicates a perfect positive correlation, -1 indicates a perfect negative correlation, and 0 indicates no correlation:

[0040]

[0041] Where: n is the number of data points. i and i are the x and y values ​​of the ith data point (x and y represent the viral load test results of two different reagents, respectively). and are the means of x and y respectively.

[0042] This formula calculates the covariance between two sets of data and standardizes it to get a value between -1 and 1 that represents the degree of linear association between the two sets of data.

[0043] This formula will be used in step S3 to calculate the Pearson correlation coefficient, and the value of the coefficient will be used to evaluate the degree of linear correlation between the test results of the two reagents. If r is close to 1, it means that there is a strong positive correlation between the test results of the two reagents; if r is close to -1, it means that there is a strong negative correlation; if r is close to 0, it means that there is almost no linear relationship between the test results of the two reagents.

[0044] 2.4 Step S4, Bland-Altman consistency analysis:

[0045] The consistency of the Pearson correlation coefficient r of the two reagent methods is displayed graphically; the difference between the measurement results of the two methods is calculated (diff = method 1 - method 2). The mean of the difference is calculated. diff and standard deviation sd diff Then a Bland-Altman plot was drawn, with the difference as the ordinate and the mean of the two methods as the abscissa. The consistency limit (mean difference ± 1.96*standard deviation) was added during the process.

[0046] 2.4.1 Step S400, calculate the difference between the measurement results of the two methods:

[0047] For each pair of data points (x i ,y i ), calculate their difference diff i :

[0048] diff_i=x i -y i ;

[0049] 2.4.2 Step S401, calculate the mean and standard deviation of the difference:

[0050] The mean of the difference diff and standard deviation sd diff It can be calculated by the following formula:

[0051]

[0052] Here, n is the number of data points.

[0053] 2.4.3 Step S402, draw a Bland-Altman plot:

[0054] The vertical axis is the difference between the two methods. i The horizontal axis is the mean of the measurement results of the two methods ((x i +y i ) / 2). For each data point in the dataset, we plot a point on the graph.

[0055] 2.4.4 Step S403, adding consistency limits:

[0056] The limits of agreement are calculated by calculating the mean of the differences plus or minus 1.96 times the standard deviation. On the Bland-Altman plot, we draw two horizontal lines to represent these limits:

[0057] [Upper bound = mean diff +1.96×sd diff ];

[0058] [lower bound = mean diff -1.96×sd diff ];

[0059] If most of the points fall between these two lines, then we can assume that the two measurements are in good agreement.

[0060] 2.5 Step S5, Chi-square test:

[0061] The continuous data obtained from the Bland-Altman consistency analysis were converted into categorical data of high / low viral load, and a contingency table was constructed for the chi-square test. The P values ​​of the differences in the classification results of the two reagents were compared.

[0062] 2.5.1 Step S500, establish a contingency table:

[0063] a represents the number of times both reagents were judged as high loading, b represents the number of times reagent 1 was judged as low loading and reagent 2 was judged as high loading, c represents the number of times reagent 1 was judged as high loading and reagent 2 was judged as low loading, and d represents the number of times both reagents were judged as low loading.

[0064] 2.5.1 Step S501, establish Chi-Square Statistic:

[0065] Used to measure the difference between the observed frequency and the expected frequency x 2 :

[0066]

[0067] Among them, O i is the observed frequency (i.e. a, b, c, d in the contingency table), E i is the expected frequency, which is calculated based on the independence assumption (i.e., the classification results of the two reagents are independent of each other). In the 2*2 contingency table, the expected frequency E ij It can be calculated by the following formula:

[0068]

[0069] Among them, R i is the row sum, C j is the column sum and N is the total sample size.

[0070] 2.5.2 Step S502, calculate the degrees of freedom df (Degrees of Freedom):

[0071] In the chi-square test of a 2*2 contingency table, the degree of freedom is 1 and the calculation formula is:

[0072] df = (number of rows - 1) × (number of columns - 1);

[0073] 2.5.3 Step S503, P value calculation:

[0074] According to the expected frequency E ijand degrees of freedom df, you can consult the chi-square distribution table or use statistical software to calculate the P value. The P value is used to determine whether there is a significant difference in the classification results of the two reagents. If the P value is less than the significance level (such as 0.05), the original hypothesis (that the classification results of the two reagents are independent of each other) is rejected, and it is considered that there is a significant difference in the classification results of the two reagents.

[0075] 2.6 Step S6, Kappa consistency test:

[0076] The Kappa coefficient was calculated based on the P value classification results, which is the consistency k of the classification results of the two reagents.

[0077]

[0078] in:

[0079] p o is the observed concordance rate, that is, the proportion of classification results of the two reagents that are the same.

[0080] p e is the expected agreement rate, that is, the proportion of agreements that are expected under random circumstances.

[0081] Among them, the observed consistency rate p o : is the ratio of the number of samples with the same classification results of the two reagents to the total number of samples. If the total number of samples is N, and the number of samples with the same classification results of the two reagents is a, then:

[0082]

[0083] Among them, the expected consistency rate p e : It is the expected random consistency rate based on the frequency of occurrence of each classification in two reagents without actual classification information. For binary classification problems (such as high / low viral load), if the proportion of reagent 1 classified as high is p1, and the proportion classified as low is 1-p1; the proportion of reagent 2 classified as high is p2, and the proportion classified as low is 1-p2, then:

[0084] p e =p1×p2+(1-p1)×(1-p2);

[0085] The p o and p e Substituting into the formula of Kappa coefficient, we can get the consistency of the classification results of the two reagents. The value range of Kappa coefficient is usually between -1 and 1, but in practical applications it is usually between 0 and 1. Generally speaking, the closer the Kappa coefficient is to 1, the higher the consistency of the two reagents; the closer it is to 0, the lower the consistency.

[0086] (III) Mechanism for solving technical problems:

[0087] Bland-Altman consistency analysis graphically displays the difference between the two methods and calculates the mean and standard deviation of the difference. By drawing a Bland-Altman plot and adding consistency limits, the degree of consistency between the two methods can be intuitively seen. This method is particularly suitable for the consistency assessment of continuous variables and can reveal the systematic deviations and random errors between the test results of the kit and the standard method.

[0088] The chi-square test converts continuous data into categorical data (such as high / low viral load) and constructs a contingency table for the chi-square test. This helps evaluate the performance of the test kit in classification tasks, especially when it is necessary to quickly determine whether the viral load exceeds a certain threshold. The chi-square test can reveal whether there is a significant difference in the classification results between the test kit and the standard method.

[0089] Finally, the Kappa coefficient was calculated to evaluate the consistency of the classification results of the two reagents. This factor takes into account the possibility of accidental consistency and therefore reflects the consistency between the two methods more accurately than a simple consistency rate. This step helps to further verify the reliability of the kit in the classification task.

[0090] Second, HIV-1 viral load detection performance evaluation system:

[0091] like Figure 2 As shown, the system includes a processor and a memory connected to the processor, wherein program instructions are stored in the memory. When the program instructions are executed by the processor, the processor executes the performance evaluation method as described above and feeds back the consistency result to a display screen.

[0092] Compared with the prior art, the present invention has the following beneficial effects:

[0093] 1. Comprehensive performance evaluation: The present invention provides a comprehensive performance evaluation framework for the viral load detection kit through multi-dimensional analysis, including correlation, consistency, classification accuracy, chi-square test, Bland-Altman consistency analysis and Kappa coefficient. This comprehensive evaluation helps to ensure that the kit can provide reliable results in various application scenarios. Through rigorous experimental design and data analysis, possible problems with the kit can be discovered and corrected in a timely manner, thereby improving the accuracy and reliability of its detection.

[0094] Second, standardization and enhanced comparability: The present invention uses unified evaluation standards and processes to make the performance of different test kits more comparable. This helps medical institutions and researchers choose the test kit that best suits their needs, and also promotes fair competition and technological progress in the test kit market.

[0095] 3. Risk reduction: The comprehensive performance evaluation of the present invention can reduce the risk of misdiagnosis caused by poor performance of the test kit, and protect the health and safety of patients. At the same time, for test kit manufacturers, it can also reduce the legal and economic risks caused by product quality issues.

[0096] 4. Bridge between scientific research and clinical application: The technical solution of the present invention is not only applicable to the development of test kits in the scientific research stage, but can also be used as a standard process for performance verification in clinical trials and practical applications. This helps to transform scientific research results into clinical applications more quickly and serve a large number of patients. BRIEF DESCRIPTION OF THE DRAWINGS

[0097] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or technical descriptions will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0098] Figure 1 It is a schematic diagram of the method flow of the present invention;

[0099] Figure 2 It is a schematic diagram of the system composition of the present invention;

[0100] Figure 3 Schematic diagram of linearity analysis of Xpert reagent detection;

[0101] Figure 4 This is a schematic diagram of the correlation analysis of two viral load detection reagents;

[0102] Figure 5 Schematic diagram of Bland-Altman analysis of two viral load detection reagents;

[0103] Figure 6 This is a schematic diagram of the correlation analysis of two viral load detection reagents when the viral load is less than 5000 copies mL;

[0104] Figure 7 This is a schematic diagram of the correlation analysis of two viral load detection reagents when the viral load is less than 1000 copies mL;

[0105] Figure 8 Schematic diagram of Bland-Altman analysis of two viral load detection reagents when the viral load is less than 1000 copies mL. DETAILED DESCRIPTION

[0106] In order to make the above-mentioned purposes, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are described in detail below in conjunction with the accompanying drawings. In the following description, many specific details are set forth to facilitate a full understanding of the present invention. However, the present invention can be implemented in many other ways different from those described herein, and those skilled in the art can make similar improvements without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below;

[0107] It should be noted that the various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments, and the same or similar parts between the various embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part description.

[0108] Embodiment: This embodiment discloses a method for evaluating HIV-1 viral load detection performance; it includes the following steps S1 to S6.

[0109] In this embodiment, regarding step S1, data preparation and arrangement: ensure that there are two sets of corresponding viral load test results, which come from two different reagents, and remove or process outliers and missing values ​​to form a data set D.

[0110] Specifically, step S100, data correspondence: ensure that each pair of data points (x i ,y i ), x i and i They represent the viral load test results of the same sample using reagent A and reagent B respectively.

[0111] It is understandable that in this step, it is necessary to ensure that each pair of data points (x i ,y i ) are paired, that is, x i , and y i They represent the viral load test results of the same sample using reagent A and reagent B. This is the basis for the subsequent performance comparison of the two reagents.

[0112] Data correspondence ensures that the test results of the two reagents are carried out under the same conditions, making the comparison more fair and accurate. i ,y i ) represent two different test results for the same sample, which helps us analyze the consistency and difference between the two reagents.

[0113] Specifically, step S101, outlier processing: when using the IQR rule, values ​​lower than (Q1-1.5*IQR) or higher than (Q3+1.5*IQR) are defined as outliers, where Q1 is the first quartile, Q3 is the third quartile, and (IQR=Q3-Q1) is the interquartile range. In this step, the IQR rule is used to identify and process outliers. Outliers may affect the accuracy and reliability of data analysis, so they need to be identified and processed. By removing or replacing outliers, the stability and reliability of the data set can be improved, thereby more accurately evaluating the performance of the two reagents.

[0114] Specifically, missing value processing: In a data set, missing values ​​are sometimes encountered, that is, some data points are missing or not recorded. These missing values ​​will affect the integrity of data analysis, so they need to be properly handled. There are many ways to deal with missing values, such as deleting observations with missing values, filling with mean / median / mode, or using interpolation / prediction models for estimation. Which method to choose depends on the characteristics of the data and the purpose of analysis.

[0115] Specifically, data set construction: After the above processing, we get a clean data set D, which contains n sample points, each of which has test results from two reagents:

[0116] D=[(x1,y1),(x2,y2),…,(x n ,y n )];

[0117] Where n is the number of samples, x i and i are the test results of reagent A and reagent B for the i-th sample. The construction of data set D is the basis of the entire evaluation process, which ensures the accuracy and reliability of subsequent analysis. i ,y i ) represent the test results of the two reagents on the same sample, which provides direct data support for subsequent performance comparison.

[0118] In this embodiment, regarding step S2, Passing-Bablok regression analysis: for each pair of data points (x i ,y i ), calculate the slope a and intercept b of all possible lines between two points. The estimate of the slope a is the median of all these slopes. The estimate of the intercept b is obtained by calculating the residual for each data point and finding the median of these residuals.

[0119] Specifically, step S200, slope calculation: for each pair of different data points (x i ,yi ) and (x j ,y j ), we calculate the slope a of the line between the two points ij After calculating the slopes of all possible two-point combinations, we sort these slope values ​​from small to large and then select the middle value, the median, as the slope estimate a for Passing-Bablok regression:

[0120]

[0121] After calculating the slopes of all possible two-point combinations, we sort these slope values ​​and take the median as the slope estimate a for Passing-Bablok regression.

[0122] Using the slope of the straight line between two points to represent the relationship between data points can reflect the degree of linear correlation between the test results of the two reagents. The median is chosen as the slope estimate because it is insensitive to outliers and can provide a robust estimate. Calculating the slope in this way can effectively resist the interference of outliers in the data and obtain a more accurate description of the linear relationship.

[0123] Specifically, step S201, intercept calculation: After determining the slope a, we then calculate the residual of each data point i , which is the vertical offset of each data point to the regression line. Similarly, we choose the median of these residuals as the estimate of the intercept b:

[0124] residual i =y i -(a*x i );

[0125] Again, we take the median of these residuals as our estimate of the intercept, b.

[0126] The residual represents the difference between the actual observed value and the value predicted by the regression line. By taking the median of the residual to estimate the intercept, the impact of outliers on the regression model can be further reduced. Using the median to estimate the intercept ensures the robustness of the regression model and enables the model to better adapt to the actual situation of the data.

[0127] Specifically, step S202, output results: The output results of Passing-Bablok regression include the regression line equation (y=ax+b), slope a, intercept b, and their confidence intervals. The calculation of the confidence interval can use the bootstrap statistical method to estimate the variability of the parameters. The regression line equation describes the linear relationship between the test results of the two reagents. The confidence interval provides the uncertainty range of the parameter estimate, which helps to evaluate the reliability of the results.

[0128] In this embodiment, regarding step S3, correlation analysis: for each pair of data points (x i ,y i ) and its intercept b, calculated the Pearson correlation coefficient, and analyzed the regression results to evaluate the degree of linear correlation between the test results of the two reagents.

[0129] The Pearson correlation coefficient r is an indicator that measures the strength and direction of the linear relationship between two variables. Its value range is between -1 and 1, where 1 indicates a perfect positive correlation, -1 indicates a perfect negative correlation, and 0 indicates no correlation:

[0130]

[0131] Where: n is the number of data points. i and i are the x and y values ​​of the ith data point (x and y represent the viral load test results of two different reagents, respectively). and are the means of x and y respectively.

[0132] As you can understand, this formula takes into account the difference between each data point and the mean and quantifies the linear relationship between the two sets of data by calculating the covariance. Specifically, the numerator in the formula measures the i and i The denominator normalizes the deviation from their respective means.

[0133] This formula calculates the covariance between two sets of data and standardizes it to get a value between -1 and 1 that represents the degree of linear association between the two sets of data.

[0134] This formula will be used in step S3 to calculate the Pearson correlation coefficient, and the value of the coefficient will be used to evaluate the degree of linear correlation between the test results of the two reagents. If r is close to 1, it means that there is a strong positive correlation between the test results of the two reagents; if r is close to -1, it means that there is a strong negative correlation; if r is close to 0, it means that there is almost no linear relationship between the test results of the two reagents.

[0135] It can be understood that the Pearson correlation coefficient provides a quantitative indicator that can accurately reflect the degree of linear correlation between the test results of two reagents. This is of great significance for comparing the performance of different reagents, selecting the best reagent combination, or identifying potential problems. The value of the Pearson correlation coefficient is between -1 and 1, and this range makes the interpretation of the coefficient intuitive and easy to understand. In addition, this coefficient is widely used in various statistical analyses and scientific research, so its results are highly versatile and comparable. By analyzing the Pearson correlation coefficient, researchers can understand the strength of the correlation between the two reagents, thereby providing a strong basis for subsequent experimental design, data analysis, and reagent optimization.

[0136] In this embodiment, regarding step S4, Bland-Altman consistency analysis: the consistency of the Pearson correlation coefficient r of the two reagent methods is displayed graphically; the difference between the measurement results of the two methods is calculated (diff = method 1 - method 2). The mean of the difference is calculated. diff and standard deviation sd diff Then a Bland-Altman plot was drawn, with the difference as the ordinate and the mean of the two methods as the abscissa. The consistency limit (mean difference ± 1.96*standard deviation) was added during the process.

[0137] Specifically, step S400 calculates the difference between the measurement results of the two methods: for each pair of data points (x i ,y i ), calculate their difference. For each pair of data points (x i ,y i ), calculate their difference diff i :diff_i=x i -y i ;

[0138] Specifically, step S401 calculates the mean and standard deviation of the differences: Next, we calculate the mean of all differences. diff and standard deviation sd diff The mean reflects the average level of differences between the two methods’ measurements, while the standard deviation reflects the dispersion of these differences:

[0139]

[0140] Here, n is the number of data points.

[0141] Specifically, step S402, draw a Bland-Altman graph: In the Bland-Altman graph, we use the difference between the measurement results of the two methods as the vertical coordinate, and the mean of the measurement results of the two methods as the horizontal coordinate. For each data point in the data set, we draw a point on the graph. This graphical display method can intuitively reflect the consistency between the two methods.

[0142] Specifically, step S403, adding consistency limits: The consistency limits are obtained by calculating the difference mean plus or minus 1.96 times the standard deviation. On the Bland-Altman plot, we draw two horizontal lines to represent these limits:

[0143] [Upper bound = mean diff +1.96×sd diff ];

[0144] [lower bound = mean diff -1.96×sd diff ];

[0145] If most of the points fall between these two lines, then we can assume that the two measurements are in good agreement.

[0146] Understandably, the beneficial effect of Bland-Altman consistency analysis is that it provides an intuitive and quantitative way to evaluate the consistency between two measurement methods. Through graphical display and the setting of consistency limits, we can clearly see on which samples the two methods perform consistently and on which samples there are differences. This analysis method helps us discover and understand the systematic biases and random errors between the two methods, thereby providing strong support for the improvement and optimization of the methods. At the same time, this analysis method is also widely used in clinical trials, biomarker measurements, medical imaging and other fields to ensure the consistency and reliability between different measurement methods or tools.

[0147] In this embodiment, regarding step S5, chi-square test: the continuous data obtained by Bland-Altman consistency analysis is converted into categorical data of high / low viral load, a contingency table is constructed, and a chi-square test is performed. The difference P values ​​of the classification results of the two reagents are compared.

[0148] Specifically, in step S500, a contingency table is established:

[0149]

[0150]

[0151] The continuous data obtained by Bland-Altman consistency analysis were converted to categorical data, that is, the viral load was classified as high or low according to a certain cutoff value. Subsequently, a 2x2 contingency table was constructed to record the classification results of the two reagents. In this contingency table:

[0152] a represents the number of times both reagents were judged as high loading;

[0153] b indicates the number of times that reagent 1 was judged as low loading and reagent 2 was judged as high loading;

[0154] c represents the number of times that reagent 1 judged it as high loading and reagent 2 judged it as low loading;

[0155] d indicates the number of times both reagents were judged as low loading.

[0156] Specifically, step S501, establish the Chi-Square Statistic: In order to measure the difference between the observed frequency (i.e., a, b, c, d in the contingency table) and the expected frequency under the independence hypothesis, it is necessary to calculate the Chi-Square statistic. 2 :

[0157]

[0158] Among them, O i is the observed frequency (i.e. a, b, c, d in the contingency table), E i is the expected frequency, which is calculated based on the independence assumption (i.e., the classification results of the two reagents are independent of each other). In the 2*2 contingency table, the expected frequency E ij It can be calculated by the following formula:

[0159]

[0160] Among them, R i is the row sum, C j is the column sum and N is the total sample size.

[0161] Specifically, step S502 calculates the degrees of freedom df (Degrees of Freedom). In the chi-square test, the degrees of freedom is an important parameter. For a 2*2 contingency table, the calculation formula for the degrees of freedom is: In the chi-square test of a 2*2 contingency table, the degrees of freedom is 1, and the calculation formula is:

[0162] df = (number of rows - 1) × (number of columns - 1);

[0163] Specifically, step S503, P value calculation: according to the expected frequency E ijand degrees of freedom df, you can consult the chi-square distribution table or use statistical software to calculate the P value. The P value is used to determine whether there is a significant difference in the classification results of the two reagents. If the P value is less than the significance level (such as 0.05), the original hypothesis (that the classification results of the two reagents are independent of each other) is rejected, and it is considered that there is a significant difference in the classification results of the two reagents.

[0164] It can be understood that the chi-square test step provides a statistical method to objectively evaluate whether the results of the two reagents are consistent when classifying high / low viral loads. By constructing a contingency table and calculating the chi-square statistic, the difference between the observed data and the expected data under the independence hypothesis can be quantified to determine whether there is a significant difference in the classification results of the two reagents. This analysis method helps researchers understand the similarities and differences in the classification performance of different reagents and provides a reference for selecting appropriate reagents or improving reagents.

[0165] In this embodiment, regarding step S6, Kappa consistency test: the Kappa coefficient is calculated based on the P value classification result, and the consistency degree k of the classification results of the two reagents. That is, the Kappa coefficient is an indicator to measure the consistency of the results of the two classification methods, which takes into account the possibility of random consistency and corrects it:

[0166]

[0167] in:

[0168] p o is the observed concordance rate, that is, the proportion of classification results of the two reagents that are the same.

[0169] p e is the expected agreement rate, that is, the proportion of agreements that are expected under random circumstances.

[0170] Among them, the observed consistency rate p o : is the ratio of the number of samples with the same classification results of the two reagents to the total number of samples. If the total number of samples is N, and the number of samples with the same classification results of the two reagents is a, then:

[0171]

[0172] Among them, the expected consistency rate p e : It is the expected random consistency rate based on the frequency of occurrence of each classification in two reagents without actual classification information. For binary classification problems (such as high / low viral load), if the proportion of reagent 1 classified as high is p1, and the proportion classified as low is 1-p1; the proportion of reagent 2 classified as high is p2, and the proportion classified as low is 1-p2, then:

[0173] p e=p1×p2+(1-p1)×(1-p2);

[0174] The p o and p e Substituting into the formula of the Kappa coefficient, the consistency of the classification results of the two reagents can be obtained. The value range of the Kappa coefficient is usually between -1 and 1, but in practical applications it is usually between 0 and 1. Generally speaking, the closer the Kappa coefficient is to 1, the higher the consistency of the two reagents; the closer it is to 0, the lower the consistency. The Kappa consistency test can accurately evaluate the consistency of the two reagents in classifying HIV-1 viral load, thereby helping researchers understand the performance differences between different reagents. Compared with the simple consistency rate, the Kappa coefficient takes into account the possibility of random consistency and can therefore more accurately reflect the consistency between the two reagents. This correction mechanism makes the Kappa coefficient a powerful tool for evaluating the consistency of classification methods.

[0175] Test example: This test example was made based on the above embodiment.

[0176] 1. Research background and purpose:

[0177] This study aims to evaluate the correlation and consistency between Xpert-HIV-1 viral load test reagent and TaqMan-HIV-1 viral load test reagent in detecting HIV-1 viral load. By comparing and analyzing the test results of the two reagents on 383 clinical samples, a reference basis for auxiliary diagnosis of HIV infection and monitoring of antiviral treatment effects is provided.

[0178] 2. Materials and Methods

[0179] 2.1 Sample source:

[0180] This study used 383 EDTAK2 / 3 anticoagulated plasma samples collected by the laboratory of Guizhou AIDS Confirmation Center from December 2021 to March 2022 by Western blot (WB).

[0181] 2.2 Reagents and instruments:

[0182] (1) TaqMan viral load detection reagent: The human immunodeficiency virus (type I) nucleic acid detection kit produced by Roche, USA, was used and operated on the Roche fully automatic nucleic acid detection system.

[0183] (2) Xpert viral load test reagent: includes reagents for detecting HIV-1 RNA in samples and two internal quality control products. All detection steps are performed on the GeneXpert fully automatic medical PCR analysis system.

[0184] 2.3 Experimental methods:

[0185] (1) A total of 383 clinical samples were tested using two reagents.

[0186] (2) Using 1000 CPs / mL and 5000 CPs / mL as the detection lines, the consistency of the results of the two detection reagents was analyzed.

[0187] (3) Statistical analysis methods (S1-S6), including chi-square test, Kappa consistency test, correlation and Bland-Altman consistency analysis, were used to analyze and process the data.

[0188] 3. Sample composition:

[0189] In this study, males accounted for 62.10% and females accounted for 37.90%. The age range of the subjects was 18 to 91 years old, with an average age of 50.14±15.09 years old. Among them, 62.74% were married and 60.63% had primary school to junior high school education.

[0190] 4. Reagent performance analysis:

[0191] 4.1 Detection performance analysis:

[0192] The Xpert viral load detection reagent was analyzed for detection performance using a fixed-value quality control product. The results showed that the linear slope of the gradient sample detection result was -1.073, and the correlation coefficient R2 = 0.9922, indicating that the reagent and the quality control product showed a good linear relationship. At the same time, the precision analysis results showed that the intra-batch coefficient of variation (CV) and inter-batch CV values ​​of the Xpert reagent in each dilution gradient were less than 10%, showing a high detection precision. As shown in the following table:

[0193]

[0194] 4.2 Comparison of quantitative results of clinical testing:

[0195] When comparing the viral load test results of clinical samples from the two reagents (Xpert and TaqMan), we found that they showed a good correlation. By calculation, the correlation coefficient of the test results of the two reagents was R2 = 0.946, which shows that their test results on most samples are relatively consistent. In order to more intuitively show this relationship, we used the logarithm of the Xpert viral load test results as the x-axis and the logarithm of the TaqMan reagent viral load test results as the y-axis to draw a fitting graph ( Figure 3 ). As can be seen from the figure, the data points are basically distributed around a straight line, further confirming the good correlation between the two reagents.

[0196] The regression analysis yielded a correlation coefficient of R2 = 0.896, and a significance test of F = 2827.604, P < 0.001, indicating that the regression equation was statistically significant. The statistical test of the partial regression coefficient of the independent variable (x) was t = 53.175, P < 0.001, indicating that the viral load test results of the Xpert reagent had a significant effect on the test results of the TaqMan reagent.

[0197] To evaluate the consistency between the two reagents, we performed a Bland-Altman analysis. The results showed that the mean detection bias of the two reagents was 0.18Log (copy / mL), and 98.79% (327 / 331) of the samples fell between the 95% upper and lower limits of the detection consistency ( Figure 4 ). This shows that in most cases, the test results of the two reagents are not much different and have good detection consistency.

[0198] 4.3 Comparison of clinical application results of samples with viral load <5000 copies / mL:

[0199] For samples with viral loads below 5000 copies / mL, we again compared the test results of the two reagents. The results showed that the correlation coefficient of the test results of the two reagents in these samples was R2=0.894, which was slightly lower than the correlation of the overall samples, but still showed a good correlation. By drawing a fitting graph ( Figure 5 ), we can intuitively see the distribution of data points, which further confirms this conclusion.

[0200] Regression analysis showed that the correlation coefficient of the regression equation was R2 = 0.800, and the significance test F = 135.586, P < 0.001, indicating that the regression equation was still statistically significant in samples with a viral load below 5000 copies / mL. The statistical test of the partial regression coefficient of the independent variable x was t = 11.644, P < 0.001, indicating that within this sample range, the detection results of the Xpert reagent still had a significant impact on the detection results of the TaqMan reagent.

[0201] 4.4 Comparison of clinical application results of low-load (<1000cps / mL) samples:

[0202] In samples with low viral load (<1000cps / mL), we compared the test results of the two reagents again. The results showed that the correlation coefficient of the test results of the two reagents in these samples was R2=0.814, which was slightly lower than the correlation of the overall and viral load <5000cps / mL samples, but still within an acceptable range. By drawing a fitting graph ( Figure 6 ), we can understand the distribution of data points more intuitively.

[0203] Regression analysis showed that the correlation coefficient of the regression equation was R2 = 0.662, and the significance test F = 17.666, P = 0.002, indicating that the regression equation was still statistically significant in low viral load samples. The statistical test of the partial regression coefficient of the independent variable x was t = 4.203, P = 0.002, indicating that within this specific sample range, the detection results of the Xpert reagent still had a significant impact on the detection results of the TaqMan reagent.

[0204] Finally, we performed a Bland-Altman analysis on the test results of the two reagents in low viral load samples. The results showed that the test results of some samples exceeded the upper and lower limits of 95% consistency. Specifically, the test results of the Xpert reagent and TaqMan reagent on some samples were 68cps / mL and 264cps / mL, and 275cps / mL and 134cps / mL ( Figure 7 ). These results suggest that there may be some differences in the test results of the two reagents within the low viral load range. Therefore, these results need to be interpreted with caution in clinical applications and a comprehensive judgment should be made based on the specific conditions of the patient.

[0205] All the above embodiments only express the implementation methods of the relevant practical applications of the present invention, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for those of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present invention, and these all belong to the protection scope of the present invention. Therefore, the protection scope of the patent of the present invention shall be subject to the attached claims.

[0206] For those skilled in the art, it can be further appreciated that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in the above description according to function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0207] At the same time, those skilled in the art can understand that all or part of the processes in all the above-mentioned embodiments can be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media provided in this application and used in the embodiments can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double-speed data rate SDRAM (SSRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

Claims

1. A method for evaluating HIV-1 viral load detection performance, characterized in that: The steps include: S1, data preparation and organization: for at least two groups of viral load test results from two different reagents and their corresponding results, outliers and missing values ​​are removed or processed to form a data set D; S2, Passing-Bablok regression analysis: For each pair of data points (x i ,y i ), calculate the slope a and intercept b of all straight lines between two points; S3, each pair of data points (x i ,y i ) and its intercept b, calculate the Pearson correlation coefficient r, and analyze the regression results to evaluate the degree of linear correlation between the test results of the two reagents; S4, Bland-Altman consistency analysis: graphically display the consistency of the Pearson correlation coefficient r of each two reagent methods; S5, Chi-square test: The continuous data obtained from the Bland-Altman consistency analysis were converted into categorical data of high / low viral load, and a contingency table was constructed to perform a chi-square test; the P values ​​of the differences in the classification results of the two reagents were compared; S6, Kappa consistency test: Calculate the Kappa coefficient based on the P value classification results, and the consistency k of the classification results of the two reagents.

2. The performance evaluation method according to claim 1, characterized in that: In S1, using the IQR rule, values ​​lower than (Q1-1.5*IQR) or higher than (Q3+1.5*IQR) are defined as outliers, where Q1 is the first quartile, Q3 is the third quartile, and (IQR=Q3-Q1) is the interquartile range; a data set D is obtained: D=[(x1,y1),(x2,y2),…,(x n ,y n )]; Where n is the number of samples, x i and i are the test results of reagent A and reagent B for the i-th sample.

3. The performance evaluation method according to claim 1, characterized in that: In said S2, the following steps are included: S200, slope a ij calculate: S201, after determining the slope a, calculate the residual of each data point i : residual i =y i -(a*x i ); Take these residuals i The median of is taken as the intercept b.

4. The performance evaluation method according to claim 1, characterized in that: In S3, the calculation method of Pearson correlation coefficient r is: Where: n is the number of data points; x i and i are the x and y values ​​of the ith data point respectively; and are the means of x and y respectively.

5. The performance evaluation method according to claim 1, characterized in that: In S4, the difference between the measurement results of the two methods is calculated to calculate the mean of the difference. diff and standard deviation sd diff ; Then draw a Bland-Altman plot, with the difference as the vertical axis and the mean of the measurement results of the two reagent methods as the horizontal axis, and add consistency limits during the process.

6. The performance evaluation method according to claim 5, characterized in that: In said S4, it includes: S400, for each pair of data points (x i ,y i ), calculate their difference diff i : diff_i=x i -y i ; S401, calculate the mean of the difference diff and standard deviation sd diff : Where n is the number of data points; S402, draw the Bland-Altman plot: The vertical axis is the difference between the two methods. i The horizontal axis is the mean of the measurement results of the two methods ((x i +y i ) / 2); For each data point in the data set, draw a point on the graph; S403, adding consistency limits: calculated by adding or subtracting 1.96 times the standard deviation of the difference mean.

7. The performance evaluation method according to claim 1, characterized in that: In said S5, it includes: S500, establishing a contingency table: a represents the number of times both reagents are judged as high loading, b represents the number of times reagent 1 is judged as low loading and reagent 2 is judged as high loading, c represents the number of times reagent 1 is judged as high loading and reagent 2 is judged as low loading, d represents the number of times both reagents are judged as low loading; S501, establish the chi-square statistic to measure the difference between the observed frequency and the expected frequency x 2 : Among them, O i is the observed frequency, E i is the expected frequency; S502, calculate the degrees of freedom df: In the chi-square test of a 2*2 contingency table, the degree of freedom is 1: df = (number of rows - 1) × (number of columns - 1); S503, P value calculation: According to the expected frequency E ij and degrees of freedom df, and consult the chi-square distribution table to obtain the P value.

8. The performance evaluation method according to claim 7, characterized in that: In said S6: Where: p o is the observed agreement rate, that is, the proportion of classification results of the two reagents that are the same; p e is the expected agreement rate, that is, the proportion of agreements that are expected under random circumstances.

9. The performance evaluation method according to claim 8, characterized in that: In said S6: The observed agreement rate p o : is the ratio of the number of samples with the same classification results of the two reagents to the total number of samples; the total number of samples is N, and the number of samples with the same classification results of the two reagents is a, then: Among them, the expected consistency rate p e : In the absence of actual classification information, the expected random agreement rate based only on the frequency of occurrence of each classification in the two reagents; if the proportion of reagent 1 classified as high is p1, and the proportion classified as low is 1-p1; the proportion of reagent 2 classified as high is p2, and the proportion classified as low is 1-p2, then: p e =p1×p2+(1-p1)×(1-p2); The p o and p e Substituting into the Kappa coefficient, the consistency of the classification results of the two reagents can be obtained.

10. HIV-1 viral load detection performance evaluation system, characterized by: The system includes a processor and a memory connected to the processor, wherein program instructions are stored in the memory, and when the program instructions are executed by the processor, the processor executes the performance evaluation method according to any one of claims 1 to 9.

Images