Evaluation method for evaluating detection quality of multiple sample analyzers

Abstract

The invention provides an evaluation method for evaluating the detection quality of a plurality of sample analyzers. The evaluation method comprises the following steps: acquiring reference assignments of a series of references for specifying a detection item; obtaining a detection value of each sample analyzer on the reference product; determining a standard point for each sample analyzer by means of a reference assignment and a detection value of the sample analyzer in order to establish a transfer function for the sample analyzer; converting the detection value of each sample analyzer by using a conversion function corresponding to the sample analyzer to obtain a conversion value of each sample analyzer; and performing detection result consistency comparison among the sample analyzers by using the conversion values so as to evaluate the detection quality of the sample analyzers. According to the method, a brand new algorithm is adopted, direct conversion of product reading values of different sample analyzers is achieved only according to measured values of the sample analyzers, and daily inspection quality control for a test room is achieved.

Description

Technical Field

[0001] The present invention belongs to the field of medical tests, and particularly relates to an evaluation method and device for evaluating the detection quality of multiple sample analyzers. Background Art

[0002] In the daily work of a laboratory, there often occurs a problem that the detection results of multiple different sample analyzers for the same detection item cannot be compared with each other. In a hospital laboratory, there are usually multiple test instruments of different sample analyzers to meet the requirements of a large number of tests. However, due to different test methodologies and calibration standards of different sample analyzers, there are significant differences in the detection results of different sample analyzer products for the same test sample and the same detection item. Therefore, it is impossible to judge the test accuracy or test level of each sample analyzer. There is a great challenge for the quality control of tests in the laboratory.

[0003] Therefore, it is necessary to establish an evaluation method that can better meet the actual needs for evaluating the detection quality of multiple sample analyzers for in vitro tests. Summary of the Invention

[0004] In order to overcome the deficiencies in the prior art, the present invention provides an evaluation method for evaluating the detection quality of multiple sample analyzers and a device for performing the method.

[0005] In a first aspect, the present invention provides an evaluation method for evaluating the detection quality of multiple sample analyzers, characterized in that the evaluation method includes:

[0006] Obtaining the reference value assignment of a series of reference materials for a specified detection item;

[0007] Obtaining the detection values of each of the sample analyzers for the reference materials;

[0008] For each sample analyzer, determining a standard point by means of the reference value assignment and the detection value of the sample analyzer to establish a conversion function for the sample analyzer;

[0009] Converting the detection values of each of the sample analyzers using the conversion function corresponding to the sample analyzer to obtain the conversion values of each of the sample analyzers;

[0010] Performing a consistency comparison of detection results among the sample analyzers using the conversion values to evaluate the detection quality of the sample analyzers.

[0011] In some embodiments, the reference assignment values are arranged in ascending order of rank and divided into multiple concentration gradient groups, and the standard points in each group are determined by means of the detection values corresponding to the reference assignment values in each group, and each standard point corresponds to a reference assignment value and a detection value.

[0012] In some embodiments, an interpolation method, especially a cubic spline interpolation method, is used to establish a conversion function between the respective standard points of each of the sample analyzers.

[0013] In some embodiments, the total mean value or the total robust mean value of the repeated detection values of each reference product by each of the sample analyzers is used as the reference assignment value.

[0014] In some embodiments, the reference assignment values are grouped according to the formula N = ceiling(n / 10), where n is the total number of reference products.

[0015] In some embodiments, in the grouping, the multiples of the upper and lower limits of the concentration range of each group are calculated according to the following formula: where X max is the maximum value of all reference assignment values, X min is the minimum value of the reference assignment values, and the concentration range of the first group is The concentration range of the second group is The concentration range of the third group is The concentration range of the Nth group is

[0016] In some embodiments, N ≥ 6.

[0017] In some embodiments, the reference assignment value in each group is used as the independent variable X, and the detection value of the reference product by each sample analyzer is used as the dependent variable Y, and the mean value of the percentage deviation [(Y / X - 1)×100%] of Y from X in each group is calculated; the group of values (X, Y) in which the percentage deviation of Y from X in each group is closest to the mean value of the percentage deviation of this group is selected as the standard point, and a total of N standard points are determined.

[0018] In some embodiments, the absolute value of the percentage deviation of X between two adjacent standard points among the N standard points ≥ 10%.

[0019] In some embodiments, if the percentage deviation of X between two adjacent standard points < 10%, the standard points within the concentration interval segment where these two standard points are located are adjusted.

[0020] In some embodiments, one of the two standard points is replaced with the point in this group where the percentage deviation of Y from X is the second closest to the mean value of the percentage deviation of this group as the adjusted standard point, and so on.

[0021] In some embodiments, the closest one is the one with the smallest absolute value of the difference between the percentage deviation of Y from X in a certain group and the mean of the percentage deviations of this group.

[0022] In some embodiments, the second closest one is the one with the second smallest absolute value of the difference between the percentage deviation of Y from X in a certain group and the mean of the percentage deviations of this group, and so on.

[0023] In some embodiments, the conversion value and the corresponding reference assignment are used to draw a Bland-Altman plot, so as to perform a comparison of the consistency of the test results among the sample analyzers.

[0024] In some embodiments, the method further includes using the Bland-Altman plot to perform an equivalence verification on the repeated test values of the adjusted sample analyzer.

[0025] In some embodiments, the conditions for compliance with equivalence include that the Bland-Altman plot meets one or more of the following (1)-(4):

[0026] (1) All the scatter points in the Bland-Altman plot are evenly distributed above and below the equal line corresponding to the mean, and the mean is the mean of all relative biases Bias% in the Bland-Altman plot;

[0027] (2) The relative biases Bias% of the scatter points in the Bland-Altman plot are evenly distributed within the limits of agreement LoA;

[0028] (3) The limits of agreement LoA in the Bland-Altman plot are less than the maximum allowable difference;

[0029] (4) There is no group of adjacent-ranked reference products distributed on one side of the equal line in the Bland-Altman plot,

[0030] Preferably, if the Bland-Altman plot does not meet any of the above (1)-(4), it is not equivalent;

[0031] Preferably, the mean of the Bland-Altman plot falls within the threshold, and preferably, the threshold is ±2%;

[0032] Preferably, in the Bland-Altman plot, the scatter points within the LoA range account for more than 90%, preferably more than 95% of all the scatter points;

[0033] Preferably, in the Passing-Bablok regression analysis, the proportional bias is less than or equal to 1 / 2 - 3 / 4 of the allowable bias.

[0034] In some embodiments, the proportional bias in the Passing-Bablok regression analysis = (slope - 1) × 100%.

[0035] In some embodiments, the equivalence further includes that the relative bias Bias% of the Bland-Altman plot shows a constant CV change.

[0036] In some embodiments, determining that the relative bias Bias% of the Bland-Altman plot shows a constant CV change includes the following steps:

[0037] (1) Using the sorting rank of the assigned values of the reference product from low to high as the independent variable X, and the relative bias Bias% as the dependent variable Y. The relative bias Bias% = (Y - X) / X * 100%, where X is the assigned value of the reference product and Y is the measured mean value obtained by each sample analyzer for measuring the reference product.

[0038] (2) Conduct a general linear regression analysis on X and Y, and calculate the 95% confidence intervals of the slope and intercept to determine whether one or more of the following parameters are satisfied:

[0039] The p-value of the t-test for the slope ≥ 0.1; the p-value of the t-test for the intercept ≥ 0.1; the 95% confidence intervals of both the slope and the intercept contain 0; the intercept of the general linear regression analysis is less than or equal to the mean value of the adjusted Bland-Altman plot of the working calibrator setting value, and the mean value of the Bland-Altman plot is the mean of all relative biases Bias% of the Bland-Altman plot; and perform curve estimation on X and Y using a first-order linear model, a second-order linear model, and / or a third-order linear model, and require that the p-values of the t-tests for all coefficients except the constant ≥ 0.1.

[0040] Preferably, if any of the parameters is not satisfied, readjust the setting value of the working calibrator until the relative bias Bias% of the Bland-Altman plot shows a constant CV change.

[0041] In some embodiments, curve estimation is performed on X and Y using a first-order linear model, a second-order linear model, and / or a third-order linear model according to the CLSI EP6-A protocol.

[0042] In some embodiments, the first-order linear model for curve estimation is:

[0043] Y = b0 + (b1 × X);

[0044] The second-order linear model for curve estimation is:

[0045] Y = b0 + (b1 × X) + (b2 × X 2 )

[0046] The third-order linear model for curve estimation is as follows:

[0047] Y = b0 + (b1 × X) + (b2 × X 2 ) + (b3 × X 3 ).

[0048] In a second aspect, the present invention provides an evaluation device for evaluating the detection quality of multiple sample analyzers, the device comprising:

[0049] A processor;

[0050] A memory having stored thereon a computer program that runs on the processor;

[0051] Wherein, when the computer program is executed by the processor, the steps of the method described in the first aspect are implemented.

[0052] The method of the present invention can be used for the daily inspection quality control of a laboratory, does not depend on the theoretical values of the original product calibrator / working calibrator of the sample analyzer and the algorithm of the calibration curve of the sample analyzer, and only realizes the direct conversion of the product readings of different sample analyzers based on the measured values of the sample analyzer, so as to realize the daily comparison of the same detection item for the same sample. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] Figure 1 Shows a schematic diagram of an exemplary cubic spline interpolation function in the method of the present invention.

[0054] Figure 2 Shows a Bland-Altman plot drawn after adjustment of the data group corresponding to the exemplary Bland-Altman plot in the method of the present invention.

[0055] Figure 3 Shows a scatter plot of curve estimation of an exemplary data group B in the method of the present invention.

[0056] Figure 4 Shows a scatter plot of curve estimation of the exemplary data group B after adjustment by a cubic spline function relationship in the method of the present invention.

[0057] Figure 5 Shows the Passing-Bablok regression analysis result of the exemplary data group B after conversion by a cubic spline function relationship in the method of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0058] The following are the preferred embodiments of the present invention, and the present invention is not limited to the following preferred embodiments. It should be noted that for those skilled in the art, based on the inventive concept of this invention, several variations and improvements made all fall within the protection scope of the present invention.

[0059] In the present invention, the "detection program" can be understood as a series of prescribed steps for performing the detection process.

[0060] The Bland - Altman plot is a method for evaluating the consistency between two measurements in a visual way.

[0061] The equality line in the Bland - Altman plot is the equality line corresponding to the mean value, and the mean value is the mean of all relative biases Bias% in the Bland - Altman plot.

[0062] The 95% limits of agreement are calculated as Bias ± 1.96 (SD of Bias). For example, if the calculated Bias and SD of Bias are 0.2381 and 6.964 respectively, then the 95% limits of agreement are 0.2381 ± 1.96×6.964 = - 13.41 to 13.89.

[0063] The uniform distribution refers to the "uniform distribution" in probability theory and statistics, also called rectangular distribution. It is a symmetric probability distribution, and the distribution probabilities in the same length intervals are equally likely.

[0064] When measuring the same set of data by two methods, generally, exactly the same results will not be obtained, and there is always a certain trend of difference. For example, the measurement results of one method are often greater than (less than) those of the other method. This difference is called bias.

[0065] Accuracy refers to the degree of agreement between the average value obtained from a large number of test results and the accepted reference value. The measurement of accuracy is usually expressed by bias, and bias is the difference between the test result and the true value (accepted reference value). According to the biological variation (intra - individual biological variation CV I and inter - individual biological variation CV G ), the allowable imprecision, allowable bias and allowable total error are derived, and are divided into three levels: optimal, appropriate and minimum as the performance specifications that can be relied on.

[0066] Precision can be further divided into repeatability, intermediate precision, and reproducibility according to different conditions. Repeatability refers to the precision obtained under the same conditions (time, calibration, operator, instrument, etc.), that is, the so-called within-batch precision; intermediate precision refers to the precision obtained when one or several condition factors change, but within the same laboratory; reproducibility is the precision obtained under different laboratories and different conditions, so it is also called inter-laboratory precision.

[0067] In the present invention, the robust mean, t-test, Grubbs method test, Passing-Bablok regression, etc. can all be carried out according to the conventional methods of mathematical statistics.

[0068] The "maximum allowable difference" refers to the maximum error allowed between comparable measured values.

[0069] In some embodiments, the conversion function of each sample analyzer is established by the following method:

[0070] ① Arrange the reference sample assignments in ascending order of rank as the independent variable X, and the percentage deviation of the mean of the multiple test results of each sample analyzer of the corresponding reference sample from the reference sample assignment as the dependent variable Y. Group the independent variable X, and the number of groups is determined by referring to the formula N = n / 10 (n is the total number of reference samples, rounded up). The number of groups can be adjusted up and down according to the data characteristics of the reference sample assignments. Generally, N ≥ 6. The multiples of the upper and lower limits of each concentration interval are calculated according to the following formula: where X max is the maximum value of all reference sample assignments, X min is the minimum value of the reference sample assignment. Therefore, the first concentration range is The second concentration range is And so on;

[0071] ② Calculate the mean of the percentage deviation [(Y / X - 1) × 100%] of Y from X in each group (the percentage deviations exceeding the consistency limits in the Bland-Altman plot need to be excluded);

[0072] ③ Find the set of values in each group where the percentage deviation of Y from X is closest to (the absolute value of the difference is the smallest) the mean percentage deviation of the group as a point (X, Y) in the functional relationship. A total of N points can be found;

[0073] ④ Calculate the percentage deviation of X between adjacent two points among the N points found in ③, ensuring that the absolute value of the percentage deviation ≥ 10%. If the percentage deviation of X between two certain points < 10%, then adjust the standard points within the concentration interval where these two points are located. For example, replace a certain point with a set of values in the group where the percentage deviation of Y from X is the second closest (the absolute value of the difference is the second smallest) to the average percentage deviation of the group.

[0074] ⑤ Use the above N points as standard points to establish a cubic spline function relationship, and each sample analyzer converts the detection results according to the established cubic spline function relationship.

[0075] To make the present invention easier to understand, the following further details the present invention through specific embodiments. These embodiments are only illustrative and not limited to the application scope of the present invention.

[0076] Embodiment 1: An evaluation method for evaluating the detection quality of multiple sample analyzers

[0077] (1) Obtain the reference assignment of a series of reference materials for a specified detection item

[0078] ① Calculate the mean value of the repeated detection results of each reference material measured by each included sample analyzer;

[0079] ② Calculate the total mean value (MEAN total- i) or the total robust mean value (e.g., R-MEAN total -i) of the repeated detection results of each reference material, which is the assignment of the reference material. Among them, compared with MEAN total -i, R-MEAN total -i is preferably used because considering the problem that although some sample analyzers have good correlation, the overall measured values are too high or too low, resulting in the assignment of the reference material being greatly affected by individual sample analyzers.

[0080] (2) Obtain the detection values of each sample analyzer for the reference materials

[0081] Arrange the reference assignments obtained in step (1) in ascending order of rank and divide them into multiple concentration gradient groups. With the detection values corresponding to the reference assignments in each group, determine the standard points in each group. Each standard point corresponds to a reference assignment and a detection value.

[0082] Specifically, the reference assignments are grouped according to the formula N = n / 10, where n is the total number of reference materials, and round up. In the grouping, the multiples of the upper and lower limits of each concentration interval are calculated according to the following formula: Where X max is the maximum value of all reference assignments, and X min is the minimum value of the reference assignments. Among them, the concentration range of the first group is The second group of concentration ranges is and so on.

[0083] (3) By means of the cubic spline interpolation method, for each sample analyzer, standard points are determined by using the reference assignment and the measured values of the sample analyzer, so as to establish a conversion function for the sample analyzer.

[0084] Specifically, taking the reference assignment in each group of step (2) as the independent variable X and the measured value of the reference by each sample analyzer as the dependent variable Y, calculate the mean value of the percentage deviation [(Y / X - 1)×100%] of Y from X in each group; select the group of values (X, Y) in each group where the percentage deviation of Y from X is closest to the mean value of the percentage deviation of this group as the standard points, and a total of N standard points are determined. Among them, the absolute value of the percentage deviation of X between two adjacent standard points among the N standard points ≥ 10%; if the percentage deviation of X between two adjacent standard points < 10%, replace one of the two standard points with the one in this group where the percentage deviation of Y from X is the second closest to the mean value of the percentage deviation of this group as the adjusted standard point, and so on.

[0085] (4) Use the conversion function obtained in step (3) corresponding to the sample analyzer to convert the measured values of each sample analyzer to obtain the conversion values of each sample analyzer;

[0086] (5) Use the conversion values to perform a comparison of the consistency of the test results among the sample analyzers to evaluate the test quality of the sample analyzers.

[0087] The following further illustrates with sample analyzer B as an example. Select standard points according to the above method, as shown in Table 1. Establish a cubic spline function relationship for each point in Table 1 according to the above method, as Figure 1 shown. Then, substitute the measured values of the sample analyzer as the independent variable into the established cubic spline function relationship to calculate the conversion values of the non-standard points respectively. Finally, plot the conversion values and the corresponding reference assignments in a Bland-Altman plot, as shown in Figure 2 . It can be obtained that the mean value of the Bland-Altman plot is 1.1%, the upper and lower limits of LoA are -21.3% to 23.5% respectively, and each point is evenly distributed within LoA. Then it can be determined that the two test procedures corresponding to the two groups of data are in line with consistency and meet the requirements related to Passing-Bablok regression and medical decision levels.

[0088] Table 1

[0089]

[0090] Among them, to determine whether the data group meets "consistency (constant CV)", the algorithm judgment is as follows:

[0091] ①The reference standards are ranked from low to high in terms of assigned values. Using the ranked order as the independent variable X1 and the relative deviation Bias% mentioned above as the dependent variable Y1

[0092] ②Perform a simple linear regression analysis on X1 and Y1, calculate the 95% confidence intervals of the slope and intercept. It is required that the p-value of the t-test for the slope ≥ 0.1 and the p-value of the t-test for the intercept ≥ 0.1, and the 95% confidence intervals of both the slope and intercept contain 0;

[0093] ③According to the CLSI EP6-A protocol, perform curve estimation on X1 and Y1, calculate the first-order, second-order, and third-order linearity. It is required that for all coefficients except the constant, the p-value of the t-test ≥ 0.1, and the statistical analysis results of the constant are not included in the evaluation;

[0094] ④The absolute value of the intercept in the simple linear regression analysis is less than or equal to 2.

[0095] ⑤If any of the above requirements is not met, it means that the characteristic of "constant CV" is not presented.

[0096] The following uses data set B as an example to illustrate the above judgment method.

[0097] The results of the simple linear regression for data set B are shown in Table 2, and the results of the curve estimation are shown in Table 3. The scatter plot of the curve estimation is as Figure 3 shown. It can be obtained that the intercept of the simple linear regression is greater than 2, and all non-constant coefficients of the third-order linearity in the curve estimation are less than 0.1. In summary, the data does not show the characteristic of "constant CV".

[0098] Table 2 Results of the simple linear regression analysis for data set B

[0099]

[0100] Table 3 Summary of the curve estimation results for data set B

[0101]

[0102] Using the above cubic spline interpolation function to adjust the test results of non-standard points in data set B, refitting the test results, the results of the simple linear regression are shown in Table 4, and the results of the curve estimation are shown in Table 5. The scatter plot of the curve estimation is as Figure 4 shown, and all meet the requirements. In summary, the data shows the characteristic of "constant CV". The results of the Passing-Bablok regression analysis for data set B are shown in Figure 5 , and the results meet the requirements.

[0103] Table 4 Results of the simple linear regression analysis after adjusting the assigned values of the calibrators in data set B

[0104]

[0105] Summary Table of Curve Estimation Results after Assigning Values to the Calibrator of Adjusted Data Group B

[0106]

[0107] The technical solution of the present invention is not limited to the limitations of the above specific embodiments. Any technical deformation made according to the technical solution of the present invention falls within the protection scope of the present invention.

Claims

1. An evaluation method for evaluating the detection quality of multiple sample analyzers, characterized in that The evaluation method includes: Obtaining the reference assignment of a series of reference products for a specified test item; Obtaining the test values of each of the sample analyzers for the reference products; For each sample analyzer, determining a standard point by means of the reference assignment and the test value of the sample analyzer to establish a conversion function for the sample analyzer; Converting the test values of each of the sample analyzers using the conversion function corresponding to the sample analyzer to obtain the conversion values of each of the sample analyzers; Performing a consistency comparison of test results among the sample analyzers using the conversion values to evaluate the test quality of the sample analyzers.

2. The method according to claim 1, wherein, Arranging the reference assignments in ascending order of rank and dividing them into multiple concentration gradient groups, and determining the standard points in each group by means of the test values corresponding to the reference assignments in each group, each standard point corresponding to a reference assignment and a test value.

3. The method according to claim 1, wherein Establishing a conversion function between the standard points of each of the sample analyzers by means of an interpolation method, the interpolation method being in particular a cubic spline interpolation method.

4. The method according to claim 1, wherein Taking the total mean or total robust mean of the repeated test values of each sample analyzer for each reference product as the reference assignment.

5. The method according to claim 2, wherein, The reference assignments are grouped according to the formula N = n / 10, where n is the total number of reference products, rounded up.

6. The method according to claim 5, wherein In the said grouping, the multiple of the upper and lower limits of each concentration range is calculated according to the following formula: where X max is the maximum value assigned to all reference samples, and X min is the minimum value assigned to the reference samples; Preferably, N≥6.

7. The method according to claim 2, wherein Taking the reference assignment in each group as the independent variable X and the test value of each sample analyzer for the reference product as the dependent variable Y, and calculating the mean of the percentage deviation [(Y / X - 1)×100%] of Y from X in each group; Selecting a set of values (X, Y) in each group where the percentage deviation of Y from X is closest to the mean percentage deviation of the group as the standard point, and a total of N standard points are determined.

8. The method according to claim 7, wherein The absolute value of the percentage deviation of X between two adjacent standard points among the N standard points is ≥10%.

9. The method according to claim 1, wherein Drawing a Bland - Altman plot using the conversion values and the corresponding reference assignments, so as to perform a consistency comparison of test results among the sample analyzers.

10. An evaluation device for evaluating the test quality of multiple sample analyzers, the device includes: A processor; A memory, on which a computer program running on the processor is stored; Wherein, when the computer program is executed by the processor, the steps of the method according to any one of claims 1 to 9 are implemented.

Images