Assessment method for assessing test quality of multiple sample analyzers

Abstract

Provided in the present invention is an assessment method for assessing the test quality of multiple sample analyzers. The assessment method comprises: acquiring assigned reference sample values of a series of reference samples for a specified test item; acquiring test values of the reference samples from each sample analyzer; for each sample analyzer, determining a standard point by means of the assigned reference sample values and the test values from the sample analyzer, so as to establish a conversion function for the sample analyzer; converting the test values from each sample analyzer by using the conversion function corresponding to the sample analyzer, so as to obtain converted values from each sample analyzer; and executing test result consistency comparison between the sample analyzers by using the converted values, so as to assess the test quality of the sample analyzers. In the present invention, a new algorithm is used, the direct conversion of product read values from different sample analyzers is realized only according to measured values from the sample analyzers, and daily inspection quality control for a laboratory is realized.

Description

Evaluation method for evaluating the detection quality of multiple sample analyzers

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] The present invention claims priority to Chinese patent application No. 202311785101X filed on December 12, 2023, entitled “Evaluation Method for Evaluating the Detection Quality of Multiple Sample Analyzers”, and the entire contents of the above patent application are incorporated into the present invention as a whole. Technical Field

[0003] The present invention belongs to the field of medical testing, and in particular relates to an evaluation method and device for evaluating the detection quality of multiple sample analyzers. Background Art

[0004] In the daily work of the laboratory, the problem of being unable to compare the test results of multiple different sample analyzers for the same test item often arises. In hospital laboratories, there are usually multiple different sample analyzers for testing to meet large-scale testing requirements. However, due to the different test methodologies based on different sample analyzers and the different calibration standards of each instrument, the test results of different sample analyzer products for the same test sample and the same test item are significantly different. Therefore, it is impossible to judge the test accuracy or test level of each sample analyzer. This poses a great challenge to the test quality control of the laboratory.

[0005] Therefore, it is necessary to establish an evaluation method for evaluating the detection quality of multiple sample analyzers for in vitro testing that can better meet actual needs.

[0006] Summary of the Invention

[0007] In order to overcome the deficiencies in the prior art, the present invention provides a method for evaluating the detection quality of multiple sample analyzers and a device for performing the method.

[0008] In a first aspect, the present invention provides an evaluation method for evaluating the detection quality of multiple sample analyzers, characterized in that the evaluation method comprises:

[0009] Obtain reference material assignments for a series of reference materials used for a specified test item;

[0010] Obtaining the detection value of each sample analyzer on the reference product;

[0011] For each sample analyzer, determining a standard point by means of the reference sample assignment value and the detection value of the sample analyzer to establish a transfer function for the sample analyzer;

[0012] converting the detection value of each of the sample analyzers using the conversion function corresponding to the sample analyzer to obtain a conversion value of each of the sample analyzers;

[0013] The conversion value is used to perform consistency comparison of detection results between the sample analyzers to evaluate the detection quality of the sample analyzers.

[0014] In some embodiments, the reference values ​​are assigned in order from low to high and divided into multiple concentration gradient groups, and the standard points in each group are determined with the help of the detection values ​​corresponding to the reference values ​​in each group, and each standard point corresponds to a reference value and a detection value.

[0015] In some embodiments, a transfer function is established between the standard points of each sample analyzer using an interpolation method, and the interpolation method is particularly a cubic spline interpolation method.

[0016] In some embodiments, the total mean or total robust mean of the repeated detection values ​​of each reference product by each sample analyzer is used as the reference product value.

[0017] In some embodiments, the reference article assignments are grouped according to the formula N=n / 10, where n is the total number of reference articles, rounded up.

[0018] In some embodiments, in the grouping, the multiples of the upper and lower limits of each concentration interval are calculated according to the following formula: where X max The maximum value assigned to all references, X min The minimum value assigned to the reference product, where the first group of concentrations ranges from [X min , The second concentration range is The third concentration range is ..., the concentration range of group N is

[0019] In some embodiments, N≥6.

[0020] In some embodiments, the reference value in each group is assigned as the independent variable X, and the detection value of the reference by each sample analyzer is used as the dependent variable Y. The mean of the percentage deviation between Y and X in each group [(Y / X-1)×100%] is calculated; and the set of values ​​(X, Y) in each group whose percentage deviation between Y and X is closest to the mean of the percentage deviation of the group is selected as the standard point, and a total of N standard points are determined.

[0021] In some embodiments, the absolute value of the percentage deviation of X between two adjacent standard points among the N standard points is ≥10%.

[0022] In some embodiments, if the percentage deviation of X between two adjacent standard points is less than 10%, the standard points within the concentration interval where the two standard points are located are adjusted.

[0023] In some embodiments, one of the two standard points is replaced with a point in the group whose percentage deviation of Y from X is second closest to the mean of the percentage deviation of the group as the adjusted standard point, and so on.

[0024] In some embodiments, the closest value is the smallest absolute value of the difference between the percentage deviation of Y and X in a group and the mean of the percentage deviation of the group.

[0025] In some embodiments, the second closest is the second smallest absolute value of the difference between the percentage deviation of Y and X in a certain group and the mean of the percentage deviation of the group, and so on.

[0026] In some embodiments, a Bland-Altman plot is drawn using the converted values ​​and the corresponding reference values, thereby performing a consistency comparison of the detection results between the various sample analyzers.

[0027] In some embodiments, the method further comprises performing equivalence verification on the adjusted repeated detection values ​​of the sample analyzer using a Bland-Altman plot.

[0028] In some embodiments, meeting the equivalence condition includes a Bland-Altman plot meeting one or more of the following (1)-(4):

[0029] (1) All scattered points in the Bland-Altman plot are evenly distributed above and below the equalization line corresponding to the mean, and the mean is the mean of all relative deviations Bias% in the Bland-Altman plot;

[0030] (2) The relative deviations (Bias%) of the scattered points in the Bland-Alman plot are evenly distributed within the consistency limit (LoA);

[0031] (3) the LoA in the Bland-Alman plot is less than the maximum allowable difference;

[0032] (4) In the Bland-Altman plot, there is no set of reference items with adjacent ranks distributed on one side of the equal line.

[0033] Preferably, if the Bland-Altman plot does not satisfy any one of the above (1)-(4), then it is not equivalent;

[0034] Preferably, the mean of the Bland-Altman plot falls within a threshold, preferably, the threshold is ±2%;

[0035] Preferably, in the Bland-Altman plot, the scatter points within the LoA range account for more than 90% of all scatter points, preferably more than 95%;

[0036] Preferably, in the Passing-Bablok regression analysis, the proportional bias is less than or equal to 1 / 2-3 / 4 of the allowed bias.

[0037] In some embodiments, the proportional bias in the Passing-Bablok regression analysis = (slope - 1) x 100%.

[0038] In some embodiments, the equivalence further comprises the relative deviation Bias% of the Bland-Altman plot presenting a constant CV change.

[0039] In some embodiments, determining whether the relative deviation Bias% of the Bland-Altman plot presents a constant CV change comprises the following steps:

[0040] (1) The ranking of the reference sample values ​​from low to high is used as the independent variable X, and the relative deviation Bias% is used as the dependent variable Y. The relative deviation Bias% = (YX) / X*100%, where X is the reference sample value and Y is the mean value obtained by each sample analyzer when measuring the reference sample.

[0041] (2) Perform a general linear regression analysis on X and Y, and calculate the 95% confidence interval of the slope and intercept to determine whether one or more of the following parameters are met:

[0042] The slope t-test p-value is ≥ 0.1; the intercept t-test p-value is ≥ 0.1; the 95% confidence intervals for the slope and intercept both include 0; the intercept of the general linear regression analysis is less than or equal to the mean of the Bland-Altman plot after adjustment for the working calibrant set value, where the mean of the Bland-Altman plot is the mean of all relative biases in the Bland-Altman plot; and a curve is estimated for X and Y using a first-order linear model, a second-order linear model, and / or a third-order linear model, requiring that the t-test p-value for all coefficients except the constant is ≥ 0.1.

[0043] Preferably, if any of the parameters are not met, the set value of the working calibrant is readjusted until the relative bias Bias% in the Bland-Altman plot exhibits a constant CV variation.

[0044] In some embodiments, curve estimation is performed for X and Y using a first-order linear model, a second-order linear model, and / or a third-order linear model according to the CLSI EP6-A protocol.

[0045] In some embodiments, the first-order linear model for curve estimation is:

[0046] Y = b0 + (b1 × X);

[0047] The second-order linear model for curve estimation is:

[0048] Y=b0+(b1×X)+(b2×X 2 );

[0049] The third-order linear model for curve estimation is:

[0050] Y=b0+(b1×X)+(b2×X 2 )+(b3×X 3 ).

[0051] In a second aspect, the present invention provides an evaluation device for evaluating the detection quality of multiple sample analyzers, the device comprising:

[0052] processor;

[0053] a memory storing a computer program executed on the processor;

[0054] Wherein, when the computer program is executed by the processor, the steps of the method described in the first aspect are implemented.

[0055] The method of the present invention can be used for routine quality control of laboratory tests. It does not rely on the theoretical values ​​of the original product calibrators / working calibrators of the sample analyzers or the algorithm of the sample analyzer calibration curve. It can directly convert the product readings of different sample analyzers based solely on the measured values ​​of the sample analyzers, thereby realizing routine comparison of the same test items of the same sample. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] FIG1 shows a schematic diagram of an exemplary cubic spline interpolation function in the method of the present invention.

[0057] FIG2 shows an exemplary Bland-Altman plot in the method of the present invention, which is a Bland-Altman plot obtained after the data set is adjusted.

[0058] FIG3 shows an exemplary scatter plot of curve estimation of data set B in the method of the present invention.

[0059] FIG4 shows a scatter plot of curve estimation of an exemplary data set B after adjustment by a cubic spline function relationship in the method of the present invention.

[0060] FIG5 shows the Passing-Bablok regression analysis results of an exemplary data set B after being transformed by a cubic spline function relationship in the method of the present invention. DETAILED DESCRIPTION

[0061] The following are preferred embodiments of the present invention, and the protection of the present invention is not limited to the following preferred embodiments. It should be noted that for those skilled in the art, several variations and improvements made based on this invention concept all fall within the scope of protection of the present invention.

[0062] In the present invention, a "test procedure" may be understood as a series of prescribed steps for performing a test process.

[0063] The Bland-Altman plot is a method for visually evaluating the agreement between two measurements.

[0064] The equalization line in the Bland-Altman plot is the equalization line corresponding to the mean, and the mean is the mean of all relative deviations Bias% in the Bland-Altman plot.

[0065] The 95% limits of agreement are calculated as Bias ± 1.96 (SD of Bias). For example, if the calculated Bias and (SD of Bias) are 0.2381 and 6.964, respectively, then the 95% limits of agreement are 0.2381 ± 1.96 × 6.964 = -13.41 to 13.89.

[0066] Uniform distribution refers to the "uniform distribution" in the sense of probability theory and statistics, also called rectangular distribution. It is a symmetrical probability distribution, and the distribution probability in intervals of the same length is equally likely.

[0067] When measuring the same set of data using two methods, we generally will not obtain exactly the same results. There will always be differences with certain trends. For example, the measurement results of one method are often greater (less than) than the results of another method. This difference is called bias.

[0068] Trueness refers to the degree of agreement between the average of a large number of test results and the accepted reference value. Trueness is often measured in terms of bias, which is the difference between the test result and the true value (accepted reference value). Based on biological variation (intra-individual biological variation (CVI) and inter-individual biological variation (CVG), allowable imprecision, allowable bias, and allowable total error are derived and categorized as optimal, appropriate, and minimum performance standards.

[0069] Precision can be further divided into repeatability, intermediate precision, and reproducibility, depending on the conditions. Repeatability refers to the precision achieved under identical conditions (time, calibration, operator, instrument, etc.), also known as within-batch precision. Intermediate precision refers to the precision achieved within the same laboratory despite changes in one or more factors. Reproducibility refers to the precision achieved in different laboratories under different conditions, also known as between-laboratory precision.

[0070] In the present invention, robust mean, t-test, Grubbs test, Passing-Bablok regression and the like can all be performed according to conventional methods of mathematical statistics.

[0071] “Maximum permissible difference” refers to the maximum error allowed between comparable measurements.

[0072] In some embodiments, the transfer function for each sample analyzer is established as follows:

[0073] ① Arrange the reference sample values ​​in descending order as the independent variable X, and the percentage deviation between the mean of the multiple test results of each sample analyzer of the corresponding reference sample and the reference sample value as the dependent variable Y. Group the independent variable X, and determine the number of groups with reference to the formula N = n / 10 (n is the total number of reference samples, rounded up). The number of groups can be fine-tuned up or down according to the data characteristics of the reference sample values. Generally, N ≥ 6. The multiples of the upper and lower limits of each group of concentration ranges are calculated according to the following formula: where X max The maximum value assigned to all references, X min is the minimum value assigned to the reference product, so the concentration range of the first group is The second concentration range is And so on;

[0074] ② Calculate the mean of the percentage deviation between Y and X in each group [(Y / X-1)×100%] (excluding percentage deviations that exceed the limits of agreement in the Bland-Altman plot);

[0075] ③ Find the set of values ​​in each group whose percentage deviation between Y and X is closest to the mean of the percentage deviation of the group (the absolute value of the difference is the smallest) as a point (X, Y) in the function relationship. A total of N points can be found;

[0076] ④ Calculate the percentage deviation of X between two adjacent points of the N points found in ③, ensuring that the absolute value of the percentage deviation is ≥10%. If the percentage deviation of X between two points is less than 10%, adjust the standard point within the concentration range of the two points. For example, replace a point with a set of values ​​whose percentage deviation of Y and X is the second closest to the mean of the percentage deviation of the group (the absolute value of the difference is the second smallest);

[0077] ⑤ The above N points are used as standard points to establish a cubic spline function relationship, and each sample analyzer converts the test results according to the established cubic spline function relationship.

[0078] In order to make the present invention easier to understand, the present invention is further described in detail below through specific examples. These examples are only for illustration and do not limit the scope of application of the present invention.

[0079] Example 1: Evaluation Method for Evaluating the Detection Quality of Multiple Sample Analyzers

[0080] (1) Obtain reference material assignments for a series of reference materials used in a specified test item

[0081] ① Calculate the mean of the repeated test results of each reference product measured by each sample analyzer included;

[0082] ②Calculate the total mean (MEAN) of the repeated test results of each reference product total- i) or the overall robust mean (eg. R-MEAN total -i), which is the value assigned to the reference. total -i, prefer R-MEAN total -i. The reason is that although some sample analyzers have good correlation, the overall measured values ​​are too high or too low, which leads to the fact that the value of the reference material is greatly affected by individual sample analyzers.

[0083] (2) Obtain the detection value of each sample analyzer for the reference product

[0084] The reference sample assignments obtained in step (1) are arranged in order from low to high and divided into multiple concentration gradient groups. The standard points in each group are determined with the help of the test values ​​corresponding to the reference sample assignments in each group. Each standard point corresponds to a reference sample assignment and a test value.

[0085] Specifically, the reference samples are assigned values ​​in groups according to the formula N = n / 10, where n is the total number of reference samples, rounded up. Within the grouping, the multiples of the upper and lower limits of each concentration range are calculated according to the following formula: where X max The maximum value assigned to all references, X min The minimum value assigned to the reference product, where the first group of concentration ranges is The second concentration range is And so on.

[0086] (3) With the help of the cubic spline interpolation method, for each sample analyzer, the standard point is determined by means of the reference value and the detection value of the sample analyzer to establish a conversion function for the sample analyzer.

[0087] Specifically, the reference sample value in each group in step (2) is assigned as the independent variable X, and the detection value of the reference sample by each sample analyzer is used as the dependent variable Y. The mean of the percentage deviation between Y and X in each group [(Y / X-1)×100%] is calculated; the set of values ​​(X, Y) in each group whose percentage deviation between Y and X is closest to the mean of the percentage deviation of the group is selected as the standard point, and a total of N standard points are determined. Among the N standard points, the absolute value of the percentage deviation of X between two adjacent standard points is ≥10%; if the percentage deviation of X between two adjacent standard points is less than 10%, one of the two standard points is replaced with the point in the group whose percentage deviation between Y and X is the second closest to the mean of the percentage deviation of the group as the adjusted standard point, and so on.

[0088] (4) converting the detection value of each sample analyzer using the conversion function obtained in step (3) corresponding to the sample analyzer to obtain a conversion value of each sample analyzer;

[0089] (5) Use the conversion values ​​to perform consistency comparison of test results between various sample analyzers to evaluate the test quality of the sample analyzers.

[0090] The following uses sample analyzer B as an example for further explanation. Standard points were selected according to the above method, as shown in Table 1. A cubic spline function relationship was established for each point in Table 1 according to the above method, as shown in Figure 1. The sample analyzer's measured values ​​were then substituted into the established cubic spline function relationship as independent variables to calculate the conversion values ​​for each non-standard point. Finally, a Bland-Altman plot was plotted between the conversion values ​​and the corresponding reference values, as shown in Figure 2. The Bland-Altman plot has a mean of 1.1%, and the upper and lower limits of the LoA range from -21.3% to 23.5%, respectively. All points are evenly distributed within the LoA, indicating that the two test procedures corresponding to the two data sets are consistent and meet the requirements of Passing-Bablok regression and the medical decision level.

[0091] Table 1

[0092] Among them, to determine whether the data group meets the "consistency (constant CV)" algorithm, the following judgment is adopted:

[0093] ① Sort the reference values ​​from low to high, with the ranking as the independent variable X1, and the relative deviation Bias% mentioned above as the dependent variable Y1

[0094] ② Conduct a simple linear regression analysis on X1 and Y1 and calculate the 95% confidence interval of the slope and intercept. The t-test p-value of the slope and the intercept must be ≥ 0.1, and the 95% confidence interval of the slope and intercept must contain 0.

[0095] ③According to the CLSI EP6-A protocol, curve estimation is performed for X1 and Y1, and first-order, second-order, and third-order linearity is calculated. All coefficients except the constant are required to have a t-test p-value ≥ 0.1. The statistical analysis results of the constant are not included in the evaluation;

[0096] ④The absolute value of the intercept of simple linear regression analysis is less than or equal to 2.

[0097] ⑤ If any of the above requirements are not met, it means that the "constant CV" change characteristic is not presented.

[0098] The above judgment method is illustrated below using data group B as an example.

[0099] The results of simple linear regression for data set B are shown in Table 2, the results of curve estimation are shown in Table 3, and the curve estimation scatter plot is shown in Figure 3. It can be seen that the intercept of simple linear regression is greater than 2, and the non-constant coefficient of the third-order linear estimation of the curve is less than 0.1. In summary, the data do not show the characteristics of "constant CV" change.

[0100] Table 2 Simple linear regression analysis results of data set B

[0101] Table 3 Summary of curve estimation results for data set B

[0102] The aforementioned cubic spline interpolation function was used to adjust the test results for nonstandard points in data set B. The test results were refitted. The results of simple linear regression are shown in Table 4, and the results of curve estimation are shown in Table 5. The curve estimation scatter plot is shown in Figure 4, both of which meet the requirements. In summary, the data show a "constant CV" variation characteristic. The results of Passing-Bablok regression analysis on data set B are shown in Figure 5, which also meet the requirements.

[0103] Table 4 Simple linear regression analysis results after adjusting the calibrator assignments for data set B

[0104] Table 5 Summary of curve estimation results after adjusting the calibrator assignment for data set B

[0105] The technical solution of the present invention is not limited to the above-mentioned specific embodiments. Any technical variations made according to the technical solution of the present invention fall within the protection scope of the present invention.

Claims

1. A method for evaluating the detection quality of multiple sample analyzers, characterized in that: The evaluation methods include: Get reference material assignments for a series of reference materials used for a specified test item; Obtaining the detection value of each of the sample analyzers on the reference product; For each sample analyzer, determining a standard point by means of the reference sample value and the detection value of the sample analyzer to establish a conversion function for the sample analyzer; converting the detection values ​​of each of the sample analyzers using the conversion function corresponding to the sample analyzer to obtain conversion values ​​of each of the sample analyzers; The conversion value is used to perform a consistency comparison of the detection results between the sample analyzers to evaluate the detection quality of the sample analyzers.

2. The method according to claim 1, wherein: The reference values ​​are arranged in order from low to high and divided into multiple concentration gradient groups, and the standard points in each group are determined with the help of the detection values ​​corresponding to the reference values ​​in each group, and each standard point corresponds to a reference value and a detection value.

3. The method according to claim 1, wherein: A transfer function is established between the standard points of each sample analyzer by means of an interpolation method, wherein the interpolation method is in particular a cubic spline interpolation method.

4. The method according to claim 1, wherein: The total mean or total robust mean of the repeated detection values ​​of each reference product by each of the sample analyzers is used as the reference product value.

5. The method according to claim 2, wherein: The reference items are assigned values ​​and grouped according to the formula N=n / 10, where n is the total number of reference items, rounded up.

6. The method according to claim 5, wherein: In the grouping, the multiples of the upper and lower limits of each concentration range are calculated according to the following formula: Where X max The maximum value assigned to all references, X min The minimum value assigned to the reference product; Preferably, N≥6.

7. The method according to claim 2, wherein: The reference value in each group is assigned as the independent variable X, and the detection value of each sample analyzer for the reference is used as the dependent variable Y, and the mean of the percentage deviation [(Y / X-1)×100%] between Y and X in each group is calculated; Select a set of values ​​(X, Y) in each group whose percentage deviation between Y and X is closest to the mean of the percentage deviation of the group as the standard point, and determine a total of N standard points.

8. The method according to claim 7, wherein: The absolute value of the percentage deviation of X between two adjacent standard points among the N standard points is ≥10%.

9. The method according to claim 1, wherein: The conversion values ​​and the corresponding reference values ​​are used to draw a Bland-Altman plot, thereby performing a consistency comparison of the detection results between the various sample analyzers.

10. An evaluation device for evaluating the detection quality of multiple sample analyzers, the device comprising: processor; a memory having stored thereon a computer program running on the processor; Wherein, when the computer program is executed by the processor, the steps of the method according to any one of claims 1 to 9 are implemented.

Images