A method for automatically auditing difference checking rules based on dynamic analysis of variant design

Abstract

This invention provides a method for automatically verifying difference checks based on dynamic analysis of variation design, which relates to the field of laboratory medicine. The method includes the following steps: (1) determining the distribution of historical data; (2) determining the coefficient of biological variation (CV) within individuals. I (3) Calculate the coefficient of variation (CV) of the dynamic analysis system. A (4) Based on the distribution of historical data, select the RCV calculation formula to obtain the limit line result; (5) Use the calculated detection limit to determine whether the difference between the two results should be automatically reviewed. The method of the present invention has a small amount of calculation, reduces the complexity of the difference review rules, and reduces the load of the entire automatic review process; in addition, the method of calculating the detection limit of the present invention has a theoretical basis and strong universality; at the same time, the automatic review difference review rules designed by the method of the present invention have high sensitivity and specificity, which is more conducive to the automatic judgment of difference review and has good application prospects in laboratory medicine.

Description

Technical Field

[0001] This invention relates to the field of laboratory medicine, specifically to a method for automatically reviewing difference verification rules based on dynamic analysis of variation design. Background Technology

[0002] Automated review refers to the automatic issuance of test reports by computers following validated rules or logic. This process requires pre-setting review rules. One rule in the data analysis process compares the differences between two results for the same patient, called difference verification. According to industry standards in my country, most laboratories design difference verification rules primarily based on pre-defined percentage deviations and / or absolute deviations.

[0003] The main difference verification rule setting schemes at home and abroad are as follows: (1) Empirical Difference Method (EDC): Based on existing experience, the detection limit is set to 0-200%. If the detection limit is exceeded, the rule is triggered and the system enters the pending review area; if the detection limit is not exceeded, the system enters the automatic review area. The advantage of this method is that it is simple to set up. The disadvantage is that it relies heavily on experience and there is no clear theoretical basis. (2) Weighted Revised Difference Method (RwCDI): The weight is calculated by transforming the data of the detection results. The weight formula is w = 1 / (aSD) 2 ), and then by setting different batches, such as 5 tests / batch, 10 tests / batch, 22 tests / batch, the final weight is calculated for judgment. The advantage of this method is that the theory is relatively complete and the sensitivity and specificity are also high. The disadvantage is that it depends on the calculation of data in the early stage and the method system is relatively complex. (3) Machine learning method (ML): Establish a machine learning model, train the model in the existing laboratory, and then make a judgment based on the model results. This method improves the sensitivity and specificity through machine learning, but it is too complex. Not all laboratories have enough data and hardware and software capabilities to implement machine learning. (4) Reference change value method (RCV, also known as reference coefficient of variation): The theoretical relative deviation obtained through biological variation is the reference change value, and the reference change value is used as the difference between the two results. This method has strong applicability, but the sensitivity and specificity based on RCV are not high at present. The main reason is that RCV is based on the data of normal distribution, while more clinical data is not normally distributed and often shows a non-normal distribution. In addition, one of the parameters of RCV calculation is CV. A (Analysis of variation) When calculating, a fixed result is used. However, when applied to automated auditing, other means are not single, fixed values, and the corresponding CV... A The levels will differ. Therefore, using a fixed CV... A When the calculated RCV is ultimately applied to automated auditing, it will introduce significant errors. Therefore, it is clear that the existing difference verification rules all have certain shortcomings.

[0004] A new method for setting difference verification rules has been found that does not require hardware with large computing power. Its detection limit has a sound theoretical basis, the theoretical model can be applied to most laboratories, and the detection limit has high sensitivity and specificity in most laboratories, which is of great significance in laboratory medicine. Summary of the Invention

[0005] The purpose of this invention is to provide a method for automatically reviewing difference verification rules based on dynamic analysis of variation design.

[0006] This invention provides a method for automatically verifying difference check rules based on dynamic analysis of variation design, which includes the following steps:

[0007] (1) Determine whether the distribution of historical data is normal, right-skewed, or left-skewed;

[0008] (2) Determine the coefficient of biological variation (CV) within an individual. I ;

[0009] (3) Calculate the coefficient of variation (CV) of the dynamic analysis system. A ;

[0010] (4) Based on the distribution of historical data, select the RCV calculation formula to obtain the limit line result;

[0011] (5) Use the calculated detection limit to determine whether the difference between the two results should be automatically checked.

[0012] Furthermore, in step (1), the sample size of the historical data is ≥50.

[0013] Further, in step (2), the determination of CV I It is obtained through a biological variation database; or calculated based on existing laboratory data.

[0014] Furthermore, in step (3), the calculation of dynamic CV A It involves designing a mean systematic error fitting experiment based on historical data to obtain the systematic error CV. A The fitting formula is used; each time a rule judgment is performed, the mean of the current result is substituted into the fitting curve to simulate and calculate the dynamic CV. A .

[0015] Further, in step (4), the RCV calculation formula for the normal distribution is as follows:

[0016]

[0017] Where RCV represents the reference variation value of the normal distribution; Z αCV represents the critical value for probability α in a standard normal distribution. I CV represents the coefficient of biological variation within an individual. A This represents the coefficient of variation of the analysis system;

[0018] And / or, the RCV calculation formula for the right-skewed distribution is as follows:

[0019]

[0020] Among them, InN-RCV R Z represents the reference variation value of the right-skewed distribution within a non-normal distribution; α CV represents the critical value for probability α in a standard normal distribution. I CV represents the coefficient of biological variation within an individual. A The coefficient of variation of the analysis system is represented by e; the natural constant is represented by ln; and the natural logarithm is represented by ln.

[0021] And / or, the RCV calculation formula for the left-skewed distribution is as follows:

[0022]

[0023] InN-RCV L Z represents the reference variation value for the left-skewed distribution within a non-normal distribution; α CV represents the critical value for probability α in a standard normal distribution. I CV represents the coefficient of biological variation within an individual. A represents the coefficient of variation of the analysis system; e represents the natural constant; ln represents the natural logarithm.

[0024] Furthermore, in step (5), if the difference between the two results is less than or equal to the detection limit, the result enters the automatic review area; if the difference between the two results is greater than or equal to the detection limit, the result enters the non-automatic review area.

[0025] Furthermore, in step (5), the difference between the two results is the value of the second result minus the value of the first result.

[0026] The present invention also provides a system for automatically reviewing difference verification rules based on dynamic analysis of variation design, wherein the system executes the aforementioned method.

[0027] The present invention also provides a computer-readable storage medium having stored thereon a computer program for implementing the aforementioned system for automatically auditing difference verification rules based on dynamic analysis of variation design.

[0028] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0029] (1) Simplified calculation: According to the design method of the present invention, the limit line of the difference verification rule can be set in advance. The calculation of distribution judgment and biological variation coefficient is outside the review process. There is no need to design a large number of models, which also reduces the amount of calculation, reduces the complexity of the difference verification rule, and reduces the load of the entire automatic review process.

[0030] (2) It is convenient for most laboratories to use. Biological variation data can be obtained from biological variation databases. The calculation method has a theoretical basis and is widely recognized by researchers at home and abroad. The reference variation values ​​derived from biological variation data also have good reliability and can be accepted by most laboratories.

[0031] (3) The current RCV (Regional Value Verification) method has gained some acceptance as a difference verification rule, but its sensitivity and specificity are relatively low. Some literature reports that the area under the ROC curve (AUC) is ML > RwCDI >> EDC > RCV, while other literature reports that the AUC for some items varies between 0.65 and 0.87. The automatic difference verification rule designed in this invention has high sensitivity and specificity.

[0032] In summary, this invention provides a method for automatically reviewing difference verification rules based on dynamic analysis of variation. This method requires less computation, reduces the complexity of the difference verification rules, and lightens the load on the entire automatic review process. Furthermore, the method for calculating the detection limit in this invention has a theoretical basis and strong universality. At the same time, the automatic review difference verification rules designed by this invention have high sensitivity and specificity, which is more conducive to the automatic judgment of difference verification and has good application prospects in laboratory medicine.

[0033] Obviously, based on the above description of the present invention, and according to common technical knowledge and conventional methods in the field, various other modifications, substitutions or alterations can be made without departing from the basic technical concept of the present invention.

[0034] The following detailed embodiments further illustrate the above-described content of the present invention. However, this should not be construed as limiting the scope of the present invention to the following examples. All technologies implemented based on the above-described content of the present invention fall within the scope of the present invention. Attached Figure Description

[0035] Figure 1 The flowchart shows the design of the automatic audit difference verification rules for accuracy level one in Embodiment 1 of the present invention.

[0036] Figure 2 This is a flowchart illustrating the automatic audit difference verification rule design for accuracy level two of Embodiment 1 of the present invention.

[0037] Figure 3The flowchart shows the design of the automatic audit difference verification rules for accuracy level one in Embodiment 2 of the present invention.

[0038] Figure 4 This is a scatter plot showing the difference between the initial and final results, along with the range of variation of the difference under each calculation formula.

[0039] Figure 5 The ROC curves are for each calculation formula.

[0040] Figure 6 Evaluation of ROC curve models for each calculation formula. Detailed Implementation

[0041] This invention provides a method for automatically reviewing difference verification rules based on dynamic analysis of variation design, which can achieve two levels of accuracy. Accuracy Level 1: Setting difference verification rules based on existing publicly available biological variation data; Accuracy Level 2: Setting difference verification rules based on more accurate biological variation data calculated from existing laboratory data.

[0042] Example 1: Automatically Reviewing Difference Verification Rules Based on Dynamic Analysis of Variation

[0043] The design flowchart of the automatic difference verification rules of this invention is as follows: Figure 1 As shown, this rule design is based on existing publicly available biological variation data to set difference verification rules. The specific design method is as follows:

[0044] (1) Conduct statistical analysis on the project using existing laboratory data to determine the distribution of existing laboratory data (the number of data to be analyzed is ≥50 cases, and random screening is required);

[0045] (2) Obtain the intra-individual coefficient of biological variation (CV) for the specified items by consulting biological variation databases. I );

[0046] (3) Combining the existing laboratory analysis system with the coefficient of variation (CV) A Calculate RCV. The formula for calculating RCV is as follows:

[0047] normal distribution:

[0048] Right-skewed distribution:

[0049] Left-skewed distribution:

[0050] In formulas a, b, and c, RCV represents the reference variation value of the normal distribution; InN-RCV R InN-RCV represents the reference variation value for a right-skewed distribution within a non-normal distribution.L Z represents the reference variation value for the left-skewed distribution within a non-normal distribution; α CV represents the critical value for probability α in the standard normal distribution (which can be found in the standard normal distribution Z-value table); I CV represents the coefficient of biological variation within an individual. A represents the coefficient of variation of the analysis system; e represents the natural constant; ln represents the natural logarithm.

[0051] CV A The calculation method is as follows: establish CV A Obtain CV from model data that varies with the mean. A The formula for the fitted curve that varies with the mean. Each time a rule is judged, the mean of the current result is substituted into the fitted curve to simulate and calculate the current dynamic CV. A Then calculate RCV.

[0052] (4) Based on the distribution determined by the existing laboratory data, select the corresponding RCV calculation formula to obtain the calculation result as the limit line. After comparing the difference between the two results with the limit line, classify the patient into the automatic review area or the non-automatic review area.

[0053] The above describes a method for automatically reviewing difference verification rules in the design of accuracy level one labs, which should meet the needs of most laboratories. For laboratories with higher requirements for difference verification rules, the accuracy level two design method can be used. The flowchart for this method is shown below. Figure 2 As shown, this rule design is based on existing laboratory data and adopts different calculation formulas according to different data distributions. It is more accurate than the RCV calculated by conventional methods, and the setting of difference verification rules is more reliable.

[0054] Example 2: Automatically Reviewing Difference Verification Rules Based on Dynamic Analysis of Variation

[0055] The design flowchart of the automatic difference verification rules of this invention is as follows: Figure 3 As shown, this rule design is based on Example 1, with one formula omitted. During the design process, based on the existing formulas that can calculate the mirror image of the original data, one formula is omitted, and only combinations of formulas a and b, or formulas a and c, are used. The implementation method is as follows:

[0056] Method 1: First determine the distribution of historical data. If it is a normal distribution, choose formula a for calculation. If it is right-skewed, choose formula b. If it is left-skewed, convert the sign of the difference between the current result and the historical result and then choose formula b.

[0057] Method 2: First determine the distribution of historical data. If it is a normal distribution, choose formula a for calculation. If it is left-skewed, choose formula c. If it is right-skewed, convert the sign of the difference between the current result and the historical result and then choose formula c.

[0058] The present invention can also perform regulatory design by omitting one formula based on the accuracy level two method of Example 1, according to the above method.

[0059] The following specific experimental examples demonstrate the beneficial effects of the present invention.

[0060] Experimental Example 1: Study on the effectiveness of the method for automatically reviewing difference verification rules designed in this invention

[0061] 1. Experimental method: Simulate right-skewed distribution data (right-skewed distribution data is relatively common, and other distributions are similar). The following uses right-skewed distribution data as an example.

[0062] Using literature ( A,Lian IA, The protocol and data for albumin testing mentioned in IH, Mikkelsen G. Testing the limits: the diagnostic accuracy of reference change values. Scand J Clin LabInvest. 2021 Jul;81(4):318-323. doi:10.1080 / 00365513.2021.1904517.Epub 2021 Mar31.PMID:33787419.) were used to set the systematic coefficient of variation (CV). A =6.4%, intra-individual biological coefficient of variation (CV) I =7.7%, mean 37.2 g / L. Referring to the reagent instructions, and using the performance claim data, a systematic coefficient of variation (CV) was constructed. A A formula for dynamically fitting changes was developed, and statistical analysis was performed using 100 experimental data points.

[0063] 2. Statistical Software: Data simulation was performed using Python 3.0, and the data was imported into Excel 2010 for storage. A scatter plot of the difference results versus the initial results was plotted using Excel 2010. The ROC curve was calculated using Python, and statistical analysis was performed using SPSS 26. P < 0.05 indicates the experimental results are statistically significant.

[0064] 3. Experimental Results:

[0065] Based on the performance specifications in the manual, a single-cycle right-skewed dynamic CV simulation was performed. AThe fitting formula was obtained as the mean changed, and the results are shown in Table 1.

[0066] Table 1: Indoor CV Changes Corresponding to Albumin-Serium Pools in Reagent Instructions

[0067]

[0068] Using power function simulation, the simulation fitting formula is obtained as follows:

[0069] Y = 0.1972 * x^- 0.368

[0070] R 2 =0.8091

[0071] Where x is the current mean, and Y is the system coefficient of variation (CV) as a function of the mean. A

[0072] By comparing the fitted data from the quantile regression using different RCV calculation formulas, scatter plots of the difference results versus the initial results were obtained, along with the upper and lower ranges of the difference under each calculation formula. The results are as follows: Figure 4 .

[0073] The validity of the judgment under each calculation formula was statistically analyzed using ROC curves, and the area under the curve (AUC) for different formulas was obtained. The results are shown in Table 2.

[0074] Table 2: Area under the ROC curve for each formula in determining the difference:

[0075]

[0076] Fit the constraint lines separately. See the description of the constraint lines below. Figure 5 The model was evaluated under each ROC curve, and the evaluation results are shown in [link to evaluation]. Figure 6 .

[0077] By repeating the experiment on the case, 100 sets of experimental results were obtained. After removing 4 sets of invalid results, 96 sets of experimental results were obtained. The statistics of the 96 sets of experimental results are shown in Table 3 below:

[0078] Table 3: Statistics of Area Under the ROC Curve Results for Each Judgment Formula in Repeated Experiments

[0079]

[0080]

[0081] By comparing the results calculated using the right-skewed dynamic CV formula with those calculated using the existing normal distribution, the variances of the two independent samples are homogeneous (F = 1.18, P = 0.279), t = -11.891, P < 0.05 (P = 0.000). The results are shown in Table 4.

[0082] Table 4: Statistical Comparison of Right-Skewed Dynamic CV Calculation Results and Traditional RCV Calculation Results

[0083]

[0084] 4. Results Analysis:

[0085] Systematic errors are simulated using serological experiments as stated in the instruction manual. The more serological test data collected, the more effective the fitting function becomes.

[0086] As can be seen from the data in Table 2, the right-skewed dynamic CV method yielded the highest AUC (AUC = 0.818) in this study, which is 24.13% higher than the AUC of the traditional normal formula (AUC = 0.659). Combined with the experimental results... Figure 6 The model had the highest quality assessment score of 0.69 (a score greater than 0.5 indicates a good model).

[0087] The results of repeated measurements also show that the mean AUC of the right-skewed dynamic CV assessment is the highest (mean_AUC = 0.7643), which is 14.54% higher than the mean AUC of the traditional normal formula assessment (mean_AUC = 0.6673). Combined with the t-test results, the difference between the two groups is statistically significant, indicating that the right-skewed dynamic CV assessment is superior to the traditional normal formula assessment.

[0088] In summary, this invention provides a method for automatically reviewing difference verification rules based on dynamic analysis of variation. This method requires less computation, reduces the complexity of the difference verification rules, and lightens the load on the entire automatic review process. Furthermore, the method for calculating the detection limit in this invention has a theoretical basis and strong universality. At the same time, the automatic review difference verification rules designed by this invention have high sensitivity and specificity, which is more conducive to the automatic judgment of difference verification and has good application prospects in laboratory medicine.

Claims

1. A method for automatically verifying difference checking rules based on dynamic analysis of variation design, characterized in that: It includes the following steps: (1) Determine whether the distribution of historical data is normal, right-skewed, or left-skewed; (2) Determine the coefficient of biological variation (CV) within an individual. I ; (3) Calculate the coefficient of variation (CV) of the dynamic analysis system. A ; (4) Based on the distribution of historical data, select the RCV calculation formula to obtain the detection limit result; (5) Using the calculated detection limit, determine whether the difference between the two results should be automatically checked; In step (3), the dynamic CV is calculated. A It involves designing a mean systematic error fitting experiment based on historical data to obtain the systematic error CV. A The fitting formula is used; each time a rule judgment is performed, the mean of the current result is substituted into the fitting curve to simulate and calculate the dynamic CV. A ; In step (4), the RCV calculation formula for the normal distribution is as follows: ； Where RCV represents the reference variation value of the normal distribution; Z α CV represents the critical value for probability α in a standard normal distribution. I CV represents the coefficient of biological variation within an individual. A This represents the coefficient of variation of the analysis system; The RCV calculation formula for the right-skewed distribution is as follows: Among them, InN-RCV R Z represents the reference variation value of the right-skewed distribution within a non-normal distribution; α CV represents the critical value for probability α in a standard normal distribution. I CV represents the coefficient of biological variation within an individual. A The coefficient of variation of the analysis system is represented by e; the natural constant is represented by ln; and the natural logarithm is represented by ln. The RCV calculation formula for the left-skewed distribution is as follows: Among them, InN-RCV L Z represents the reference variation value for the left-skewed distribution within a non-normal distribution; α CV represents the critical value for probability α in a standard normal distribution. I CV represents the coefficient of biological variation within an individual. A represents the coefficient of variation of the analysis system; e represents the natural constant; ln represents the natural logarithm.

2. The method according to claim 1, characterized in that: In step (1), the sample size of the historical data is ≥50.

3. The method according to claim 1, characterized in that: In step (2), the determination of CV I It is obtained through a biological variation database; or calculated based on existing laboratory data.

4. The method according to claim 1, characterized in that: In step (5), if the difference between the two results is less than or equal to the detection limit, the result enters the automatic review area; if the difference between the two results is greater than or equal to the detection limit, the result enters the non-automatic review area.

5. The method according to claim 4, characterized in that: In step (5), the difference between the two results is the value of the second result minus the value of the first result.

6. A system for automatically reviewing difference verification rules based on dynamic analysis of variation design, characterized in that: The system performs the method described in any one of claims 1 to 5.

7. A computer-readable storage medium, characterized in that: It stores a computer program for implementing the system of automatic auditing difference verification rules based on dynamic analysis variation design as described in claim 6.

Images