Method for analyzing precision and accuracy of indoor quality control data of multiple laboratory rooms

Abstract

The invention discloses a precision and accuracy analysis method for quality control data in a plurality of laboratory rooms, which performs statistical calculation on quality control test result information in the plurality of laboratory rooms through big data analysis and a statistical algorithm to realize identification of abnormal test results of the same quality control batch in different laboratories. Equipment system errors (inter-equipment variation) and random errors (intra-equipment variation) are distinguished through indexes and graphs, and the problem of precision or correctness can be distinguished, so that the problem can be conveniently, accurately and efficiently positioned, hospital teachers, customer service personnel and research and development personnel are assisted to quickly analyze and process the problem, and the problem can be accurately and efficiently solved. The efficiency of checking the problems of the experimental equipment by personnel is improved, and the labor cost is saved.

Description

Technical Field

[0001] The present invention relates to the field of medical testing technology, and is particularly applicable to a precision and accuracy analysis method for indoor quality control data of multiple laboratories. Background Art

[0002] Indoor quality control is an important part of laboratory quality management. It is usually carried out by multiple measurements of the same sample or a standard sample of known concentration in parallel with the sample to be tested to evaluate the stability, precision, accuracy and consistency of the experiment to determine whether the experimental results are reliable.

[0003] However, most current laboratory instruments only provide in-house quality control result analysis of the equipment on their own clients. Even LIS (Laboratory Information System) only provides quality control result analysis of the same equipment or samples for the equipment in the laboratory. In this case, if there is a problem with the quality control of a single device or the entire laboratory, it is not easy to be discovered, which can easily lead to the problem that the quality control test results of the laboratory instrument are not out of control and the sample test process does not report abnormal indicators, but the sample test results show abnormal problems. However, since neither the client nor the LIS system outputs abnormal information, the cycle for staff to analyze and deal with problems will be greatly extended.

[0004] If we can introduce comparison of internal quality control data among multiple laboratories, we will be able to know the differences and problems in the quality levels between our laboratory and other laboratories better and faster, which will help improve the precision and accuracy of internal quality control. Summary of the invention

[0005] The present invention aims to provide a method for analyzing the precision and accuracy of indoor quality control data among multiple laboratories, so as to solve the problem that the current indoor quality control cannot be compared among multiple laboratories, resulting in difficulty in discovering abnormalities.

[0006] To achieve the above object, the present invention adopts the following technical solutions: The method for analyzing the precision and accuracy of the internal quality control data of multiple laboratories of the present invention comprises the following steps: S1, through IoT devices and message queue technology, real-time acquisition of quality control data of multiple laboratories and multiple experimental equipment, including quality control results, quality control batches, quality control time, reagent batches, quality control levels and test methodology; S2, storing the quality control data of the multiple laboratories through the state management of the flink real-time computing framework; S3, using the keyBy operation of the flink real-time computing framework, divides the quality control data of multiple laboratories into peer groups, method groups, and batch groups according to the experimental equipment type, test methodology, and reagent batch; S4, according to the quality control analysis requirements, traverse the quality control data of multiple laboratories, and calculate the mean, standard deviation and equipment variation coefficient of the laboratory's internal quality control data and the inter-laboratory quality control data of multiple laboratories contained in the peer group, method group and batch group; S5, calculate the Z score, Bias value, CVR value and SDI value of the internal quality control data of the laboratory to be analyzed and the inter-laboratory quality control data of multiple laboratories in the peer group, method group and batch group; S6, outputs the mean distribution, CV value distribution, SDI&CVR graph, performance verification graph, Youden graph, and box-and-whisker graph of the laboratory's internal quality control data to be analyzed and the reference group.

[0007] Furthermore, the peer group includes all indoor quality control data of the same experimental equipment type; the method group includes all indoor quality control data of the same test methodology; and the batch group includes all indoor quality control data of the same experimental equipment type and reagent batch.

[0008] Furthermore, the mean distribution diagram is used to indicate the position of the mean value of the indoor quality control data of the laboratory to be analyzed in the sequence of mean values ​​of the indoor quality control data of multiple laboratories contained in the reference group; the CV value distribution is used to indicate the position of the equipment coefficient of variation of the indoor quality control data of the laboratory to be analyzed in the sequence of equipment coefficient of variation of multiple laboratories contained in the reference group.

[0009] Furthermore, the SDI&CVR graph uses the SDI value as the vertical axis and the CVR value as the horizontal axis; it also indicates the degree of deviation between the average value of the in-house quality control data of the laboratory to be analyzed and the average value of the inter-laboratory quality control data of multiple laboratories included in the reference group, as well as the degree of deviation between the equipment coefficient of variation of the in-house quality control data of the laboratory to be analyzed and the equipment coefficient of variation of multiple laboratories included in the reference group.

[0010] Furthermore, the performance verification graph uses the ratio of the Bias value to the allowable total error as the vertical axis and the ratio of the CV value to the allowable total error as the horizontal axis, indicating that the Sigma value of the laboratory's internal quality control data to be analyzed is located at the position of the multiple laboratory inter-laboratory quality control data contained in the reference group.

[0011] Furthermore, the horizontal and vertical axes of the Youden diagram are respectively the means of different gradients, which are used to indicate the outliers of the laboratory's internal quality control data to be analyzed for different quality control test gradients compared with the inter-laboratory quality control data of multiple laboratories contained in the reference group.

[0012] Furthermore, the box plot uses the quality control level as the horizontal axis and the average value of the indoor quality control data as the vertical axis, indicating the position of the experimental equipment of the indoor quality control data of the laboratory to be analyzed at different quality control levels relative to the inter-laboratory quality control data of multiple laboratories contained in the reference group.

[0013] Furthermore, it also includes determining the precision and accuracy of the experimental equipment by using the CV value of the internal quality control data of the laboratory to be analyzed relative to the deviation trend of the inter-laboratory quality control data of multiple laboratories contained in the reference group.

[0014] Furthermore, the reference group includes a peer group, a method group and a batch group.

[0015] The advantages of the present invention are that it performs statistical calculations on the indoor quality control test result information of multiple laboratories through big data analysis and statistical algorithms, realizes the identification of abnormal test results of the same quality control batches between different laboratories, distinguishes equipment system errors (variation between equipment) and random errors (variation within equipment) through indicators and graphics, and can distinguish problems of precision or correctness, thereby facilitating accurate and efficient problem positioning, assisting hospital teachers, customer service personnel, and R&D personnel in quickly analyzing and processing problems, improving the efficiency of personnel in troubleshooting experimental equipment problems, and saving labor costs. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 is a flow chart of the method of the present invention.

[0017] Figure 2 It is a schematic diagram of the mean distribution drawn by the method described in the present invention.

[0018] Figure 3 It is a schematic diagram of the CV value distribution diagram drawn by the method of the present invention.

[0019] Figure 4 It is a schematic diagram of SDI&CVR drawn by the method of the present invention.

[0020] Figure 5 It is a performance verification diagram drawn by the method described in the present invention.

[0021] Figure 6 It is a Youden diagram schematically showing the method of the present invention.

[0022] Figure 7 It is a box plot diagram drawn by the method of the present invention.

[0023] Figure 8 It is a diagram showing the monthly coefficient of variation and the deviation trend drawn by the method of the present invention. DETAILED DESCRIPTION

[0024] The technical solutions in the embodiments of the present invention are described clearly and completely below. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0025] like Figure 1 As shown, the method for analyzing the precision and accuracy of the internal quality control data of multiple laboratories of the present invention comprises the following steps: S1, through IoT devices and message queue technology, obtains quality control data of multiple laboratories and multiple experimental equipment in real time, including quality control results, quality control batches, quality control time, reagent batches, quality control levels and test methodology.

[0026] S2, stores the quality control data of the multiple laboratories through the state management of the flink real-time computing framework.

[0027] S3, using the keyBy operation of the flink real-time computing framework, divide the quality control data of multiple laboratories into peer groups, method groups, and same-batch groups according to the experimental equipment type, test methodology, and reagent batch. The peer group includes all indoor quality control data of the same experimental equipment type; the method group includes all indoor quality control data of the same test methodology; the same-batch group includes all indoor quality control data of the same experimental equipment type and reagent batch.

[0028] S4, according to the quality control analysis requirements, traverse the quality control data of multiple laboratories, and calculate the mean, standard deviation and equipment variation coefficient of the laboratory's indoor quality control data and the inter-laboratory quality control data of multiple laboratories contained in the peer group, method group and batch group.

[0029] S5, calculate the Z score, Bias value, CVR value and SDI value of the internal quality control data of the laboratory to be analyzed and the inter-laboratory quality control data of multiple laboratories in the peer group, method group and batch group. The Z score indicates how many standard deviations of the reference group (peer group, method group or batch group) each quality control result differs from the mean of the reference group.

[0030] S6, outputs the mean distribution, CV value distribution, SDI&CVR graph, performance verification graph, Youden graph, and box-and-whisker graph of the laboratory's internal quality control data to be analyzed and the reference group.

[0031] like Figure 2 As shown, the mean distribution diagram is used to indicate the position of the mean value of the indoor quality control data of the laboratory to be analyzed in the sequence of the mean values ​​of the indoor quality control data of multiple laboratories contained in the reference group, that is, the position of the mean value of the indoor quality control data of the laboratory to be analyzed in the mean distribution diagram represents the situation of the mean value of the indoor quality control data of the laboratory to be analyzed in the entire reference group, and it can be intuitively seen whether its quality control deviation is too large. Figure 2 The horizontal axis represents the average value interval of the indoor quality control data, and the vertical axis represents the number of laboratories in the horizontal axis interval. Generally, when there are enough laboratories, the average value distribution of the laboratory's indoor quality control data is normally distributed or skewed normally distributed. Figure 2 The middle gray triangle is the interval where the average value of the laboratory's internal quality control data to be analyzed is located. It can be seen that the mean value of the laboratory's internal quality control data is at the peak of the normal distribution of the mean values ​​between multiple laboratories, that is, the center of the normal distribution. This proves that the mean value of the laboratory's quality control data is very good among multiple laboratories.

[0032] like Figure 3 As shown in FIG. 1 , the CV value distribution is used to indicate the position of the equipment coefficient of variation of the laboratory internal quality control data to be analyzed in the equipment coefficient of variation sequence of multiple laboratories included in the reference group. Similar to the mean distribution diagram. Figure 3 It can be seen that Figure 3 The horizontal axis represents the equipment coefficient of variation interval of indoor quality control data, and the vertical axis represents the number of laboratories in the horizontal axis interval. When there are enough laboratories, the distribution quantity graph of the equipment coefficient of variation of the laboratory's indoor quality control data presents a normal distribution or a skewed normal distribution. Figure 3 The middle gray triangle is the interval where the coefficient of variation of the equipment in the laboratory's internal quality control data to be analyzed is located. It can be seen that the mean of the laboratory's quality control CV value among multiple laboratories is at the peak of the normal distribution. However, the smaller the coefficient of variation value, the better, proving that this laboratory is better than most laboratories.

[0033] like Figure 4 As shown, the SDI&CVR graph uses the SDI value as the vertical axis and the CVR value as the horizontal axis; it indicates the degree of deviation between the average value of the indoor quality control data of the laboratory to be analyzed and the average value of multiple laboratories included in the reference group, as well as the degree of deviation between the equipment coefficient of variation of the indoor quality control data of the laboratory to be analyzed and the equipment coefficient of variation of multiple laboratories included in the reference group. Among them, SDI is achieved by comparing the mean value of the indoor quality control data of the laboratory to be analyzed with the mean value of multiple laboratories included in all reference groups. If the mean value of the indoor quality control data of the laboratory to be analyzed deviates significantly from the mean value of the reference group, it is necessary to check whether there is a problem with the quality control. CVR means that the coefficient of variation of the mean value of the indoor quality control data of the laboratory to be analyzed is compared with the equipment coefficient of variation of multiple laboratories included in the reference group. CVR>1 means that the imprecision of the quality control product of the indoor quality control data of this laboratory equipment is higher than the average imprecision of the same quality control product. Figure 4In the data set, according to the SDI value and CVR value, it is divided into 4 square areas, namely the acceptable area, the generally acceptable area, the barely acceptable area and the unacceptable area. When the SDI value and CVR value of the indoor quality control data of the laboratory to be analyzed fall into the acceptable area, it means that the status of the indoor quality control data of the laboratory to be analyzed is normal. When the SDI value and CVR value of the indoor quality control data of the laboratory to be analyzed fall into the generally acceptable area, it indicates that it may be necessary to test the detection system of the indoor quality control data of the laboratory to be analyzed. When the SDI value and CVR value of the indoor quality control data of the laboratory to be analyzed fall into the barely acceptable area, it indicates that it is recommended to test the detection system of the indoor quality control data of the laboratory to be analyzed. When the SDI value and CVR value of the indoor quality control data of the laboratory to be analyzed fall into the unacceptable area, necessary corrective measures need to be taken on the detection system of the indoor quality control data of the laboratory to be analyzed.

[0034] The performance verification graph uses the ratio of the Bias value to the allowable total error TEa as the vertical axis and the ratio of the CV value to the allowable total error TEa as the horizontal axis, indicating the position of the Sigma value of the laboratory's internal quality control data to be analyzed in the inter-laboratory quality control data of multiple laboratories contained in the reference group. Figure 5 As shown, the Sigma level of the laboratory's internal quality control data to be analyzed can be intuitively seen from the performance verification graph. According to the position of the laboratory's internal quality control data points to be analyzed, it can be determined whether to improve the degree of bias (i.e., correctness) or the degree of variation (i.e., precision) first.

[0035] Since Bias / Tea = -σCV / Tea+1, Figure 5 The slope of the straight line is the value of σ (Sigma). Figure 5 The five straight lines from left to right along the horizontal axis are based on the inter-laboratory quality control data of multiple laboratories in the reference group, and their slopes are 6σ, 5σ, 4σ, 3σ, and 2σ, respectively. The σ value is the σ value of the reference group. Figure 5 As shown by the middle dot, which falls below the third straight line, we can intuitively see the Sigma level of the laboratory's internal quality control data to be analyzed, and specifically determine whether to improve the degree of bias (i.e., accuracy) or the degree of variation (i.e., precision) first.

[0036] The horizontal and vertical axes of the Youden diagram are the means of different gradients, respectively, and are used to indicate the outliers of the laboratory internal quality control data to be analyzed for different quality control test gradients compared with the inter-laboratory quality control data of multiple laboratories contained in the reference group. Figure 6As shown in the figure, the horizontal axis is the mean of gradient 2, the vertical axis is the mean of gradient 1, and the three ellipses are 95%, 99%, and 99.9% confidence intervals from outside to inside. The green dots in the figure represent the quality control of the laboratory's indoor quality control data for a certain device to be analyzed this month, and the gray represents the quality control of the indoor quality control data of other devices this month. The '×' in the upper right or lower left of the ellipse indicates that the results of the two gradients are abnormally high or low, indicating that the equipment contains obvious systematic errors (i.e., inter-equipment variation). The laboratory should look for the cause of accuracy from the aspects of standard substances, equipment, methods, etc.

[0037] The '▲' at the upper left or lower right of the ellipse indicates two gradient results, one high and one low, indicating equipment with obvious random errors (i.e., intra-equipment variation). The laboratory should look for reasons for precision from aspects such as equipment, methods, environment, and personnel.

[0038] like Figure 7 As shown, the box plot uses the quality control level as the horizontal axis and the average value of the indoor quality control data as the vertical axis, indicating the position of the experimental equipment of the indoor quality control data of the laboratory to be analyzed at different quality control levels relative to the inter-laboratory quality control data of multiple laboratories contained in the reference group. In the figure, Q1, Q2, and Q3 are three quality control levels, and each quality control level contains a box plot of the average value distribution of the indoor quality control data of the peer group and the method group. The dots indicate that the indoor quality control data of the laboratory to be analyzed is located in the position of the inter-laboratory quality control data of multiple laboratories contained in the current reference group.

[0039] The present invention also includes utilizing the deviation trend of the CV value of the indoor quality control data to be analyzed relative to the inter-laboratory quality control data of multiple laboratories contained in the reference group, such as Figure 8 As shown, determine the precision and accuracy of the experimental equipment. Figure 8 The horizontal axis is time, and the graph represents the data collected from August to July of the following year. The vertical axis is the percentage of the CV value relative to the deviation trend of the inter-laboratory quality control data of multiple laboratories contained in the reference group. Through the observation of the deviation trend for one year, it can be seen that the deviation of the CV values ​​of the current month and the historical month relative to the inter-laboratory quality control data of multiple laboratories contained in the reference group changes. If the current month is abnormally high compared with the historical month, it can be proved that the experiment of the current month is problematic relative to the historical month. The figure can also show which month has historical problems, and the laboratory quality control level can be traced back historically. If the CV changes greatly, the precision of the laboratory should be considered; if the bias changes greatly, the accuracy of the laboratory should be considered.

[0040] The reference group for comparison with the in-house quality control data to be analyzed can be data of a peer group, a method group, or a same batch group. It can also be compared and analyzed with the peer group, the method group, and the same batch group at the same time.

Claims

1. A method for analyzing the precision and accuracy of internal quality control data between multiple laboratories, characterized in that: The following steps are involved: S1, through IoT devices and message queue technology, real-time acquisition of quality control data of multiple laboratories and multiple experimental equipment, including quality control results, quality control batches, quality control time, reagent batches, quality control levels and test methodology; S2, storing the quality control data of the multiple laboratories through the state management of the flink real-time computing framework; S3, using the keyBy operation of the flink real-time computing framework, divides the quality control data of multiple laboratories into peer groups, method groups, and batch groups according to the experimental equipment type, test methodology, and reagent batch; S4, according to the quality control analysis requirements, traverse the quality control data of multiple laboratories, and calculate the mean, standard deviation and equipment variation coefficient of the laboratory's internal quality control data and the inter-laboratory quality control data of multiple laboratories contained in the peer group, method group and batch group; S5, calculate the Z score, Bias value, CVR value and SDI value of the internal quality control data of the laboratory to be analyzed and the inter-laboratory quality control data of multiple laboratories in the peer group, method group and batch group; S6, outputs the mean distribution, CV value distribution, SDI&CVR graph, performance verification graph, Youden graph, and box-and-whisker graph of the laboratory's internal quality control data to be analyzed and the reference group.

2. The method for analyzing the precision and accuracy of the internal quality control data of multiple laboratories according to claim 1, characterized in that: The peer group includes all the indoor quality control data of the same experimental equipment type; the method group includes all the indoor quality control data of the same test methodology; and the batch group includes all the indoor quality control data of the same experimental equipment type and reagent batch.

3. The method for analyzing the precision and accuracy of internal quality control data among multiple laboratories according to claim 1, characterized in that: The mean distribution diagram is used to indicate the position of the mean value of the indoor quality control data of the laboratory to be analyzed in the sequence of mean values ​​of the indoor quality control data of multiple laboratories contained in the reference group; the CV value distribution is used to indicate the position of the equipment coefficient of variation of the indoor quality control data of the laboratory to be analyzed in the sequence of equipment coefficient of variation of multiple laboratories contained in the reference group.

4. The method for analyzing the precision and accuracy of the internal quality control data of multiple laboratories according to claim 1, characterized in that: The SDI&CVR graph uses the SDI value as the vertical axis and the CVR value as the horizontal axis; it also indicates the degree of deviation between the average value of the in-house quality control data of the laboratory to be analyzed and the average value of the inter-laboratory quality control data of multiple laboratories included in the reference group, as well as the degree of deviation between the equipment coefficient of variation of the in-house quality control data of the laboratory to be analyzed and the equipment coefficient of variation of multiple laboratories included in the reference group.

5. The method for analyzing the precision and accuracy of internal quality control data among multiple laboratories according to claim 1, characterized in that: The performance verification graph uses the ratio of the Bias value to the allowable total error as the vertical axis and the ratio of the CV value to the allowable total error as the horizontal axis, indicating that the Sigma value of the laboratory's internal quality control data to be analyzed is located at the position of the multiple laboratory inter-laboratory quality control data contained in the reference group.

6. The method for analyzing the precision and accuracy of internal quality control data among multiple laboratories according to claim 1, characterized in that: The horizontal axis and vertical axis of the Youden diagram are respectively the means of different gradients, which are used to indicate the outliers of the laboratory's internal quality control data to be analyzed for different quality control test gradients compared with the inter-laboratory quality control data of multiple laboratories contained in the reference group.

7. The method for analyzing the precision and accuracy of internal quality control data among multiple laboratories according to claim 1, characterized in that: The box plot uses the quality control level as the horizontal axis and the average value of the indoor quality control data as the vertical axis, indicating the position of the experimental equipment of the indoor quality control data of the laboratory to be analyzed at different quality control levels relative to the inter-laboratory quality control data of multiple laboratories contained in the reference group.

8. A method for analyzing the precision and accuracy of internal quality control data of multiple laboratories according to any one of claim 1, characterized in that: It also includes determining the precision and accuracy of the experimental equipment by using the CV value of the internal quality control data of the laboratory to be analyzed relative to the deviation trend of the inter-laboratory quality control data of multiple laboratories contained in the reference group.

9. A method for analyzing precision and accuracy of internal quality control data of multiple laboratories according to any one of claims 3 to 8, characterized in that: The reference groups include peer groups, method groups and batch groups.

Images