A method for evaluating data quality of automatic gas monitoring equipment

Abstract

The present invention relates to the field of environmental monitoring technology, and in particular to a method for evaluating the data quality of automatic gas monitoring equipment. The technical solution of the present invention includes: S1, deploying at least two devices of the same model to be tested under the same sampling branch of the same sampling main pipe and operating them simultaneously; S2, performing the following individual tests on each device: a single-point verification test, a real-time test gas spike recovery test, and a device measurement value correlation test; S3, calculating the comprehensive score of each device using different weighting methods based on the comparison results of the average concentration of the test gas and the quantitative limit of the device; S4, outputting a rating result for the device based on the comprehensive score.

Description

Technical Field

[0001] The present invention relates to the field of gas monitoring, and in particular to a method for evaluating data quality of automatic gas monitoring equipment. Background Art

[0002] The current methods used to evaluate the data quality of automatic gas monitoring equipment are mainly through single performance tests such as method detection limit, precision, accuracy, system residual, or single point verification to determine whether the requirements can be met.

[0003] Generally, the instrument under test must be calibrated before performance testing. For example, a single-item performance test involves repeatedly injecting a standard substance into the device after calibration. The corresponding performance indicator is then calculated based on the device's response and the corresponding test calculation method. A single-point verification involves injecting a standard substance of known target concentration into the device after calibration, either once or multiple times, and calculating the relative error between the device response and the target concentration.

[0004] Currently, these evaluation methods primarily utilize reference materials for testing and evaluation. While these materials can partially reflect instrument performance, their composition is relatively simple compared to the gas samples being measured. They typically contain only the target substance that the instrument can measure, along with a diluent that clearly does not interfere with the measurement. Therefore, tests using these reference materials only reflect the instrument's accuracy in measuring the reference material, not its reliability in testing the actual sample being measured. Furthermore, the results of existing evaluation methods are not intuitive, and the difficulty and cost of interpreting the results are high.

[0005] In summary, the purpose of the present invention is to provide a method for evaluating the data quality of gas automatic monitoring equipment that can comprehensively reflect the reliability of the equipment and can quickly and intuitively reflect the data quality results of the equipment. Summary of the Invention

[0006] In view of the above analysis, an embodiment of the present invention aims to provide a method for evaluating the data quality of automatic gas monitoring equipment, comprising:

[0007] S1. Deploy at least two devices of the same model to be tested under the same sampling branch of the same sampling main pipe and operate them simultaneously;

[0008] S2. Perform the following individual tests on each device:

[0009] (1) Single-point verification test: a standard substance with a known target concentration is introduced, the device response value is obtained, and the relative error with respect to the target concentration is calculated; the proportion of errors falling within the set range in all test times is counted to obtain a single-point verification score;

[0010] (2) Real-time test gas spike recovery test:

[0011] One device was selected as the spike recovery device, and the rest were used as reference devices;

[0012] The spike recovery device is fed with a mixture of the gas to be tested and a standard substance of known concentration, and the reference device is fed with the gas to be tested;

[0013] Calculate the spike recovery rate based on the measured values ​​of the two types of equipment and the concentration of the standard substance; count the proportion of spike recovery rates that meet the range and obtain the spike recovery score;

[0014] (3) Equipment measurement value correlation test:

[0015] Statistical monitoring period, except for the spike recovery test and single point verification test of each device measured value;

[0016] Perform correlation analysis on the measured values ​​and calculate the correlation coefficient R² and slope k;

[0017] R² and k are weighted by the preset weights and multiplied by the percentage to obtain the correlation score;

[0018] S3. Based on the comparison between the average concentration of the gas to be tested and the quantitative limit of the equipment, the comprehensive score of each device is calculated using different weighting methods;

[0019] S4. Output rating results for the device based on the comprehensive score.

[0020] In some embodiments, the single-point verification score is calculated based on the proportion of tests with a relative error below 50%.

[0021] In some embodiments, the spike recovery score is calculated based on the proportion of tests where the spike recovery is within the range of 60%-140%.

[0022] In some embodiments, in the device measurement correlation score, the weight of the correlation coefficient R² is between 0.2 and 0.8, and the weight of the slope k is between 0.2 and 0.8.

[0023] In some embodiments, when the average concentration of the gas to be measured is lower than the quantitative limit of the device, the comprehensive score of a single device is composed of a single-point verification and a spike recovery score, with the weight of the single-point verification being 20% ​​to 80% and the weight of the spike recovery score being 20% ​​to 80%.

[0024] In some embodiments, when the average concentration of the gas to be measured is higher than the quantitative limit of the equipment, the comprehensive score is composed of the single-point verification score, the spike recovery score and the equipment measurement value correlation score, and the respective weights are 20%~40%, 20%~40%, and 20%~40%, respectively.

[0025] In some embodiments, the correlation test performs correlation analysis based on monitoring data collected by two or more devices at corresponding moments.

[0026] In some embodiments, outputting a rating result for a device based on a comprehensive score includes:

[0027] The mapping relationship between score and rating results includes:

[0028] A: score >90;

[0029] B: 70~90;

[0030] C: 50~70;

[0031] D: <50.

[0032] In some embodiments, the total period of the single test is no less than 7 consecutive days.

[0033] The method provided by the present invention has at least the following advantages:

[0034] 1. This evaluation method combines the test results of three indicators: single-point verification, test gas spike recovery, and test gas equipment correlation, to comprehensively and objectively evaluate the reliability of the equipment for measuring the test gas;

[0035] 2. Use specific rating levels to reflect the final equipment evaluation results, and intuitively and quickly reflect the reliability of the equipment. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] The accompanying drawings are only for the purpose of illustrating particular embodiments and are not to be considered limiting of the present invention. Like reference symbols denote like parts throughout the drawings.

[0037] Figure 1 A schematic flow chart of a method for evaluating data quality of automatic gas monitoring equipment provided by an embodiment of the present invention;

[0038] Figure 2 A schematic diagram of a specific embodiment flow chart provided for an embodiment of the present invention. DETAILED DESCRIPTION

[0039] The preferred embodiments of the present invention will be described in detail below with reference to the accompanying drawings, wherein the accompanying drawings constitute a part of the present invention and are used together with the embodiments of the present invention to illustrate the principles of the present invention, but are not used to limit the scope of the present invention.

[0040] In the description of the embodiments of the present invention, it should be noted that, unless otherwise specified or limited, the term "connected" should be understood in a broad sense. For example, it can mean a fixed connection, a detachable connection, or an integral connection. It can be a mechanical connection, an electrical connection, a direct connection, or an indirect connection through an intermediate medium. Those skilled in the art will understand the specific meanings of the above terms in the present invention based on specific circumstances.

[0041] The terms "top," "bottom," "above," "below," and "on" used throughout the description refer to relative positions of components of a device, such as the relative positions of top and bottom substrates within a device. It will be understood that devices are multifunctional regardless of their orientation in space.

[0042] The working surface of the present invention can be a plane or a curved surface, can be inclined, or can be horizontal. For the convenience of description, the embodiment of the present invention is placed on a horizontal surface and used on the horizontal surface, and "high and low" and "up and down" are defined in this way.

[0043] The embodiment of the present invention aims to provide a method for evaluating the data quality of gas automatic monitoring equipment, such as Figure 1 Shown, including:

[0044] S1. Deploy at least two devices of the same model to be tested under the same sampling branch of the same sampling main pipe and operate them simultaneously;

[0045] S2. Perform the following individual tests on each device:

[0046] (1) Single-point verification test: a standard substance with a known target concentration is introduced, the device response value is obtained, and the relative error with respect to the target concentration is calculated; the proportion of errors falling within the set range in all test times is counted to obtain a single-point verification score.

[0047] Single-point verification assesses the device under test's ability to accurately measure a standard substance of known concentration. The procedure involves injecting a standard substance of known target concentration into the device under test once or multiple times after calibration. The relative error between the device response and the target concentration is calculated. The results of all single-point verifications are tallied and a score is assigned based on the percentage of single-point verifications with relative errors within a certain range divided by the total number of single-point verifications, multiplied by 100. This score represents the individual score for the single-point verification indicator.

[0048] (2) Real-time test gas spike recovery test:

[0049] One device was selected as the spike recovery device, and the rest were used as reference devices;

[0050] The spike recovery device is fed with a mixture of the gas to be tested and a standard substance of known concentration, and the reference device is fed with the gas to be tested;

[0051] The spike recovery rate was calculated based on the measured values ​​of the two types of equipment and the concentration of the standard substance; the proportion of spike recovery rates that met the range was counted to obtain the spike recovery score.

[0052] The real-time test gas spike recovery index evaluates whether the test equipment can accurately measure the test gas sample. Specific operation method: Select one of the multiple test equipment of the same model deployed under the same sampling main pipe and the same sampling branch pipe as the spike recovery equipment, and the remaining equipment is the reference equipment. The reference equipment only collects and analyzes the test gas, while the spike recovery equipment is responsible for collecting and analyzing the mixed gas of the test gas and the standard substance of known concentration. The actual spike amount is calculated based on the response of the reference equipment and the response of the spike equipment, and then the spike recovery rate is further calculated based on the known concentration of the added standard substance. The results of all spike recovery are counted, and points are assigned according to the ratio of the number of times the spike recovery rate is within a certain range to the total number of spike recovery times multiplied by one hundred. The scoring result is the individual score of the real-time test gas spike recovery index.

[0053] (3) Equipment measurement value correlation test:

[0054] Statistical monitoring period, except for the spike recovery test and single point verification test of each device measured value;

[0055] Perform correlation analysis on the measured values ​​and calculate the correlation coefficient R² and slope k;

[0056] R² and k are weighted by the preset weights and multiplied by the percentage to obtain the correlation score;

[0057] The correlation index of the measured values ​​of the gas equipment under test evaluates whether the measured values ​​between the instruments under test are comparable. Specific operation method: In addition to single-point verification and spike recovery during the test period, the monitoring data of the gas under test from multiple devices under the same sampling main pipe and the same sampling branch pipe are collected, and the correlation analysis is performed on all the monitoring data of multiple devices under test to calculate the correlation coefficient R 2 And the slope k. According to the correlation coefficient R 2 The calculated result of the slope k is multiplied by 100 to assign points. The assigned points result is the single score of the correlation index of the real-time measured value of the gas equipment to be tested.

[0058] S3. Based on the comparison between the average concentration of the gas to be tested and the quantitative limit of the equipment, the comprehensive score of each device is calculated using different weighting methods;

[0059] After obtaining the score of each individual indicator, a comprehensive score is given to each device under test. The specific algorithm is as follows: First, the overall average concentration of the gas under test during the test period is counted.

[0060] 1. If the average concentration of the gas under test during the test is less than the quantification limit of the equipment, the comprehensive score of the single device under test is calculated based on the single point verification and spike recovery indicators;

[0061] 2. If the average concentration of the gas under test during the test period is ≥ the quantification limit of the equipment, the comprehensive score of the single device under test is calculated based on the three indicators of single-point verification, spike recovery and correlation between the measured value of the gas under test equipment.

[0062] S4. Output rating results for the device based on the comprehensive score.

[0063] After calculating the comprehensive score for each device under test, the weighted comprehensive scores of all devices are then converted to an overall score for that category. Finally, the devices are rated based on the scores as follows: A for scores above 90, B for scores between 70 and 90, C for scores between 50 and 70, and D for scores below 50. A rating indicates reliable measurement results; B for somewhat reliable results; C for reference only; and D for unreliable results.

[0064] In order to further improve the scientificity and accuracy of the evaluation method of the present invention, the embodiment of the present invention takes into account different average concentrations of the gas to be measured when calculating the comprehensive score.

[0065] In the field of environmental monitoring, the "Limit of Quantification" (LOQ) is the lowest concentration at which an instrument can reliably perform quantitative analysis. If the actual sampled gas concentration is lower than this value, the device response signal will be weak and susceptible to noise, fluctuations, or interference. In this case, using measurement correlation (such as the correlation coefficient R² or slope) to analyze differences between devices will actually result in unstable and incomparable data, and the resulting evaluation may be based on "noise quality" rather than "performance quality."

[0066] This is like evaluating the accuracy of two thermometers. If you ask them to measure a tiny difference of 0.001°C, but their range is only accurate to ±0.1°C, then comparing their measured values ​​is meaningless. In this case, you should test their responsiveness by having them measure the temperature of a standard water sample with an artificially set offset (similar to adding a spike), rather than comparing which one is closer to the "true value."

[0067] In summary, this design has the following benefits:

[0068] 1. Avoid meaningless correlation analysis:

[0069] If the sampling concentration is too low, both devices A and B are at their extreme response state, and the correlation between the measured values ​​cannot represent the true performance - it is reasonable and professional not to include the correlation score in this case.

[0070] 2. Improve the accuracy and scientificity of evaluation:

[0071] Dynamically adjusting the evaluation structure through concentration judgment shows that this is a mechanism that adaptively adjusts the evaluation dimensions according to data quality, rather than a one-size-fits-all scoring method.

[0072] 3. Provide a basis for classifying device application scenarios:

[0073] Some devices may perform well at high concentrations but poorly at low concentrations. This mechanism can identify "applicable concentration scenarios" in advance.

[0074] In some embodiments, the single-point verification score is calculated based on the proportion of tests with a relative error below 50%. Preferably, in some embodiments, the single-point verification score is calculated based on the proportion of tests with a relative error below 20%.

[0075] In some embodiments, the spike recovery score is calculated based on the proportion of tests in which the spike recovery rate is within the range of 60%-140%. Preferably, in some embodiments, the spike recovery score is calculated based on the proportion of tests in which the spike recovery rate is within the range of 80%-120%.

[0076] In some embodiments, in the device measurement correlation score, the weight of the correlation coefficient R² is between 0.2 and 0.8, and the weight of the slope k is between 0.2 and 0.8. Preferably, in some embodiments, the weight of the correlation coefficient R² and the slope k are both 0.5.

[0077] In some embodiments, when the average concentration of the gas to be measured is below the device's limit of quantification, the comprehensive score for a single device is composed of the single-point verification and the spike recovery score, with the single-point verification weighted at 20% to 80% and the spike recovery score weighted at 20% to 80%. Preferably, in some embodiments, the single-point verification weighted at 30% and the spike recovery score weighted at 70%.

[0078] In some embodiments, when the average concentration of the gas being measured is above the device's limit of quantification, the comprehensive score is composed of the single-point verification score, the spike recovery score, and the device measurement correlation score, with their respective weights being 20%-40%, 20%-40%, and 20%-40%, respectively. Preferably, in some embodiments, the weights of the above three scores are 30%, 40%, and 30%, respectively.

[0079] In some embodiments, the correlation test performs correlation analysis based on monitoring data collected by two or more devices at corresponding moments.

[0080] In some embodiments, the total period of the single test is no less than 7 consecutive days.

[0081] In a specific embodiment provided by the present invention, Figure 2 As shown, the following process is included:

[0082] Two identical devices under test were deployed at the same branch of the same sampling manifold. The two devices were operated simultaneously for seven days. During the test, single-point verification, real-time test gas spike recovery, and correlation testing of the gas values ​​measured by the devices were performed as planned. The specific methods and scoring rules for each test are as follows.

[0083] Specific operation method of single-point verification: After the device under test is calibrated, standard substances with known target concentrations are introduced into the device under test multiple times daily as planned, and the relative error between the device response and the target concentration is calculated. The results of all single-point verifications over the 7-day test are counted and scored based on the ratio of the number of single-point verification results with a relative error within 20% to the total number of single-point verifications, multiplied by 100. This is the individual score of the single-point verification indicator.

[0084] Specific operation method of real-time test gas spike recovery: During the test period, one of the two test devices of the same model is selected daily as the spike recovery device, and the other device is the reference device. The reference device only collects and analyzes the test gas, while the spike recovery device is responsible for collecting and analyzing the mixed gas of the test gas and the standard substance of known concentration (introducing multiple standard substances of known concentrations at different concentrations). The actual spike amount is calculated based on the response of the reference device and the response of the spike device, and then the spike recovery rate is further calculated based on the known concentration of the added standard substance. The results of all spike recovery are counted, and points are assigned according to the ratio of the number of times the spike recovery rate is between 80% and 120% to the total number of spike recovery times multiplied by one hundred, which is the single score of the real-time test gas spike recovery index;

[0085] Specific operation method for correlation of measured values ​​of the gas equipment to be tested: In addition to single-point verification and spike recovery during the test period, the monitoring data of the gas to be tested of the two equipment to be tested under the same sampling main pipe and the same sampling branch pipe are counted. For all the monitoring data of the two equipment to be tested, a scatter plot is drawn for correlation analysis, and the correlation coefficient R is calculated. 2 And the slope k. According to R 2 The weights of and k are 0.5 respectively. After weighted calculation, multiply by 100 to assign a score, which is the single score of the correlation index of the measured value of the real-time gas equipment to be tested;

[0086] After obtaining the score of each individual indicator, a comprehensive score is given to each device under test. The specific algorithm is as follows: First, the overall average concentration of the gas under test during the test period is counted.

[0087] 1. If the average concentration of the gas under test during the test is less than the quantification limit of the device, the comprehensive score of the single device under test is calculated by weighting the single-point verification and spike recovery indicators: Single device comprehensive score = single-point verification score * 30% + spike recovery score * 70%;

[0088] 2. If the average concentration of the gas to be tested during the test is ≥ the quantification limit of the equipment, the comprehensive score of the single device to be tested is calculated based on the three indicators of single-point verification, spike recovery and correlation with the measured value of the gas to be tested equipment: Comprehensive score of single device = single-point verification score * 30% + spike recovery score * 40% * correlation with equipment measurement value * 40%.

[0089] After calculating the comprehensive score for each device under test, the average of the comprehensive scores of the two devices is taken to obtain the overall score for the device. Finally, the device is rated based on the score results as follows: A for scores above 90, B for scores between 70 and 90, C for scores between 50 and 70, and D for scores below 50. A rating indicates reliable measurement results; B for somewhat reliable results; C for reference only; and D for unreliable results.

[0090] The above description is only a preferred specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any changes or substitutions that can be easily thought of by any technician familiar with this technical field within the technical scope disclosed by the present invention should be covered by the scope of protection of the present invention.

Claims

1. A method for evaluating the data quality of automatic gas monitoring equipment, characterized in that: include: S1. Deploy at least two devices of the same model to be tested under the same sampling branch of the same sampling main pipe and operate them simultaneously; S2. Perform the following individual tests on each device: (1) Single-point verification test: a standard substance with a known target concentration is introduced, the device response value is obtained, and the relative error with respect to the target concentration is calculated; the proportion of errors falling within the set range in all test times is counted to obtain a single-point verification score; (2) Real-time test gas spike recovery test: One device was selected as the spike recovery device, and the rest were used as reference devices; The spike recovery device is fed with a mixture of the gas to be tested and a standard substance of known concentration, and the reference device is fed with the gas to be tested; Calculate the spike recovery rate based on the measured values ​​of the two types of equipment and the concentration of the standard substance; count the proportion of spike recovery rates that meet the range and obtain the spike recovery score; (3) Equipment measurement value correlation test: Statistical monitoring period, except for the spike recovery test and single point verification test of each device measured value; Perform correlation analysis on the measured values ​​and calculate the correlation coefficient R² and slope k; R² and k are weighted by the preset weights and multiplied by the percentage to obtain the correlation score; S3. Based on the comparison between the average concentration of the gas to be tested and the quantitative limit of the equipment, the comprehensive score of each device is calculated using different weighting methods; Among them, when the average concentration of the gas to be measured is lower than the quantitative limit of the equipment, the comprehensive score of a single device is composed of the single-point verification score and the spike recovery score, with the weight of the single-point verification score being 20% ​​to 80% and the weight of the spike recovery score being 20% ​​to 80%. When the average concentration of the gas to be measured is higher than the quantitative limit of the equipment, the comprehensive score is composed of the single-point verification score, the spike recovery score and the equipment measurement value correlation score, with their respective weights being 20% ​​to 40%, 20% to 40%, and 20% to 40%, respectively. S4. Output rating results for the device based on the comprehensive score.

2. The method for evaluating data quality of automatic gas monitoring equipment according to claim 1, characterized in that: The single-point verification score is calculated based on the proportion of tests with a relative error below 50%.

3. The method for evaluating data quality of automatic gas monitoring equipment according to claim 1, characterized in that: The spike recovery score is calculated based on the proportion of tests where the spike recovery is within the range of 60%-140%.

4. The method for evaluating data quality of automatic gas monitoring equipment according to claim 1, characterized in that: In the device measurement correlation score, the weight of the correlation coefficient R² is between 0.2 and 0.8, and the weight of the slope k is between 0.2 and 0.

8.

5. The method for evaluating data quality of automatic gas monitoring equipment according to claim 1, characterized in that: The correlation test performs correlation analysis based on monitoring data collected by two or more devices at corresponding moments.

6. The method for evaluating data quality of automatic gas monitoring equipment according to claim 1, characterized in that: Output rating results for devices based on comprehensive scores, including: The mapping relationship between score and rating results includes: A: score >90; B：70 ~ 90； C：50 ~ 70； D：<50。 7. The method for evaluating data quality of automatic gas monitoring equipment according to claim 1, characterized in that: The total period of the single test shall not be less than 7 consecutive days.

Images