A surface water monitoring data quality control statistics and trend prediction method and system

Abstract

The application provides a surface water monitoring data quality control statistics and trend prediction method and system, which comprises the following steps: constructing a standardized quality control data set of multi-category quality control sample data; performing multi-dimensional quality control data statistical analysis on the standardized quality control data set to obtain quality control statistical indexes corresponding to the multi-category quality control sample data; dynamically calculating the weights of various quality control statistical indexes based on historical quality control index data, calculating the comprehensive score of each city monitoring station and the laboratory quality control stability index; calculating the quality risk continuation trend index of the monitoring station according to the comprehensive score of the monitoring station, and optimizing the laboratory detection procedure if the quality risk continuation trend index is greater than a preset trend threshold. The application objectively distinguishes the laboratory analysis level of different city monitoring stations, considers the coupling interference between quality control indexes, improves the calculation accuracy of the comprehensive score of the monitoring station, predicts the data unqualified probability of sample analysis in the next monitoring period, and avoids monitoring quality risks in advance.

Description

Technical Field

[0001] This application relates to the field of water environment monitoring quality control technology, and in particular to a method and system for quality control statistics and trend prediction of surface water monitoring data. Background Technology

[0002] Currently, the national surface water sampling and testing separation project is operating routinely. Monitoring stations in various cities are required to regularly report internal quality control sample testing data, mainly including four types of quality control data: internal parallel samples, certified reference materials, spiked samples, and laboratory blank samples. At present, industry management methods mostly rely on post-event compliance assessments of single data points, simply reviewing whether a single quality control data point exceeds the limit, which has significant technical shortcomings. 1. Low data utilization: Only a single data point is judged as qualified or not; the massive historical quality control data is not summarized, statistically analyzed, or deeply mined, making it impossible to quantify the long-term testing capabilities of each monitoring station. 2. One-sided evaluation method: There is a lack of a unified and standardized comprehensive evaluation model, making it difficult to objectively distinguish the laboratory analysis level of monitoring stations in different cities and prefectures, and the control is highly subjective; 3. Lack of predictive and early warning capabilities: Existing technologies can only verify non-compliant data after the fact, and cannot predict the probability of non-compliant data occurring during subsequent sample testing at the monitoring station, thus failing to mitigate monitoring quality risks in advance; 4. Lagging quality control management: It is impossible to form a trend of quality control data changes, making it difficult to accurately locate hidden problems in the laboratory testing system, which is not conducive to the long-term rectification and optimization of the quality control capabilities of the monitoring station.

[0003] In summary, existing quality control technologies lack the functions of integrating and statistically analyzing multiple types of quality control data, quantifying and rating capabilities, and predicting trends, making it difficult to meet the refined, intelligent, and proactive quality control management requirements of the national sampling and testing separation project.

[0004] Therefore, the urgent technical problem to be solved is how to provide a method and system for quality control statistics and trend prediction of surface water monitoring data, so as to objectively distinguish the laboratory analysis level of monitoring stations in different cities, consider the coupling interference between quality control indicators, improve the calculation accuracy of the comprehensive score of monitoring stations, and predict the probability of data non-compliance in the next monitoring cycle, so as to avoid monitoring quality risks in advance. Summary of the Invention

[0005] The purpose of this application is to provide a method and system for quality control statistics and trend prediction of surface water monitoring data, so as to objectively distinguish the laboratory analysis level of monitoring stations in different cities, consider the coupling interference between quality control indicators, improve the calculation accuracy of the comprehensive score of monitoring stations, and predict the probability of data non-compliance of sample analysis in the next monitoring cycle, thereby avoiding monitoring quality risks in advance.

[0006] To achieve the above objectives, as a first aspect of this application, this application provides a method for quality control statistics and trend prediction of surface water monitoring data, the method comprising: Collect multi-category quality control sample data reported by monitoring stations in various cities within the national sampling and testing separation platform, and preprocess the multi-category quality control sample data to construct a standardized quality control dataset; Perform multi-dimensional statistical analysis of standardized quality control datasets to obtain quality control statistical indicators corresponding to multiple categories of quality control sample data; Based on historical quality control indicator data, the weights of various quality control statistical indicators are dynamically calculated. Based on the weights of various quality control statistical indicators and the quality control statistical indicators, the comprehensive score and laboratory quality control stability index of each city's monitoring station are calculated. Based on the comprehensive score and laboratory quality control stability index of each city's monitoring station, the monitoring stations are classified into capability levels. Based on the comprehensive score of the monitoring station, the quality risk persistence trend index of the monitoring station is calculated. If the quality risk persistence trend index is greater than the preset trend threshold, the laboratory testing procedure is optimized; otherwise, no optimization is required.

[0007] The surface water monitoring data quality control statistics and trend prediction method described above further includes: Historical continuous periodic quality control statistical data from monitoring stations were retrieved, and a time series trend model was constructed to analyze the fluctuation patterns of quality control sample data. Based on the time series trend model, the probability of data non-compliance in the sample analysis of the next monitoring cycle is calculated, and early warning is given to monitoring stations where the probability of non-compliance exceeds the preset threshold. Automatically generate visual quality control statistical reports.

[0008] The surface water monitoring data quality control statistics and trend prediction method described above includes preprocessing of multi-category quality control sample data, which includes: Use regular expressions to remove erroneous data from multi-category quality control sample data; Anomaly detection principles were used to identify and remove outliers from multiple categories of quality control sample data. Based on the site codes and testing timestamps of monitoring stations in various cities, duplicate data of multiple categories of quality control samples are removed; Heterogeneous data reported by different monitoring stations are uniformly converted into a standardized data structure containing fields such as station identifier, monitoring time, detection index, quality control type, measured value, and standard value, which serves as a standardized quality control dataset.

[0009] The surface water monitoring data quality control statistics and trend prediction method described above includes multi-dimensional quality control data statistical analysis of the standardized quality control dataset to obtain quality control statistical indicators corresponding to multiple categories of quality control sample data, including: For internal parallel samples, calculate the relative deviation of the measurement results for the internal parallel samples; For certified reference materials, calculate the relative error of the measurement results of the certified reference materials; For the spiked sample, calculate the spike recovery rate; For laboratory blank samples, obtain the detection value of the laboratory blank samples.

[0010] The surface water monitoring data quality control statistics and trend prediction method described above includes: dynamically calculating the weights of various quality control statistical indicators based on historical quality control indicator data; calculating the comprehensive score and laboratory quality control stability index of each city's monitoring station based on the weights and the quality control statistical indicators; and classifying the monitoring stations into capability levels based on the comprehensive score and laboratory quality control stability index of each city's monitoring station, including: Based on historical quality control indicator data, the weights of various quality control statistical indicators are dynamically calculated; Based on the weights of various quality control statistical indicators and the quality control statistical indicators themselves, calculate the comprehensive score of each city's monitoring station; Calculate the laboratory quality control stability index based on various quality control statistical indicators; The monitoring stations are classified into capability levels based on their comprehensive scores and laboratory quality control stability index.

[0011] The surface water monitoring data quality control statistics and trend prediction method described above includes retrieving historical continuous period quality control statistical index data from monitoring stations, constructing a time series trend model, and analyzing the fluctuation patterns of quality control sample data, including: Retrieve historical quality control data from the monitoring station for consecutive periods; Constructing time series trend models; Predict future cycle quality control index values ​​and fluctuation ranges by constructing a time series trend model; By combining historical fluctuation ranges and industry standards, an early warning threshold is set. The predicted quality control indicator values ​​are compared with the early warning threshold, and the fluctuation amplitude is compared with historical normal levels to determine whether an early warning is triggered.

[0012] The surface water monitoring data quality control statistics and trend prediction methods described above include the following methods for constructing time series trend models: Historical quality control statistical indicators were constructed into a time series in chronological order and divided into training and testing sets; The ADF unit root test is used to determine the stationarity of the series. For non-stationary series, successive differencing is performed to determine the difference order. For series with seasonality, additional seasonal differencing is performed and a seasonal period is set. The optimal order of the model is determined by combining the autocorrelation function, partial autocorrelation function, and AIC and BIC information criteria. The model parameters are fitted using maximum likelihood estimation or least squares method to complete model training.

[0013] The surface water monitoring data quality control statistics and trend prediction method described above includes the following visualized quality control statistical reports: Summary table of monthly / quarterly quality control indicators for each monitoring station; Heatmap of pass rates for each testing item; The ability rating list is arranged in descending order of overall score; The trend change curve shows the fluctuation trajectory of quality control indicators for each site over multiple periods.

[0014] As a second aspect of this application, this application provides a surface water monitoring data quality control statistics and trend prediction system, the system comprising: The standard dataset module is used to collect multi-category quality control sample data reported by monitoring stations in various cities within the national sampling and testing separation platform, and to preprocess the multi-category quality control sample data to construct a standardized quality control dataset. The quality control statistical indicator acquisition module is used to perform multi-dimensional quality control data statistical analysis on standardized quality control datasets and obtain quality control statistical indicators corresponding to multiple categories of quality control sample data. The data processor is used to dynamically calculate the weights of various quality control statistical indicators based on historical quality control indicator data, and calculate the comprehensive score and laboratory quality control stability index of monitoring stations in various cities based on the weights of various quality control statistical indicators and the quality control statistical indicators. Based on the comprehensive score and laboratory quality control stability index of monitoring stations in various cities, the processor is used to classify the monitoring stations into capability levels. The data processor is also used to calculate the quality risk persistence trend index of the monitoring station based on the comprehensive score of the monitoring station. If the quality risk persistence trend index is greater than the preset trend threshold, the laboratory testing program is optimized; otherwise, no optimization is required.

[0015] The surface water monitoring data quality control statistics and trend prediction system described above further includes: The module is used to retrieve historical continuous periodic quality control statistical index data from monitoring stations, construct time series trend models, and analyze the fluctuation patterns of quality control sample data. The early warning module is used to calculate the probability of non-compliance of sample analysis data in the next monitoring period based on the time series trend model, and to provide early warning prompts for monitoring stations where the probability of non-compliance exceeds a preset threshold. The generation module is used to automatically generate visual quality control statistical reports.

[0016] The beneficial effects achieved by this application are as follows: (1) This application standardizes and processes four types of heterogeneous data, namely internal parallel samples, certified reference materials, spiked samples and laboratory blank samples, to achieve the summary and statistics of massive historical quality control data.

[0017] (2) This application uses dynamic weighting to calculate the comprehensive score and quality control stability index, and divides the monitoring station into four capability levels: excellent, qualified, early warning and unqualified, so as to achieve an objective and standardized comprehensive evaluation.

[0018] (3) The dynamic weight of the quality control index in this application depends not only on its own historical error, but also on the coupling interference of other indicators. By analyzing the coupling interference of other indicators, the dynamic weight of the quality control index is calculated, which improves the accuracy and reliability of the dynamic weight calculation of the quality control index, thereby improving the accuracy of the comprehensive score calculation of the monitoring station.

[0019] (4) This application automatically increases the frequency of parallel sample testing and calculates the recommended testing frequency according to the optimal sampling theory, thereby automatically optimizing the laboratory testing procedure and improving the optimization effect. The adjusted testing parameters need to go through a verification cycle and the quality risk persistence trend index is recalculated. If the quality risk persistence trend index does not decrease, it is determined that non-random error is dominant, and the warning level is automatically upgraded and pushed to the management level.

[0020] (5) This application uses the standard normal cumulative distribution function to convert the degree of deviation of the predicted value from the qualified range into the benchmark probability of non-compliance. It uses the predicted standard deviation of the neighboring stations of the current monitoring station as an influencing factor and introduces spatial neighborhood correlation into the calculation of non-compliance probability, which improves the accuracy of the calculation of data non-compliance probability. When a batch of reagents is contaminated and causes multiple stations in the province to have systematic deviations at the same time, the traditional method only issues warnings to a few stations with large fluctuations. This method uses the average standard deviation of the neighboring stations to trigger warnings for all affected stations in advance. It realizes the early identification and full warning of systemic quality risks (such as reagent batch contamination and regional method deviations), and improves the timeliness and accuracy of warnings. Attached Figure Description

[0021] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this application. For those skilled in the art, other drawings can be obtained based on these drawings.

[0022] Figure 1 This is a flowchart illustrating a method for quality control statistics and trend prediction of surface water monitoring data, as described in an embodiment of this application.

[0023] Figure 2This is a schematic diagram of the structure of a surface water monitoring data quality control statistics and trend prediction system according to an embodiment of this application. Detailed Implementation

[0024] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the scope of protection of this application.

[0025] like Figure 1 As shown, this application provides a method for quality control statistics and trend prediction of surface water monitoring data, which includes the following steps: Step S1: Collect multi-category quality control sample data reported by monitoring stations in various cities within the national sampling and testing separation platform, and preprocess the multi-category quality control sample data to construct a standardized quality control dataset.

[0026] The multi-category quality control sample data includes: internal parallel samples, certified reference materials, spiked samples, laboratory blank samples, and other categories of quality control sample data.

[0027] Internal parallel samples are two or more identical subsamples of the same surface water sample, prepared under identical experimental conditions, by the same personnel, and using the same instruments. Pre-treatment and analysis are carried out simultaneously, serving as a precision quality control measure within the laboratory. The purpose of internal parallel samples is to verify the precision of the testing process (the consistency of repeated measurements) and to reflect random errors introduced by personnel operation, instruments, reagents, and the environment.

[0028] Certified reference materials (CRPs) are reference samples with known concentrations, assigned by authoritative institutions and accompanied by standard certificates and true values. Their composition and matrix are similar to actual surface water samples, making them universal quality control samples for water environment testing. The purpose of CRPs is to verify the accuracy of test results, determine whether there are systematic biases in instruments, methods, and reagents, and they are a core criterion for evaluating a laboratory's testing capabilities.

[0029] The spiked sample involves precisely adding a certain amount of a standard analyte to a known volume of actual surface water sample. This spiked sample is then pretreated and analyzed simultaneously with the original sample, and the theoretical and measured values ​​are compared. The purpose of the spiked sample is to verify the reliability of the entire analytical method and to eliminate errors caused by sample matrix interference, pretreatment losses, extraction / digestion, and other steps.

[0030] The laboratory blank sample refers to a sample obtained by not adding the surface water sample throughout the whole process, using only experimental pure water and reagents, and following the exact same pretreatment and detection procedures as the actual sample, which is divided into a reagent blank and a whole-process blank. The function of the laboratory blank sample is to troubleshoot exogenous pollution caused by the laboratory environment, experimental vessels, pure water, chemical reagents, and consumables, so as to avoid falsely high detection results caused by blank interference.

[0031] The preprocessing of multi-category quality control sample data includes: Step S110: Using a regular expression to eliminate data with format errors in the multi-category quality control sample data.

[0032] Step S120: Adopting the abnormal data discrimination principle to identify and eliminate abnormal outliers in the multi-category quality control sample data.

[0033] The abnormal data discrimination principle adopts the principle, the principle is an abnormal data discrimination criterion under normal distribution, also known as the Pauta criterion.

[0034] Step S130: Performing deduplication of duplicate data on the multi-category quality control sample data based on the station codes of the monitoring stations in each city and the detection timestamp.

[0035] Step S140: Uniformly converting heterogeneous data reported by different monitoring stations into a standardized data structure comprising fields of station identifier, monitoring time, detection index, quality control type, measured value and standard value, to serve as a standardized quality control data set.

[0036] Step S2: Performing multi-dimensional quality control data statistical analysis on the standardized quality control data set to obtain quality control statistical indicators corresponding to the multi-category quality control sample data.

[0037] Step S2 includes: Step S210: Calculating the relative deviation of the measurement results of the internal parallel samples for the internal parallel samples.

[0038] For the internal parallel samples, the relative deviation of the measurement results of the internal parallel samples is calculated to determine whether the data is qualified, which is used to evaluate the precision level of laboratory detection. The calculation formula for the relative deviation of the measurement results of the internal parallel samples is as follows: ; wherein, represents the relative deviation of the measurement results of the internal parallel samples; represents the measured value of the first subsample in the internal parallel samples; represents the measured value of the second subsample in the internal parallel samples.

[0039] As a specific embodiment of the present invention, when it is determined to be qualified.

[0040] Step S220: For certified reference materials, calculate the relative error of the measurement results of the certified reference materials.

[0041] For certified reference materials (CART), calculate the relative error of the test results, statistically analyze the accuracy, and use this as a representative measure of the detection accuracy. The formula for calculating the relative error is as follows: ; in, Indicates the relative error of the results measured by certified reference materials; This indicates the measured value of a certified reference material; Represents the standard truth value.

[0042] As a specific embodiment of the present invention, if If the error is less than the allowable error range specified in the certificate of reference material, the measurement of the certified reference material is deemed accurate.

[0043] Step S230: Calculate the spike recovery rate of the spiked sample.

[0044] For spiked samples, calculate the spike recovery rate, statistically analyze the recovery rate pass rate, and evaluate the overall reliability of the detection method. The formula for calculating the spike recovery rate of spiked samples is as follows: ; in, Indicates the spiked recovery rate of the spiked sample; Indicates the measured concentration of the spiked sample; This indicates the sample background value (i.e., the original content of the analyte in the sample before the addition of the standard substance); This indicates the concentration of the standard substance added to the sample.

[0045] As a specific embodiment of the present invention, when If the value is between 70% and 130%, the recovery capability of the analytical method for this spiked sample is considered acceptable.

[0046] Step S240: Obtain the detection value of the laboratory blank sample.

[0047] As a specific embodiment of the present invention, when the detection value of the laboratory blank sample is lower than the method detection limit, it is determined to be without pollution risk; when it is higher than the detection limit, it is classified into three levels of pollution risk: low, medium, and high, based on the multiple of exceedance. This evaluates the laboratory's pollution control capabilities.

[0048] The quality control statistical indicators include: the relative deviation of the results of internal parallel sample determinations, the relative error of the results of certified reference materials determinations, the spike recovery rate of spiked samples, and the detection value of laboratory blank samples.

[0049] Step S3: Based on historical quality control index data, dynamically calculate the weights of various quality control statistical indicators, and calculate the comprehensive score and laboratory quality control stability index of each city monitoring station according to the weights of various quality control statistical indicators and the quality control statistical indicators. Then, classify the monitoring stations into capability levels based on the comprehensive score and laboratory quality control stability index of each city monitoring station.

[0050] Step S3 includes: Step S310: Based on historical quality control index data, dynamically calculate the weights of various quality control statistical indicators.

[0051] The formulas for calculating the weights of various quality control statistical indicators are as follows: ; in, Indicates the current period T. Dynamic weights of quality control indicators; Indicates the first The benchmark weights of quality control indicators; It is a natural exponential function, used to achieve the exponential decay of weights with error; This represents the attenuation coefficient, used to characterize the sensitivity of the weight to the quality control anomaly index. In this embodiment... The value range is [0.1, 1.0], preferably... =0.5; Indicates the first The quality control abnormality index of each quality control indicator in the previous period. ;in, Indicates the first Target values ​​for quality control indicators; Indicates the previous period ( ) No. Measured values ​​of quality control indicators; This indicates taking the maximum value; Indicates the first Class of quality control indicators for the first The correlation influencing factors of quality control indicators are assigned values ​​based on the degree of coupling between indicators, with a value range of [0,1]. Indicates the relationship with the first The first quality control indicator associated with The quality control anomaly index of similar quality control indicators in the previous period; Indicates the relationship with the first The total number of quality control indicators associated with each quality control indicator.

[0052] The dynamic weights of the quality control indicators in this application depend not only on their own historical errors, but also on the coupling interference of other indicators. By analyzing the coupling interference of other indicators, the dynamic weights of the quality control indicators are calculated, which improves the accuracy and reliability of the calculation of the dynamic weights of the quality control indicators, thereby improving the accuracy of the calculation of the comprehensive score of the monitoring station.

[0053] For example, for internal parallel samples, the associated quality control indicators include certified reference materials (CART) and laboratory blanks. Blank contamination systematically increases the measured values ​​of both parallel samples, reducing precision deviation. Poor CART accuracy reflects instrument system bias, simultaneously affecting the absolute levels of both parallel sample values. For CART, the associated quality control indicators include laboratory blanks and spiked samples. Blank contamination causes baseline drift, resulting in systematically higher CART values. Abnormal spike recovery reflects pretreatment losses and systematic errors shared with CART measurements. For spiked samples, the associated quality control indicators include CART and internal parallel samples. CART accuracy serves as the benchmark for spike recovery calculation; poor parallel sample precision indicates large random errors, inevitably impairing the repeatability of spike recovery. For laboratory blanks, the associated quality control indicators include CART, internal parallel samples, and spiked samples. Blank contamination is a global source of interference, causing a systematic positive bias in all quantitative determinations.

[0054] Among them, the relative error benchmark weight of the test results of certified reference materials. This reflects the accuracy of the detection; the baseline weight of the spiked sample recovery rate. This reflects the reliability of the method; the relative deviation of the results from internal parallel sample determinations is weighted according to the benchmark. This reflects precision; the benchmark weight of the detection value of the laboratory blank sample. This reflects the level of pollution control.

[0055] Step S320: Calculate the comprehensive score of each city's monitoring station based on the weights of various quality control statistical indicators and the quality control statistical indicators themselves.

[0056] The formula for calculating the comprehensive score of the monitoring station is as follows: ; in, This indicates the overall score of the monitoring station; This indicates the total number of categories in the quality control sample data; Indicates the first Dynamic weights of quality control indicators; Indicates the first The percentage of non-compliance in the quality control statistical indicators of sample data.

[0057] in, ; This indicates the period within the statistical period (e.g., the past month, the past quarter, etc.). The pass rate of quality control statistical indicators for quality control sample data; It represents the sum of the non-compliance probabilities of quality control statistical indicators for all types of quality control sample data within the statistical period.

[0058] Step S330: Calculate the laboratory quality control stability index based on various quality control statistical indicators.

[0059] The formula for calculating the laboratory quality control stability index is as follows: ; in, Indicates the stability index of laboratory quality control; Indicates the first The percentage of non-compliance in quality control statistical indicators of sample data; This represents the average percentage of non-compliance rates for all types of quality control sample data. This represents the variance correction factor; The value should be between 0.5 and 2.0, preferably... 1. This represents the natural logarithm operator.

[0060] Step S340: Classify the monitoring stations into capability levels based on the comprehensive scores and laboratory quality control stability index of each city's monitoring stations.

[0061] As a specific embodiment of the present invention, if ,and The monitoring station will be rated as excellent. , The monitoring station will be rated as qualified. ,or The monitoring station will be rated as an early warning level. ,or The monitoring station will then be rated as unqualified.

[0062] Step S4: Calculate the quality risk continuation trend index of the monitoring station based on its comprehensive score. If the quality risk continuation trend index is greater than the preset trend threshold, the laboratory testing procedure will be optimized; otherwise, no optimization is required.

[0063] The formula for calculating the quality risk continuation trend index is as follows: ; in, An index indicating the continuation trend of quality risk; This indicates the overall score of the monitoring station in the previous period; This indicates the overall score of the monitoring station in the current period; This indicates the duration of each monitoring cycle; The preferred factor represents the influence of the non-compliance probability of quality control statistical indicators on the quality risk persistence trend index. ; This represents the sum of the non-compliance probabilities of all types of quality control sample data within the previous monitoring period.

[0064] If the quality risk continuation trend index is greater than the preset trend threshold, it is determined that the laboratory has a risk of quality inertia decline, and the laboratory testing procedures should be optimized; otherwise, there is no need to optimize the laboratory testing procedures.

[0065] like Methods for optimizing laboratory testing procedures when the trend exceeds a preset threshold include: Step S410: Automatically increase the detection frequency of parallel samples and calculate the recommended detection frequency based on the optimal sampling theory.

[0066] The recommended detection frequency calculation formula is as follows: ; in, Indicates the recommended testing frequency; Indicates the frequency of original parallel sample testing; This represents the standard deviation of the time series trend model's prediction for the next period. This indicates the upper limit of the allowed standard deviation.

[0067] Step S420: Increase the frequency of standard substance verification, increase the frequency of certified standard substance insertion, and add 1-2 quality control samples per batch.

[0068] Step S430: Monitor the density of blank samples, and add one full-process blank and one reagent blank for each batch.

[0069] Step S440: Add spike recovery verification to the abnormal indicators of spike recovery verification.

[0070] In step S450, the adjusted detection parameters need to undergo a verification cycle, and the quality risk persistence trend index is recalculated. If the quality risk persistence trend index does not decrease, it is determined that the error is not random, and the warning level is automatically upgraded and pushed to the management level.

[0071] Step S5: Retrieve historical continuous periodic quality control statistical index data from the monitoring station, construct a time series trend model, and analyze the fluctuation pattern of quality control sample data.

[0072] Step S5 includes: Step S510: Retrieve historical continuous period quality control data from the monitoring station.

[0073] The quality control data retrieved from the monitoring station for consecutive historical periods include: relative deviation of parallel samples, relative error of standard substances, spike recovery rate, blank detection value, etc. These are then summarized into a time series based on timestamps.

[0074] Step S520: Construct a time series trend model.

[0075] Specifically, ARIMA (Autoregressive Differential Integrated Moving Average) or SARIMA (Autoregressive Seasonal Integrated Moving Average) is used to construct time series trend models. The ARIMA model is used for quality control indicator series without significant seasonal fluctuations, while the SARIMA model is used for quality control indicator series with seasonal or periodic fluctuations. Trend analysis and short-term forecasting of quality control data are achieved through model fitting.

[0076] The methods for constructing time series trend models include: Step S521: Construct a time series of historical quality control statistical indicators in chronological order and divide them into training set and test set.

[0077] Step S522: Use the ADF unit root test to determine the stationarity of the sequence. Perform successive differencing on non-stationary sequences to determine the differencing order. For sequences with seasonality, perform additional seasonal differencing and set a seasonal cycle.

[0078] The ADF unit root test is an existing method, and will not be elaborated upon here.

[0079] Step S523: Combine the autocorrelation function, partial autocorrelation function, and AIC and BIC information criteria to determine the optimal order of the model.

[0080] Step S524: Fit the model parameters using maximum likelihood estimation or least squares method to complete model training.

[0081] The parameters of the time series trend model include: autoregression order, moving average order, and difference order.

[0082] Among them, AIC and BIC are the two most commonly used information criteria in statistical model selection. They are used to weigh goodness of fit against model complexity among multiple candidate models and select the optimal model.

[0083] Step S525: Use the Ljung-Box test to perform a white noise test on the model residuals to verify the model's effectiveness.

[0084] The Ljung-Box test is a multi-level joint test, using the Runge-Box test method. The Ljung-Box test is an existing method and will not be elaborated upon further here.

[0085] Step S526: Perform backtesting using the test set, and select the optimal time series trend model based on the prediction error index.

[0086] Step S530: Predict the quality control index values ​​and fluctuation range of future cycles using the constructed time series trend model.

[0087] Preferably, a time series trend model is used to predict the quality control indicator values ​​and fluctuation ranges for the next 2-3 periods. Based on the validated ARIMA / SARIMA model, predicted values ​​are generated for each of the next 2-3 monitoring periods. After model fitting, the prediction standard error is output, which reflects the inherent fluctuation of the predicted values.

[0088] A validated time series trend model is used to predict quality control indicators for future monitoring periods, yielding predicted indicator values ​​and standard errors. A 95% confidence interval is constructed based on the principle of normal distribution, and the difference between the upper and lower limits of the confidence interval is used as the predicted fluctuation range of the indicator. Simultaneously, the predicted coefficient of variation is calculated to characterize the relative degree of fluctuation. The predicted fluctuation range is compared with historical normal fluctuation thresholds; if it exceeds the threshold, the quality control data is deemed to have abnormal fluctuations, triggering a risk warning.

[0089] As an example, let's define the predicted value of the model output as... The prediction standard deviation is The relative degree of fluctuation The calculation method is as follows: ; in, A larger value indicates a higher proportion of fluctuation relative to the indicator itself, poor consistency in testing, weak overall quality control level in the laboratory, and prominent weaknesses. A smaller value indicates a lower percentage of fluctuation, better repeatability of test results, standardized laboratory operations and equipment management, and excellent quality control stability.

[0090] The absolute fluctuation range is 3.92. .

[0091] It should be explained that a larger absolute fluctuation range indicates a wider range of future indicator values, greater data fluctuations, poor stability of the testing process, significant random errors, a high risk of quality control instability, and a higher likelihood of unqualified data. This requires close attention and early warning.

[0092] It should be explained that the smaller the absolute fluctuation range, the more likely the future indicators will be concentrated in a narrow range, the data changes smoothly, and the state of instruments, personnel, reagents, pretreatment and other links is stable, and the quality control is good.

[0093] As a specific embodiment of the present invention, if both the absolute fluctuation amplitude and the relative fluctuation degree are large (exceeding the preset threshold), it indicates that the data fluctuates drastically, has poor stability, and the risk of non-compliance is increasing, triggering an early warning and investigating personnel, instruments, reagents, and experimental procedures. If both are small, routine monitoring is performed without intervention.

[0094] As a specific embodiment of the present invention, if the absolute fluctuation amplitude is too large (exceeding the preset threshold) and the relative fluctuation amplitude is too small (below the preset threshold), the overall value is too high, but the relative stability is still acceptable, and the risk of loss of control caused by random fluctuations is low. The focus should be on verifying the root causes of the high benchmark value (such as contamination, calculation bias, matrix enhancement effect), and there is no need to worry excessively about repeatability issues.

[0095] As a specific embodiment of the present invention, if the absolute fluctuation amplitude is too small (below the preset threshold) and the relative fluctuation amplitude is too large (exceeding the preset threshold), the index itself has a low value, but poor repeatability and weak stability. Even a small error can easily touch the qualified threshold, resulting in a high hidden risk. A stability warning is triggered, and priority is given to checking the trace detection links: cleanliness of glassware, purity of reagents, sensitivity of instruments, and operating methods. These scenarios are very prone to intermittent exceedances.

[0096] Step S540: Based on historical fluctuation ranges and industry standards, set an early warning threshold, compare the predicted quality control indicator value with the early warning threshold, compare the fluctuation amplitude with historical normal level values, and determine whether an early warning is triggered.

[0097] Specifically, if the predicted quality control index value and the warning threshold are reached, a trend warning is triggered; if the predicted fluctuation range exceeds the historical normal level, an instability warning is triggered.

[0098] Step S6: Based on the time series trend model, calculate the probability of data non-compliance in the sample analysis of the next monitoring period, and issue an early warning to the monitoring station where the probability of non-compliance exceeds the preset threshold.

[0099] The formula for calculating the probability of non-compliance in sample analysis for the next monitoring cycle is as follows: ； ; ; in, This indicates the probability that the data from the sample analysis in the next monitoring cycle will be substandard. It is the standard normal cumulative distribution function; Indicates the median of the acceptable range; Indicates the standard deviation of the acceptable range; Weighting factors that represent the risk values ​​of the predicted values ​​output by time series trend models; This represents the risk value of the predicted value from the time series trend model. This indicates the upper limit of the acceptable range for the predicted value; This represents the lower limit of the acceptable range for the predicted value; The weighting of the historical non-compliance frequency factor; This indicates the weight of the data volatility factor. This indicates the weighting of the instrument calibration cycle factor. Preferably, ; ; ; . Indicates the spatial fluctuation amplification factor, preferably... ; This represents the average of the forecast standard deviations of the current monitoring station's neighboring stations; This represents the forecast standard deviation for the current monitoring station. The forecast standard deviation is obtained from the output of the time series trend model.

[0100] This application uses the standard normal cumulative distribution function to convert the degree of deviation of predicted values ​​from the qualified range into a baseline probability of non-compliance. It incorporates the predicted standard deviation of neighboring stations as an influencing factor, introducing spatial neighborhood correlation into the calculation of the probability of non-compliance, thus improving the accuracy of the calculated probability of non-compliance. When contamination of a reagent batch causes systematic deviations to occur simultaneously at multiple stations across the province, traditional methods only issue warnings for a few stations with large fluctuations; this method, through the influence of the average standard deviation of neighboring stations, triggers warnings for all affected stations in advance. This achieves early identification and comprehensive warning of systemic quality risks (such as reagent batch contamination and regional method deviations), improving the timeliness and accuracy of warnings.

[0101] in, Historical non-compliance frequency factor: Statistically count the number of times non-compliance data occurred within the most recent (e.g., 6) monitoring periods; Data fluctuation amplitude factor: Calculates the relative fluctuation degree over the most recent (e.g., 6) monitoring periods; The instrument calibration cycle factor is the ratio of the number of days since the last calibration to the standard calibration cycle.

[0102] Step S7: Automatically generate a visual quality control statistical report.

[0103] Visualized quality control statistical reports include: Summary table of monthly / quarterly quality control indicators for each monitoring station; Heatmap of pass rates for each testing item; The ability rating list is arranged in descending order of overall score; The trend change curve chart shows the fluctuation trajectory of quality control indicators at each site over multiple periods; The rectification recommendations are automatically generated based on the risk type, including: When the deviation pass rate decreases, it is recommended to increase the frequency of parallel sample testing and personnel operation training; When the accuracy of reference materials decreases, it is recommended to check the expiration date of the reference materials and the instrument calibration status. When spiked recovery is abnormal, it is recommended to check the pretreatment process and matrix interference. When the risk of blank contamination increases, it is recommended to strengthen laboratory environmental control and reagent purity verification.

[0104] Example 2 like Figure 2 As shown, this application provides a surface water monitoring data quality control statistics and trend prediction system 100, which includes: Standard Dataset Module 10 is used to collect multi-category quality control sample data reported by monitoring stations in various cities within the national sampling and testing separation platform, and to preprocess the multi-category quality control sample data to construct a standardized quality control dataset.

[0105] The quality control statistical indicator acquisition module 20 is used to perform multi-dimensional quality control data statistical analysis on the standardized quality control dataset to obtain the quality control statistical indicators corresponding to the quality control sample data of multiple categories. The data processor 30 is used to dynamically calculate the weights of various quality control statistical indicators based on historical quality control indicator data, and to calculate the comprehensive score and laboratory quality control stability index of monitoring stations in various cities based on the weights of various quality control statistical indicators and the quality control statistical indicators, and to classify the monitoring stations into capability levels based on the comprehensive score and laboratory quality control stability index of monitoring stations in various cities.

[0106] The data processor 30 is also used to calculate the quality risk continuation trend index of the monitoring station based on the comprehensive score of the monitoring station. If the quality risk continuation trend index is greater than the preset trend threshold, the laboratory testing program is optimized; otherwise, no optimization is required.

[0107] Module 40 is used to retrieve historical continuous periodic quality control statistical index data from monitoring stations, construct time series trend models, and analyze the fluctuation patterns of quality control sample data.

[0108] The early warning module 50 is used to calculate the probability of data non-compliance in the sample analysis of the next monitoring period based on the time series trend model, and to issue early warning prompts to monitoring stations where the probability of non-compliance exceeds a preset threshold.

[0109] Module 60 is used to automatically generate visual quality control statistical reports.

[0110] This application also provides a computer storage medium storing computer instructions, which, when invoked, execute the address mapping method of the large-capacity solid-state drive. The computer storage medium includes one or more program instructions, which are executed by a processor as a method for quality control statistics and trend prediction of surface water monitoring data.

[0111] The embodiments disclosed in this invention provide a computer-readable storage medium storing computer program instructions. When the computer program instructions are executed on a computer, the computer performs the above-described method for quality control statistics and trend prediction of surface water monitoring data.

[0112] This invention provides a processor for processing the above-described method for quality control statistics and trend prediction of surface water monitoring data.

[0113] In this embodiment of the invention, the processor can be an integrated circuit chip with signal processing capabilities. The processor can be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0114] The various methods, steps, and logic diagrams disclosed in the embodiments of this invention can be implemented or executed. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this invention can be directly implemented by a hardware decoding processor, or implemented by a combination of hardware and software modules in the decoding processor. The software modules can reside in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. The processor reads information from the storage medium and, in conjunction with its hardware, completes the steps of the above methods.

[0115] The storage medium can be memory, such as volatile memory or non-volatile memory, or may include both volatile and non-volatile memory.

[0116] The non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EEPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), Synchronous DRAM (SDRAM), Double Data Rate SDRAM (DDRSDRAM), Enhanced Synchronous DRAM (ESDRAM), Synchlink DRAM (SLDRAM), and Direct Rambus RAM (DRRAM).

[0117] The beneficial effects achieved by this application are as follows: (1) This application standardizes and processes four types of heterogeneous data, namely internal parallel samples, certified reference materials, spiked samples and laboratory blank samples, to achieve the summary and statistics of massive historical quality control data.

[0118] (2) This application uses dynamic weighting to calculate the comprehensive score and quality control stability index, and divides the monitoring station into four capability levels: excellent, qualified, early warning and unqualified, so as to achieve an objective and standardized comprehensive evaluation.

[0119] (3) The dynamic weight of the quality control index in this application depends not only on its own historical error, but also on the coupling interference of other indicators. By analyzing the coupling interference of other indicators, the dynamic weight of the quality control index is calculated, which improves the accuracy and reliability of the dynamic weight calculation of the quality control index, thereby improving the accuracy of the comprehensive score calculation of the monitoring station.

[0120] (4) This application automatically increases the frequency of parallel sample testing and calculates the recommended testing frequency according to the optimal sampling theory, thereby automatically optimizing the laboratory testing procedure and improving the optimization effect. The adjusted testing parameters need to go through a verification cycle and the quality risk persistence trend index is recalculated. If the quality risk persistence trend index does not decrease, it is determined that non-random error is dominant, and the warning level is automatically upgraded and pushed to the management level.

[0121] (5) This application uses the standard normal cumulative distribution function to convert the degree of deviation of the predicted value from the qualified range into the benchmark probability of non-compliance. It uses the predicted standard deviation of the neighboring stations of the current monitoring station as an influencing factor and introduces spatial neighborhood correlation into the calculation of non-compliance probability, which improves the accuracy of the calculation of data non-compliance probability. When a batch of reagents is contaminated and causes multiple stations in the province to have systematic deviations at the same time, the traditional method only issues warnings to a few stations with large fluctuations. This method uses the average standard deviation of the neighboring stations to trigger warnings for all affected stations in advance. It realizes the early identification and full warning of systemic quality risks (such as reagent batch contamination and regional method deviations), and improves the timeliness and accuracy of warnings.

[0122] In the description of this application, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the stated features. In the description of this application, "multiple" means two or more, unless otherwise explicitly specified.

[0123] In the description of this application, the word "for example" is used to mean "used as an example, illustration, or description." Any embodiment described as "for example" in this application is not necessarily to be construed as being more preferred or advantageous than other embodiments. The following description is provided to enable any person skilled in the art to make and use the invention. Details are set forth in the following description for purposes of explanation. It should be understood that those skilled in the art will recognize that the invention can be made without using these specific details. In other instances, well-known structures and processes will not be described in detail to avoid obscuring the description of the invention with unnecessary detail. Therefore, the invention is not intended to be limited to the embodiments shown, but is consistent with the broadest scope of the principles and features disclosed in this application.

[0124] The above description is merely an embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principle of the present invention should be included within the scope of the claims of the present invention.

Claims

1. A method for quality control statistics and trend prediction of surface water monitoring data, characterized in that, The method includes: Collect multi-category quality control sample data reported by monitoring stations in various cities within the national sampling and testing separation platform, and preprocess the multi-category quality control sample data to construct a standardized quality control dataset; Perform multi-dimensional statistical analysis of standardized quality control datasets to obtain quality control statistical indicators corresponding to multiple categories of quality control sample data; Based on historical quality control indicator data, the weights of various quality control statistical indicators are dynamically calculated. Based on the weights of various quality control statistical indicators and the quality control statistical indicators, the comprehensive score and laboratory quality control stability index of each city's monitoring station are calculated. Based on the comprehensive score and laboratory quality control stability index of each city's monitoring station, the monitoring stations are classified into capability levels. Based on the comprehensive score of the monitoring station, the quality risk persistence trend index of the monitoring station is calculated. If the quality risk persistence trend index is greater than the preset trend threshold, the laboratory testing procedure is optimized; otherwise, no optimization is required.

2. The method for quality control statistics and trend prediction of surface water monitoring data according to claim 1, characterized in that, The method also includes: Historical continuous periodic quality control statistical data from monitoring stations were retrieved, and a time series trend model was constructed to analyze the fluctuation patterns of quality control sample data. Based on the time series trend model, the probability of data non-compliance in the sample analysis of the next monitoring cycle is calculated, and early warning is given to monitoring stations where the probability of non-compliance exceeds the preset threshold. Automatically generate visual quality control statistical reports.

3. The method for quality control statistics and trend prediction of surface water monitoring data according to claim 1, characterized in that, Preprocessing of multi-category quality control sample data includes: Use regular expressions to remove erroneous data from multi-category quality control sample data; Anomaly detection principles were used to identify and remove outliers from multiple categories of quality control sample data. Based on the site codes and testing timestamps of monitoring stations in various cities, duplicate data of multiple categories of quality control samples are removed; Heterogeneous data reported by different monitoring stations are uniformly converted into a standardized data structure containing fields such as station identifier, monitoring time, detection index, quality control type, measured value, and standard value, which serves as a standardized quality control dataset.

4. The method for quality control statistics and trend prediction of surface water monitoring data according to claim 1, characterized in that, Multi-dimensional statistical analysis of standardized quality control datasets was performed to obtain quality control statistical indicators corresponding to multiple categories of quality control sample data, including: For internal parallel samples, calculate the relative deviation of the measurement results for the internal parallel samples; For certified reference materials, calculate the relative error of the measurement results of the certified reference materials; For the spiked sample, calculate the spike recovery rate; For laboratory blank samples, obtain the detection value of the laboratory blank samples.

5. The method for quality control statistics and trend prediction of surface water monitoring data according to claim 1, characterized in that, Based on historical quality control indicator data, the weights of various quality control statistical indicators are dynamically calculated. Then, based on these weights and the overall quality control statistical indicators, the comprehensive scores and laboratory quality control stability indices of each city's monitoring stations are calculated. Finally, the monitoring stations are classified into capability levels according to their comprehensive scores and laboratory quality control stability indices, including: Based on historical quality control indicator data, the weights of various quality control statistical indicators are dynamically calculated; Based on the weights of various quality control statistical indicators and the quality control statistical indicators themselves, calculate the comprehensive score of each city's monitoring station; Calculate the laboratory quality control stability index based on various quality control statistical indicators; The monitoring stations are classified into capability levels based on their comprehensive scores and laboratory quality control stability index.

6. The method for quality control statistics and trend prediction of surface water monitoring data according to claim 2, characterized in that, Historical continuous-cycle quality control statistical data from monitoring stations were retrieved, and a time-series trend model was constructed to analyze the fluctuation patterns of quality control sample data, including: Retrieve historical quality control data from the monitoring station for consecutive periods; Constructing time series trend models; Predict future cycle quality control index values ​​and fluctuation ranges by constructing a time series trend model; By combining historical fluctuation ranges and industry standards, an early warning threshold is set. The predicted quality control indicator values ​​are compared with the early warning threshold, and the fluctuation amplitude is compared with historical normal levels to determine whether an early warning is triggered.

7. The method for quality control statistics and trend prediction of surface water monitoring data according to claim 6, characterized in that, Methods for constructing time series trend models include: Historical quality control statistical indicators were constructed into a time series in chronological order and divided into training and testing sets; The ADF unit root test is used to determine the stationarity of the series. For non-stationary series, successive differencing is performed to determine the difference order. For series with seasonality, additional seasonal differencing is performed and a seasonal period is set. The optimal order of the model is determined by combining the autocorrelation function, partial autocorrelation function, and AIC and BIC information criteria. The model parameters are fitted using maximum likelihood estimation or least squares method to complete model training.

8. The method for quality control statistics and trend prediction of surface water monitoring data according to claim 2, characterized in that, Visualized quality control statistical reports include: Summary table of monthly / quarterly quality control indicators for each monitoring station; Heatmap of pass rates for each testing item; The ability rating list is arranged in descending order of overall score; The trend change curve shows the fluctuation trajectory of quality control indicators for each site over multiple periods.

9. A surface water monitoring data quality control, statistical analysis, and trend prediction system, characterized in that, The system includes: The standard dataset module is used to collect multi-category quality control sample data reported by monitoring stations in various cities within the national sampling and testing separation platform, and to preprocess the multi-category quality control sample data to construct a standardized quality control dataset. The quality control statistical indicator acquisition module is used to perform multi-dimensional quality control data statistical analysis on standardized quality control datasets and obtain quality control statistical indicators corresponding to multiple categories of quality control sample data. The data processor is used to dynamically calculate the weights of various quality control statistical indicators based on historical quality control indicator data, and calculate the comprehensive score and laboratory quality control stability index of monitoring stations in various cities based on the weights of various quality control statistical indicators and the quality control statistical indicators. Based on the comprehensive score and laboratory quality control stability index of monitoring stations in various cities, the processor is used to classify the monitoring stations into capability levels. The data processor is also used to calculate the quality risk persistence trend index of the monitoring station based on the comprehensive score of the monitoring station. If the quality risk persistence trend index is greater than the preset trend threshold, the laboratory testing program is optimized; otherwise, no optimization is required.

10. The surface water monitoring data quality control statistics and trend prediction system according to claim 9, characterized in that, The system also includes: The module is used to retrieve historical continuous periodic quality control statistical index data from monitoring stations, construct time series trend models, and analyze the fluctuation patterns of quality control sample data. The early warning module is used to calculate the probability of non-compliance of sample analysis data in the next monitoring period based on the time series trend model, and to provide early warning prompts for monitoring stations where the probability of non-compliance exceeds a preset threshold. The generation module is used to automatically generate visual quality control statistical reports.

Images