Method, device, equipment and storage medium for automatically identifying abnormal information of pollutant discharge permit

Abstract

The present application provides a method, device, equipment and storage medium for automatically identifying abnormal information of pollutant discharge permits. It relates to the field of data monitoring technology. The method includes: using the Pearson correlation coefficient matrix of the dynamic time window, fitting the statistical distribution, determining the key parameter pairs suitable for calculating the ratio coefficient, calculating the ratio coefficient matrix of the dynamic time window, constructing a distribution function of the upper and lower edge values ​​of the ratio coefficient based on quantile regression, filtering multiple groups of quantile function parameters, and selecting the optimal parameter group as the quantile function parameter when using it. When identifying data abnormalities, the ratio coefficient is calculated with the reported data, and whether it is within the upper and lower edge values ​​of the ratio coefficient calculated by the quantile distribution function is judged to determine whether the reported data is abnormal, so as to provide the basis for the review of the basic data of the relevant industry for the relevant personnel.

Description

Technical Field

[0001] The present application relates to the field of data monitoring technology, and in particular to a method, device, equipment and storage medium for automatically identifying abnormalities in pollutant discharge permit information. Background Art

[0002] The basic information of the enterprise and relevant environmental management requirements are required to be reported in the pollutant discharge permit, so the accuracy of the content reported in the pollutant discharge permit is crucial. In the actual production process, the data reported by the enterprise often have filling errors in various scenarios such as omissions, incorrect filling, unconverted numerical units, or subjective data falsification. However, it is difficult for non-professionals in a specific industry to review the relevant content of the industry, and it is easy to make mistakes in review. Summary of the invention

[0003] The present application provides a method, device, equipment and storage medium for automatically identifying abnormal information of pollutant discharge permits, so as to solve the problem that the existing review of the content of pollutant discharge permits is prone to misreview and error.

[0004] In a first aspect, the present application provides a method for automatically identifying abnormalities in pollutant discharge permit information, comprising:

[0005] Obtaining a key parameter pair; wherein the key parameter pair is selected from the information reported in the enterprise's pollutant discharge permit;

[0006] Based on the key parameter pairs, a Pearson correlation coefficient matrix of the key parameter pairs is calculated using a dynamic time window;

[0007] Performing distribution statistics with the correlation coefficient matrix to calculate the mathematical expectation and confidence interval of the correlation coefficient;

[0008] Determine the key parameter pairs that pass the distribution test according to the correlation coefficient matrix after distribution statistics, calculate the ratio coefficient matrix of the key parameter pairs using a dynamic time window based on the key parameter pairs that pass the distribution test, and calculate the mathematical expectation of the ratio coefficient matrix;

[0009] Construct the upper and lower edge value distribution function of the ratio coefficient based on quantile regression;

[0010] Using the key parameter pairs and their correlation coefficient matrices, ratio coefficient matrices, and mathematical expectations of ratio coefficient matrices, the parameters of multiple groups of ratio coefficient upper and lower edge value distribution functions are filtered, and the optimal quantile distribution function is determined by verification among the parameters of multiple groups of ratio coefficient upper and lower edge value distribution functions;

[0011] The upper and lower edge values ​​of the ratio coefficients of the key parameter pairs are calculated using the optimal quantile distribution function, and the abnormalities in the pollution discharge permit information are identified based on the upper and lower edge values.

[0012] In a possible design, the key parameter pair includes a first key parameter and a second key parameter. Based on the key parameter pair, a calculation formula for calculating the Pearson correlation coefficient matrix of the key parameter pair using a dynamic time window is:

[0013] ;

[0014] In the formula, represents the elements of the Pearson correlation coefficient matrix, represents the first key parameter in the jth key parameter pair under the i-th dynamic time window, represents the arithmetic mean of the first key parameter in the i-th dynamic time window, Indicates the length of the parameter pair, Indicates that the jth parameter pair has been calculated. represents the second key parameter in the jth parameter pair under the i-th dynamic time window, represents the arithmetic mean of the second key parameter in the i-th dynamic time window, represents the variance of the first key parameter in the key parameter pair under the i-th dynamic time window, represents the variance of the second key parameter in the key parameter pair under the i-th dynamic time window, Indicates the number of sliding windows.

[0015] In a possible design, distribution statistics are performed using the correlation coefficient matrix to calculate the mathematical expectation and confidence interval of the correlation coefficient, including:

[0016] Fitting the correlation coefficient matrix based on Gaussian distribution;

[0017] Based on the KS test, the distribution test is calculated for the data obtained from the fitted distribution. If the distribution test passes, it means that the correlation coefficient matrix conforms to the Gaussian distribution. The KS test calculation formula is as follows:

[0018] ;

[0019] In the formula, represents the statistic of the ks test, represents the distance supremum of the calculated sequence, Represents the sequence value to be detected, represents the sequence value of the distribution theory;

[0020] For the correlation coefficient matrix that conforms to the Gaussian distribution, the mathematical expectation and confidence interval of the correlation coefficient matrix are calculated; the mathematical expectation is , the confidence interval is , is a general expression for the arithmetic mean, Represents the calculated variance of the data.

[0021] In a possible design, the key parameter pairs that pass the distribution test are determined according to the correlation coefficient matrix after distribution statistics, and the ratio coefficient matrix of the key parameter pairs that pass the distribution test is calculated using a dynamic time window based on the key parameter pairs, and the mathematical expectation of the ratio coefficient matrix is ​​calculated, including:

[0022] The key parameter pairs corresponding to the correlation coefficient matrix that conforms to the Gaussian distribution are taken as the key parameter pairs that pass the distribution test;

[0023] Based on the key parameter pairs that pass the distribution test, calculate the ratio coefficient matrix , The matrix elements of constitute, ;in, A vector representing the first key parameter in a key parameter pair, a vector representing the second key parameter of the key parameter pair;

[0024] For the ratio coefficient matrix Perform fitting distribution statistics, using one of Gaussian distribution, Poisson distribution, and exponential distribution for fitting;

[0025] Based on any distribution test method among KS test, AD test and T test, a distribution test is performed on the ratio coefficient matrix after fitting distribution statistics;

[0026] Compute the mathematical expectation of the ratio coefficient matrix that passes the distribution test.

[0027] In one possible design, the upper and lower marginal value distribution functions of the ratio coefficient based on quantile regression are constructed, including:

[0028] Constructing a neural network model of quantile target as a distribution function of upper and lower edge values ​​of ratio coefficient based on quantile regression, the neural network model takes key parameter pairs and their correlation coefficient matrix and ratio coefficient matrix as input, and the upper and lower edge values ​​of ratio coefficient of key parameter pairs as output;

[0029] The neural network model uses the quantile loss function as the loss function, the target value of the quantile loss function is the mathematical expectation of the ratio coefficient matrix, and the quantile loss function includes single and grouped quantile loss functions; wherein the calculation formulas of the single and grouped quantile loss functions are as follows:

[0030] ;

[0031] In the formula, represents a single quantile loss function, represents the target value, represents the model prediction value, represents the quantile level, represents a single target value, represents the grouped quantile loss function, Represents the total amount of target value, Indicates calculation up to target value.

[0032] In a possible design, the parameters of multiple groups of upper and lower edge value distribution functions of ratio coefficients are filtered using the key parameter pairs and their correlation coefficient matrices, the ratio coefficient matrix, and the mathematical expectation of the ratio coefficient matrix, and the optimal quantile distribution function is determined by verification among the parameters of multiple groups of upper and lower edge value distribution functions of ratio coefficients, including:

[0033] The data set is established based on the key parameter pairs and their correlation coefficient matrix, ratio coefficient matrix, and mathematical expectation of the ratio coefficient matrix;

[0034] Based on the data set, multiple training sets, test sets, and validation sets are divided; wherein the validation set is randomly extracted from the data set, and the target value of the quantile loss function is the mathematical expectation of the ratio coefficient matrix; the training set and the test set are multiple sets of data obtained by sliding windows from the remaining data after the validation set is extracted from the data set;

[0035] Optimizing the parameters of the upper and lower edge value distribution function of the ratio coefficient with the training set, and when the evaluation index of the upper and lower edge value distribution function of the ratio coefficient on the test set meets the standard, determining the parameters of the upper and lower edge value distribution function of the ratio coefficient, and continuing to filter the next set of parameters;

[0036] For the parameters of the filtered multiple groups of upper and lower edge value distribution functions of the ratio coefficient, verify the evaluation index of the multiple groups of upper and lower edge value distribution functions of the ratio coefficient on the verification set, select a group of parameters of the upper and lower edge value distribution functions of the ratio coefficient with the best evaluation index as the target parameter, configure the upper and lower edge value distribution function of the ratio coefficient with the target parameter, and obtain the optimal quantile distribution function;

[0037] The evaluation index is coverage probability PICP, the confidence is PINC, and the calculation formula is as follows:

[0038] ;

[0039] In the formula, Represents the predicted lower edge value, represents the predicted upper edge value, Indicates the total amount of calculated data. Indicates the calculated value of the data. A numerical expression indicating whether the actual value falls within the prediction interval. Represents a single actual value, It represents the confidence level, usually 0.05 or 0.1.

[0040] In a second aspect, the present application provides a device for automatically identifying abnormalities in pollutant discharge permit information, comprising:

[0041] A data acquisition module is configured to acquire a key parameter pair; wherein the key parameter pair is selected from the enterprise's pollutant discharge permit reporting information;

[0042] A first calculation module is configured to calculate a Pearson correlation coefficient matrix of the key parameter pairs based on the key parameter pairs using a dynamic time window;

[0043] A distribution verification module is configured to perform distribution statistics based on the correlation coefficient matrix and calculate the mathematical expectation and confidence interval of the correlation coefficient;

[0044] A second calculation module is configured to determine the key parameter pairs that pass the distribution test according to the correlation coefficient matrix after distribution statistics, calculate the ratio coefficient matrix of the key parameter pairs using a dynamic time window based on the key parameter pairs that pass the distribution test, and calculate the mathematical expectation of the ratio coefficient matrix;

[0045] A function building module, configured to build a distribution function of upper and lower edge values ​​of a ratio coefficient based on quantile regression;

[0046] The function training module is configured to filter the parameters of multiple groups of upper and lower edge value distribution functions of the ratio coefficients by using the key parameter pairs and their correlation coefficient matrices, the ratio coefficient matrix, and the mathematical expectation of the ratio coefficient matrix, and to perform verification among the parameters of the multiple groups of upper and lower edge value distribution functions of the ratio coefficients to determine the optimal quantile distribution function;

[0047] The anomaly identification module is configured to calculate the upper and lower edge values ​​of the ratio coefficient of the key parameter pair using the optimal quantile distribution function, and identify the anomaly of the pollution discharge permit information based on the upper and lower edge values.

[0048] In a third aspect, the present application provides an electronic device comprising: at least one processor and a memory; the memory stores computer-executable instructions; the at least one processor executes the computer-executable instructions stored in the memory, so that the at least one processor executes the method for automatically identifying abnormalities in pollutant discharge permit information as described in the first aspect and various possible designs of the first aspect.

[0049] In a fourth aspect, the present application provides a computer-readable storage medium, in which computer execution instructions are stored. When a processor executes the computer execution instructions, the method for automatically identifying anomalies in pollutant discharge permit information as described in the first aspect and various possible designs of the first aspect is implemented.

[0050] In a fifth aspect, the present application provides a computer program product, including a computer program, which, when executed by a processor, implements the method for automatically identifying anomalies in pollutant discharge permit information as described in the first aspect and various possible designs of the first aspect.

[0051] The method, device, equipment and storage medium for automatically identifying abnormalities in pollutant discharge permit information provided by the present application utilize the Pearson correlation coefficient matrix of the dynamic time window to fit the statistical distribution, determine the key parameter pairs suitable for calculating the ratio coefficient, calculate the ratio coefficient matrix of the dynamic time window, construct a distribution function of the upper and lower edge values ​​of the ratio coefficient based on quantile regression, filter multiple groups of quantile function parameters, and select the optimal parameter group as the quantile function parameters when using it. When identifying data errors, the anomaly is identified by whether the reported data ratio coefficient is within the upper and lower edge values ​​of the ratio coefficient calculated by the quantile distribution function, thereby providing relevant personnel with a basis for basic data review in the steel industry and data review in other industries. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0053] Figure 1 A flowchart of a method for automatically identifying abnormalities in pollutant discharge permit information provided in an embodiment of the present application;

[0054] Figure 2 A filtering flow chart of a set of ratio coefficient upper and lower edge value distribution functions provided in an embodiment of the present application;

[0055] Figure 3 A frequency distribution histogram of a distribution test of the Pearson correlation coefficient of a set of key parameter pairs provided in an embodiment of the present application;

[0056] Figure 4 A quartile box plot of ratio coefficients of multiple sets of key parameter pairs provided in the embodiments of the present application;

[0057] Figure 5 A schematic diagram of the structure of a device for automatically identifying abnormalities in pollutant discharge permit information provided in an embodiment of the present application;

[0058] Figure 6 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application.

[0059] The above drawings have shown clear embodiments of the present application, which will be described in more detail later. These drawings and text descriptions are not intended to limit the scope of the present application in any way, but to illustrate the concept of the present application to those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION

[0060] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application. Instead, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.

[0061] In the technical solution of this application, the collection, storage, use, processing, transmission, provision and disclosure of information such as financial data or user data involved shall comply with the provisions of relevant laws and regulations and shall not violate public order and good morals.

[0062] It should be noted that in the embodiments of the present application, certain software, components, models and other existing solutions in the industry may be mentioned, and they should be regarded as exemplary. Their purpose is only to illustrate the feasibility of implementing the technical solution of the present application, but it does not mean that the applicant has or will necessarily use the solution.

[0063] An embodiment of the present application also provides a method for automatically identifying abnormalities in pollutant discharge permit information. Figure 1 Flow chart of the method for automatically identifying abnormalities in pollutant discharge permit information provided in the embodiment of the present application. Figure 1 As shown, the method for automatically identifying abnormalities in pollutant discharge permit information includes:

[0064] S1. Obtain key parameter pairs; wherein the key parameter pairs are selected from the information reported in the enterprise's pollutant discharge permit.

[0065] In this embodiment, taking the steel industry as an example, the key parameter pair includes a pair of key parameters, wherein the key parameters are selected from the enterprise pollutant discharge permit reporting information of the implementing industry, in the form of at least two corresponding sequence data, such as Among them, Represents a key parameter respectively. and Represents key parameters X and Y The corresponding feature vector.

[0066] In one embodiment, the steel industry is selected, and the key parameters are selected from sintered ore usage, iron ore concentrate usage, coke usage, molten iron output, crude steel output, hot-rolled material output, and power generation; the key parameter pairs are combinations of two key parameters, such as crude steel output-sintered ore usage parameter pair.

[0067] S2. Based on the key parameter pairs, a Pearson correlation coefficient matrix of the key parameter pairs is calculated using a dynamic time window.

[0068] In some embodiments, step S2 specifically includes:

[0069] S21, paired data for the key parameter pairs , dynamic time window Group data, such as ,in represents the two key parameters in the i-th key parameter pair, which are described as the first key parameter and the second key parameter here. and Indicates key parameters and The corresponding feature vector.

[0070] S22. For the multiple sets of data obtained ,calculate Matrix of group Pearson correlation coefficients , The matrix elements of The Pearson correlation coefficient that passes the significance test is included in the dynamic time window correlation coefficient matrix. The Pearson correlation coefficient is calculated as follows:

[0071] ;

[0072] In the formula, represents the elements of the Pearson correlation coefficient matrix, represents the first key parameter in the jth key parameter pair under the i-th dynamic time window, represents the arithmetic mean of the first key parameter in the i-th dynamic time window, Indicates the length of the parameter pair, Indicates that the jth parameter pair has been calculated. represents the second key parameter in the jth parameter pair under the i-th dynamic time window, represents the arithmetic mean of the second key parameter in the i-th dynamic time window, represents the variance of the first key parameter in the key parameter pair under the i-th dynamic time window, represents the variance of the second key parameter in the key parameter pair under the i-th dynamic time window, Indicates the number of sliding windows.

[0073] S3. Perform distribution statistics using the correlation coefficient matrix to calculate the mathematical expectation and confidence interval of the correlation coefficient.

[0074] In some embodiments, step S3 specifically includes:

[0075] S31. Perform statistics on the correlation coefficient matrix of the dynamic time window and fit the commonly used distribution.

[0076] In one embodiment, a Gaussian distribution is fitted , during the fitting process, Calculated from the mean of the data, Calculated from the standard deviation of the data;

[0077] S32. Calculate distribution test on the data of fitted distribution.

[0078] In one embodiment, a KS test is used. If the calculated hypothesis test passes, it means that the correlation coefficient array conforms to the distribution. The KS test calculation formula is as follows:

[0079] ;

[0080] In the formula, represents the statistic of the ks test, Represents the distance supremum of the calculated sequence, which is equivalent to , Represents the sequence value to be detected, A sequence value representing a distribution theory.

[0081] S33. For the relevant parameter matrix that conforms to the distribution, calculate its mathematical expectation and confidence interval.

[0082] For example, the mathematical expectation of the relevant parameter array that conforms to the Gaussian distribution is ,exist Confidence interval of .

[0083] Take the steel industry as an example. Figure 3 As shown in the figure, it is a frequency distribution histogram of the distribution test of the Pearson correlation coefficient of a set of key parameter pairs. Specifically, although the actual operation steps depend on the data distribution test and its significance index, in the test and fitting of Gaussian distribution, the frequency histogram shown in the figure can be used to intuitively display the data distribution. Figure 3The frequency histogram shown is the result when the number of groups is 50. The bar graph shown is a frequency histogram. The ordinate on the left corresponds to the frequency of the bar graph. The curve and other coordinates shown belong to the probability density graph (using kde, Kernel Density Estimation plot). The curve is actually a kernel density curve (through the distribution test operation in the step, it can be known that its data distribution has passed the Gaussian distribution hypothesis test). The ordinate on the right represents the probability density value, which can be derived from the area integral under the kernel density curve as 1 by manual calculation; the abscissa represents the data value in the measured data set ( Figure 3 The actual value of the Pearson correlation coefficient of this group of key parameters participating in the distribution display, so the value range is between 0 and 1).

[0084] S4. Determine the key parameter pairs that pass the distribution test according to the correlation coefficient matrix after distribution statistics, calculate the ratio coefficient matrix of the key parameter pairs using a dynamic time window based on the key parameter pairs that pass the distribution test, and calculate the mathematical expectation of the ratio coefficient matrix.

[0085] In some embodiments, step S4 specifically includes:

[0086] S41. For the key parameter pairs that pass the distribution test, calculate the dynamic time window ratio coefficient matrix of the selected key parameter pairs. , The matrix elements of constitute, .

[0087] S42, performing statistics on the dynamic time window ratio coefficient matrix to fit a common distribution.

[0088] In one embodiment, a Gaussian distribution is fitted , during the fitting process, Calculated from the mean of the data, Calculated from the standard deviation of the data.

[0089] S43. Calculate distribution test on the data of fitted distribution.

[0090] In one embodiment, the KS test is fitted, and it can also be replaced by other distribution test methods such as AD test and t test. If the calculated hypothesis test passes, it means that the ratio coefficient array conforms to the distribution. The KS test calculation formula is as follows:

[0091] ;

[0092] In the formula, represents the statistic of the ks test, Represents the distance supremum of the calculated sequence, which is equivalent to , Represents the sequence value to be detected, A sequence value representing a distribution theory.

[0093] S44. For the ratio coefficient matrix that conforms to the distribution, calculate its mathematical expectation and confidence interval.

[0094] For example, the ratio coefficient matrix that conforms to the Gaussian distribution has a mathematical expectation of ,exist Confidence interval of .

[0095] Take the steel industry as an example. Figure 4 Shown is a quartile box plot of the ratio coefficients of multiple groups of key parameter pairs.

[0096] S5. Construct the upper and lower edge value distribution function of the ratio coefficient based on quantile regression.

[0097] In some embodiments, step S5 specifically includes:

[0098] S51, using a neural network model with a quantile target as a distribution function of the upper and lower edge values ​​of the ratio coefficient based on quantile regression, wherein the neural network model uses the corresponding correlation coefficient, ratio coefficient, and key parameters as inputs, and the upper and lower edge values ​​of the ratio coefficient of the key parameters as outputs.

[0099] The neural network model can be any neural network structure that supports multi-layer networks. In one embodiment, a Self-Attention structure is used to construct the network layer of the regression model.

[0100] S52. The quantile loss function is used as the loss function of the model, and the mathematical expectation of the ratio coefficient matrix is ​​used as the target value of the quantile loss function. The calculation formulas for single and group quantile loss functions are as follows:

[0101] ;

[0102] In the formula, represents a single quantile loss function, represents the target value, represents the model prediction value, represents the quantile level, represents a single target value, represents the grouped quantile loss function, Represents the total amount of target value, Indicates calculation up to target value.

[0103] Quantile level The value is between 0 and 1. In one embodiment, the value is 0.5.

[0104] The parameters of the neural network model are adjusted using any neural network optimizer. In one embodiment, the Adam optimizer is used as a feedback algorithm for parameter adjustment during neural network training.

[0105] S6. Filter the parameters of multiple groups of upper and lower edge value distribution functions of ratio coefficients using the key parameter pairs and their correlation coefficient matrices, ratio coefficient matrices, and mathematical expectations of the ratio coefficient matrices, and verify the parameters of multiple groups of upper and lower edge value distribution functions of ratio coefficients to determine the optimal quantile distribution function.

[0106] In some embodiments, Figure 2 As shown, it is a filtering flow chart of a set of ratio coefficient upper and lower edge value distribution functions. Step S6 specifically includes:

[0107] S61. Establish a data set based on key parameter pairs and their correlation coefficient matrix, ratio coefficient matrix, and mathematical expectation of the ratio coefficient matrix.

[0108] S62. Based on the data set, multiple training sets, test sets, and validation sets are divided; wherein the validation set is randomly extracted from the data set, and the target value of the quantile loss function is the mathematical expectation of the overall ratio coefficient matrix; the training set and the test set are multiple sets of data obtained by sliding windows from the remaining data after the validation set is extracted from the data set.

[0109] In one embodiment, the training set and the test set are the data remaining after the validation set is extracted from the data set, and the sliding window is used to obtain Group data, each group of data is divided proportionally, for example, 70% is extracted as the training set and 30% is extracted as the test set. The target value of the quantile loss function of the training set of each group is the mathematical expectation of the ratio coefficient of the training set of the group, and the target value of the quantile loss function of the test set of each group is the mathematical expectation of the ratio coefficient of the test set of the group.

[0110] S63, optimizing the parameters of the upper and lower edge value distribution function of the ratio coefficient with the training set, and when the evaluation index of the upper and lower edge value distribution function of the ratio coefficient on the test set reaches the standard, determining the parameters of the upper and lower edge value distribution function of the ratio coefficient, and continuing to filter the next set of parameters. Wherein reaching the standard means that the evaluation index reaches the set value, if the evaluation index does not reach the set value, it is not up to standard.

[0111] S64. For the filtered multiple groups of parameters of the upper and lower edge value distribution functions of the ratio coefficients, verify the evaluation indicators of the multiple groups of parameters of the upper and lower edge value distribution functions of the ratio coefficients on the verification set, select a group of parameters of the upper and lower edge value distribution functions of the ratio coefficients with the best evaluation indicators as the target parameters, configure the upper and lower edge value distribution functions of the ratio coefficients with the target parameters, and obtain the optimal quantile distribution function.

[0112] The evaluation index is coverage probability PICP, the confidence is PINC, and the calculation formula is as follows:

[0113] ;

[0114] In the formula, Represents the predicted lower edge value, represents the predicted upper edge value, Indicates the total amount of calculated data. Indicates the calculated value of the data. A numerical expression indicating whether the actual value falls within the prediction interval. Represents a single actual value, It represents the confidence level, usually 0.05 or 0.1.

[0115] S7. Calculate the upper and lower edge values ​​of the ratio coefficients of the key parameter pairs using the optimal quantile distribution function, and identify abnormalities in the pollutant discharge permit information based on the upper and lower edge values.

[0116] In this embodiment, in actual implementation, the specific process of using upper and lower edge values ​​to identify abnormal pollutant discharge permit information is as follows: based on the selected key parameter pairs, the data of the pollutant discharge permit to be identified is obtained, and its ratio coefficient and Pearson correlation coefficient are calculated. The key parameter pairs to be identified and their ratio coefficients and correlation coefficients are used as inputs to the quantile regression model, and the upper and lower edge values ​​of the ratio coefficients are calculated by the model; if the ratio coefficient of the identified key parameter pair is not within the upper and lower edge values, it is considered that there is an error in the reporting of the pollutant discharge permit data.

[0117] Figure 5 The present invention also provides a structural diagram of a device for automatically identifying abnormalities in pollutant discharge permit information. Figure 5 As shown, the device for automatically identifying abnormal pollutant discharge permit information includes:

[0118] The data acquisition module 501 is configured to acquire a key parameter pair; wherein the key parameter pair is selected from the enterprise pollutant discharge permit reporting information;

[0119] A first calculation module 502 is configured to calculate a Pearson correlation coefficient matrix of the key parameter pairs based on the key parameter pairs using a dynamic time window;

[0120] The distribution verification module 503 is configured to perform distribution statistics based on the correlation coefficient matrix and calculate the mathematical expectation and confidence interval of the correlation coefficient;

[0121] The second calculation module 504 is configured to determine the key parameter pairs that pass the distribution test according to the correlation coefficient matrix after distribution statistics, calculate the ratio coefficient matrix of the key parameter pairs using the dynamic time window based on the key parameter pairs that pass the distribution test, and calculate the mathematical expectation of the ratio coefficient matrix;

[0122] A function construction module 505 is configured to construct a distribution function of upper and lower edge values ​​of a ratio coefficient based on quantile regression;

[0123] The function training module 506 is configured to filter the parameters of multiple groups of upper and lower edge value distribution functions of the ratio coefficients using the key parameter pairs and their correlation coefficient matrices, the ratio coefficient matrix, and the mathematical expectation of the ratio coefficient matrix, and verify the parameters of the multiple groups of upper and lower edge value distribution functions of the ratio coefficients to determine the optimal quantile distribution function;

[0124] The abnormality identification module 507 is configured to calculate the upper and lower edge values ​​of the ratio coefficient of the key parameter pair using the optimal quantile distribution function, and identify the abnormality of the pollutant discharge permit information based on the upper and lower edge values.

[0125] In some embodiments, the first calculation module is further configured to calculate the Pearson correlation coefficient matrix of the key parameter pair based on the key parameter pair using a dynamic time window. The calculation formula is:

[0126] ;

[0127] In the formula, represents the elements of the Pearson correlation coefficient matrix, represents the first key parameter in the jth key parameter pair under the i-th dynamic time window, represents the arithmetic mean of the first key parameter in the i-th dynamic time window, Indicates the length of the parameter pair, Indicates that the jth parameter pair has been calculated. represents the second key parameter in the jth parameter pair under the i-th dynamic time window, represents the arithmetic mean of the second key parameter in the i-th dynamic time window, represents the variance of the first key parameter in the key parameter pair under the i-th dynamic time window, represents the variance of the second key parameter in the key parameter pair under the i-th dynamic time window, Indicates the number of sliding windows.

[0128] In some embodiments, the distribution verification module is further configured to:

[0129] Fitting the correlation coefficient matrix based on Gaussian distribution;

[0130] Based on the KS test, the distribution test is calculated for the data obtained from the fitted distribution. If the distribution test passes, it means that the correlation coefficient matrix conforms to the Gaussian distribution. The KS test calculation formula is as follows:

[0131] ;

[0132] In the formula, represents the statistic of the ks test, Represents the distance supremum of the calculated sequence, which is equivalent to , Represents the sequence value to be detected, represents the sequence value of the distribution theory;

[0133] For the correlation coefficient matrix that conforms to the Gaussian distribution, the mathematical expectation and confidence interval of the correlation coefficient matrix are calculated; the mathematical expectation is , the confidence interval is , It is a general expression of the arithmetic mean. Represents the calculated variance of the data.

[0134] In some embodiments, the second computing module is further configured to:

[0135] The key parameter pairs corresponding to the correlation coefficient matrix that conforms to the Gaussian distribution are taken as the key parameter pairs that pass the distribution test;

[0136] Based on the key parameter pairs that pass the distribution test, calculate the ratio coefficient matrix , The matrix elements of constitute, ;in, A vector representing the first key parameter in a key parameter pair, a vector representing the second key parameter of the pairwise parameters;

[0137] For the ratio coefficient matrix Perform fitting distribution statistics, using one of Gaussian distribution, Poisson distribution, and exponential distribution for fitting;

[0138] Based on any distribution test method among KS test, AD test and T test, a distribution test is performed on the ratio coefficient matrix after fitting distribution statistics;

[0139] Compute the mathematical expectation of the ratio coefficient matrix that passes the distribution test.

[0140] In some embodiments, the function building block is further configured to:

[0141] Constructing a neural network model of quantile target as a distribution function of upper and lower edge values ​​of ratio coefficient based on quantile regression, the neural network model takes key parameter pairs and their correlation coefficient matrix and ratio coefficient matrix as input, and the upper and lower edge values ​​of ratio coefficient of key parameter pairs as output;

[0142] The neural network model uses the quantile loss function as the loss function, the target value of the quantile loss function is the mathematical expectation of the ratio coefficient matrix, and the quantile loss function includes single and grouped quantile loss functions; wherein the calculation formulas of the single and grouped quantile loss functions are as follows:

[0143] ;

[0144] In the formula, represents a single quantile loss function, represents the target value, represents the model prediction value, represents the quantile level, represents a single target value, represents the grouped quantile loss function, Represents the total amount of target value, Indicates calculation up to target value.

[0145] In some embodiments, the function training module is further configured to:

[0146] The data set is established based on the key parameter pairs and their correlation coefficient matrix, ratio coefficient matrix, and mathematical expectation of the ratio coefficient matrix;

[0147] Based on the data set, multiple training sets, test sets, and validation sets are divided; wherein the validation set is randomly extracted from the data set, and the target value of the quantile loss function is the mathematical expectation of the ratio coefficient matrix; the training set and the test set are multiple sets of data obtained by sliding windows from the remaining data after the validation set is extracted from the data set;

[0148] Optimizing the parameters of the upper and lower edge value distribution function of the ratio coefficient with the training set, and when the evaluation index of the upper and lower edge value distribution function of the ratio coefficient on the test set meets the standard, determining the parameters of the upper and lower edge value distribution function of the ratio coefficient, and continuing to filter the next set of parameters;

[0149] For the parameters of the filtered multiple groups of upper and lower edge value distribution functions of the ratio coefficient, verify the evaluation index of the multiple groups of upper and lower edge value distribution functions of the ratio coefficient on the verification set, select a group of parameters of the upper and lower edge value distribution functions of the ratio coefficient with the best evaluation index as the target parameter, configure the upper and lower edge value distribution function of the ratio coefficient with the target parameter, and obtain the optimal quantile distribution function;

[0150] The evaluation index is coverage probability PICP, the confidence is PINC, and the calculation formula is as follows:

[0151] ;

[0152] In the formula, Represents the predicted lower edge value, represents the predicted upper edge value, Indicates the total amount of calculated data. Indicates the calculated value of the data. A numerical expression indicating whether the actual value falls within the prediction interval. Represents a single actual value, It represents the confidence level, usually 0.05 or 0.1.

[0153] The device for automatically identifying abnormalities in pollutant discharge permit information provided in the embodiment of the present application can be used to implement the technical solution of the method for automatically identifying abnormalities in pollutant discharge permit information in the above-mentioned embodiment. Its implementation principle and technical effects are similar and will not be repeated here.

[0154] Figure 6 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. Figure 6 As shown, the electronic device may include: a processor 61 and a memory 62 , wherein the processor 61 and the memory 62 may communicate with each other; illustratively, the processor 61 and the memory 62 communicate with each other via a communication bus 63 .

[0155] The processor 61 executes the computer execution instructions stored in the memory 62, so that the processor 61 executes the scheme in the above embodiment. The processor 61 can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit ASIC, a field programmable gate array FPGA or other programmable logic devices, discrete gates or transistor logic devices, and discrete hardware components.

[0156] The communication bus 63 may be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus, etc. The system bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus. The transceiver is used to realize the communication between the database access device and other computers (such as clients, read-write libraries, and read-only libraries). The memory may include random access memory (RAM) and may also include non-volatile memory.

[0157] The electronic device provided in the embodiment of the present application may be the terminal device of the above embodiment.

[0158] An embodiment of the present application also provides a computer-readable storage medium, in which computer instructions are stored. When the computer instructions are executed on a computer, the computer executes the technical solution of the method for automatically identifying abnormalities in pollution discharge permit information of the above embodiment.

[0159] An embodiment of the present application also provides a computer program product, which includes a computer program stored in a computer-readable storage medium. At least one processor can read the computer program from the computer-readable storage medium. When at least one processor executes the computer program, it can implement the technical solution of the method for automatically identifying abnormalities in pollutant discharge permit information in the above embodiment.

[0160] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of modules is only a logical function division, and there may be other division methods in actual implementation, such as multiple modules can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or modules, which can be electrical, mechanical or other forms.

[0161] The modules described as separate components may or may not be physically separated, and the components shown as modules may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to implement the solution of this embodiment.

[0162] In addition, each functional module in each embodiment of the present application can be integrated into one processing unit, or each module can exist physically separately, or two or more modules can be integrated into one unit. The above-mentioned module-composed unit can be implemented in the form of hardware or in the form of hardware plus software functional units.

[0163] The above-mentioned integrated module implemented in the form of a software function module can be stored in a computer-readable storage medium. The above-mentioned software function module is stored in a storage medium, including a number of instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor to perform some steps of the methods of various embodiments of the present application.

[0164] It should be understood that the above processor can be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), etc. A general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in the invention can be directly embodied as being executed by a hardware processor, or can be executed by a combination of hardware and software modules in the processor.

[0165] The memory may include a high-speed RAM memory, and may also include a non-volatile storage NVM, such as at least one disk memory, and may also be a USB flash drive, a mobile hard disk, a read-only memory, a magnetic disk or an optical disk, etc.

[0166] The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, the bus in the drawings of this application is not limited to only one bus or one type of bus.

[0167] The above storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk. The storage medium can be any available medium that can be accessed by a general or special purpose computer.

[0168] An exemplary storage medium is coupled to a processor so that the processor can read information from the storage medium and write information to the storage medium. Of course, the storage medium can also be a component of the processor. The processor and the storage medium can be located in an application specific integrated circuit (ASIC). Of course, the processor and the storage medium can also exist as discrete components in an electronic control unit or a main control device.

[0169] Those skilled in the art can understand that all or part of the steps of implementing the above-mentioned method embodiments can be completed by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, the steps of the above-mentioned method embodiments are executed; and the aforementioned storage medium includes: ROM, RAM, disk or optical disk and other media that can store program codes.

[0170] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for automatically identifying abnormalities in pollutant discharge permit information, characterized in that: include: Obtaining a key parameter pair; wherein the key parameter pair is selected from the enterprise's pollutant discharge permit reporting information; the key parameter pair includes a first key parameter and a second key parameter; Based on the key parameter pairs, a Pearson correlation coefficient matrix of the key parameter pairs is calculated using a dynamic time window; Performing distribution statistics with the correlation coefficient matrix to calculate the mathematical expectation and confidence interval of the correlation coefficient; Determine the key parameter pairs that pass the distribution test according to the correlation coefficient matrix after distribution statistics, calculate the ratio coefficient matrix of the key parameter pairs using a dynamic time window based on the key parameter pairs that pass the distribution test, and calculate the mathematical expectation of the ratio coefficient matrix; Construct the upper and lower edge value distribution function of the ratio coefficient based on quantile regression; Using the key parameter pairs and their correlation coefficient matrices, ratio coefficient matrices, and mathematical expectations of ratio coefficient matrices, the parameters of multiple groups of ratio coefficient upper and lower edge value distribution functions are filtered, and the optimal quantile distribution function is determined by verification among the parameters of multiple groups of ratio coefficient upper and lower edge value distribution functions; Calculating upper and lower edge values ​​of the ratio coefficient of the key parameter pair using the optimal quantile distribution function, and identifying abnormalities in the pollutant discharge permit information based on the upper and lower edge values; Construct the upper and lower edge value distribution function of the ratio coefficient based on quantile regression, including: Constructing a neural network model of quantile target as a distribution function of upper and lower edge values ​​of ratio coefficient based on quantile regression, the neural network model takes key parameter pairs and their correlation coefficient matrix and ratio coefficient matrix as input, and the upper and lower edge values ​​of ratio coefficient of key parameter pairs as output; The neural network model uses the quantile loss function as the loss function, the target value of the quantile loss function is the mathematical expectation of the ratio coefficient matrix, and the quantile loss function includes single and grouped quantile loss functions; wherein the calculation formulas of the single and grouped quantile loss functions are as follows: ； In the formula, represents a single quantile loss function, represents the target value, represents the model prediction value, represents the quantile level, represents a single target value, represents the grouped quantile loss function, Indicates the total amount of target value, Indicates calculation up to target value.

2. The method according to claim 1, characterized in that Based on the key parameter pairs, the calculation formula for calculating the Pearson correlation coefficient matrix of the key parameter pairs using a dynamic time window is: ； In the formula, represents the elements of the Pearson correlation coefficient matrix, represents the first key parameter in the jth key parameter pair under the i-th dynamic time window, represents the arithmetic mean of the first key parameter in the i-th dynamic time window, Indicates the length of the parameter pair, Indicates that the jth parameter pair has been calculated. represents the second key parameter in the jth parameter pair under the i-th dynamic time window, represents the arithmetic mean of the second key parameter in the i-th dynamic time window, represents the variance of the first key parameter in the key parameter pair under the i-th dynamic time window, represents the variance of the second key parameter in the key parameter pair under the i-th dynamic time window, Indicates the number of sliding windows.

3. The method according to claim 1, characterized in that The correlation coefficient matrix is ​​used to perform distribution statistics and calculate the mathematical expectation and confidence interval of the correlation coefficient, including: Fitting the correlation coefficient matrix based on Gaussian distribution; Based on the KS test, the distribution test is calculated for the data obtained from the fitted distribution. If the distribution test passes, it means that the correlation coefficient matrix conforms to the Gaussian distribution. The KS test calculation formula is as follows: ； In the formula, represents the statistic of the ks test, represents the distance supremum of the calculated sequence, Represents the sequence value to be detected, A sequence value representing a distribution theory; For the correlation coefficient matrix that conforms to the Gaussian distribution, the mathematical expectation and confidence interval of the correlation coefficient matrix are calculated; the mathematical expectation is , the confidence interval is , is a general expression for the arithmetic mean, Represents the calculated variance of the data.

4. The method according to claim 3, characterized in that Determine the key parameter pairs that pass the distribution test according to the correlation coefficient matrix after distribution statistics, calculate the ratio coefficient matrix of the key parameter pairs using a dynamic time window based on the key parameter pairs that pass the distribution test, and calculate the mathematical expectation of the ratio coefficient matrix, including: The key parameter pairs corresponding to the correlation coefficient matrix that conforms to the Gaussian distribution are taken as the key parameter pairs that pass the distribution test; Based on the key parameter pairs that pass the distribution test, calculate the ratio coefficient matrix , The matrix elements of constitute, ;in, A vector representing the first key parameter in a key parameter pair, a vector representing the second key parameter of the key parameter pair; For the ratio coefficient matrix Perform fitting distribution statistics, using one of Gaussian distribution, Poisson distribution, and exponential distribution for fitting; Based on any distribution test method among KS test, AD test and T test, a distribution test is performed on the ratio coefficient matrix after fitting distribution statistics; Compute the mathematical expectation of the ratio coefficient matrix that passes the distribution test.

5. The method according to claim 1, characterized in that The parameters of multiple groups of upper and lower edge value distribution functions of ratio coefficients are filtered using the key parameter pairs and their correlation coefficient matrices, the ratio coefficient matrix, and the mathematical expectation of the ratio coefficient matrix, and the optimal quantile distribution function is determined by verification in the parameters of multiple groups of upper and lower edge value distribution functions of ratio coefficients, including: The data set is established based on the key parameter pairs and their correlation coefficient matrix, ratio coefficient matrix, and mathematical expectation of the ratio coefficient matrix; Based on the data set, multiple training sets, test sets, and validation sets are divided; wherein the validation set is randomly extracted from the data set, and the target value of the quantile loss function is the mathematical expectation of the ratio coefficient matrix; the training set and the test set are multiple sets of data obtained by sliding windows from the remaining data after the validation set is extracted from the data set; Optimizing the parameters of the upper and lower edge value distribution function of the ratio coefficient with the training set, and when the evaluation index of the upper and lower edge value distribution function of the ratio coefficient on the test set meets the standard, determining the parameters of the upper and lower edge value distribution function of the ratio coefficient, and continuing to filter the next set of parameters; For the parameters of the filtered multiple groups of upper and lower edge value distribution functions of the ratio coefficient, verify the evaluation index of the multiple groups of upper and lower edge value distribution functions of the ratio coefficient on the verification set, select a group of parameters of the upper and lower edge value distribution functions of the ratio coefficient with the best evaluation index as the target parameter, configure the upper and lower edge value distribution function of the ratio coefficient with the target parameter, and obtain the optimal quantile distribution function; The evaluation index is coverage probability PICP, the confidence is PINC, and the calculation formula is as follows: ； In the formula, Represents the predicted lower edge value, represents the predicted upper edge value, Indicates the total amount of calculated data. Indicates the calculated value of the data. A numerical expression indicating whether the actual value falls within the prediction interval. Represents a single actual value, Indicates the confidence level.

6. A device for automatically identifying abnormalities in pollutant discharge permit information, characterized in that: include: A data acquisition module is configured to acquire a key parameter pair; wherein the key parameter pair is selected from the enterprise pollutant discharge permit reporting information; the key parameter pair includes a first key parameter and a second key parameter; A first calculation module is configured to calculate a Pearson correlation coefficient matrix of the key parameter pairs based on the key parameter pairs using a dynamic time window; A distribution verification module is configured to perform distribution statistics based on the correlation coefficient matrix and calculate the mathematical expectation and confidence interval of the correlation coefficient; A second calculation module is configured to determine the key parameter pairs that pass the distribution test according to the correlation coefficient matrix after distribution statistics, calculate the ratio coefficient matrix of the key parameter pairs using a dynamic time window based on the key parameter pairs that pass the distribution test, and calculate the mathematical expectation of the ratio coefficient matrix; A function building module, configured to build a distribution function of upper and lower edge values ​​of a ratio coefficient based on quantile regression; The function training module is configured to filter the parameters of multiple groups of upper and lower edge value distribution functions of the ratio coefficients by using the key parameter pairs and their correlation coefficient matrices, the ratio coefficient matrix, and the mathematical expectation of the ratio coefficient matrix, and to perform verification among the parameters of the multiple groups of upper and lower edge value distribution functions of the ratio coefficients to determine the optimal quantile distribution function; an abnormality identification module, configured to calculate upper and lower edge values ​​of the ratio coefficient of the key parameter pair using the optimal quantile distribution function, and identify abnormalities in the pollutant discharge permit information based on the upper and lower edge values; Construct the upper and lower edge value distribution function of the ratio coefficient based on quantile regression, including: Constructing a neural network model of quantile target as a distribution function of upper and lower edge values ​​of ratio coefficient based on quantile regression, the neural network model takes key parameter pairs and their correlation coefficient matrix and ratio coefficient matrix as input, and the upper and lower edge values ​​of ratio coefficient of key parameter pairs as output; The neural network model uses the quantile loss function as the loss function, the target value of the quantile loss function is the mathematical expectation of the ratio coefficient matrix, and the quantile loss function includes single and grouped quantile loss functions; wherein the calculation formulas of the single and grouped quantile loss functions are as follows: ； In the formula, represents a single quantile loss function, represents the target value, represents the model prediction value, represents the quantile level, represents a single target value, represents the grouped quantile loss function, Indicates the total amount of target value, Indicates calculation up to target value.

7. An electronic device, characterized in that: include: A processor, and a memory communicatively connected to the processor; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory to implement the method according to any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, which are used to implement the method according to any one of claims 1 to 5 when executed by a processor.

9. A computer program product, characterized in that The invention comprises a computer program, which implements the method according to any one of claims 1 to 5 when being executed by a processor.

Images