A method and system for estimating correlation of contaminant data for skewed distributions with non-detects

Abstract

The application discloses a method and system for estimating correlation of pollutants data with skew distribution and undetected values, which comprises the following steps: S101, determining distribution characteristics of pollutants; S102, using t test to evaluate difference of shape parameters of the pollutants; S103, judging whether the difference reaches a preset significant level, if yes, using a semi-parametric two-dimensional gamma distribution model to estimate, and if not, using a parametric two-dimensional gamma distribution model.

Description

Technical Field

[0001] This invention relates to the fields of environmental science and statistical analysis technology, and in particular to a method and system for estimating the correlation of pollutant data with skewed distribution and undetected values. Background Technology

[0002] In the field of environmental science, researchers often face significant analytical challenges when processing pollutant data, especially data containing undetectable values. Undetectable values ​​refer to pollutant concentrations below the instrument's detection limit, a situation very common when monitoring pollutants in soil, water, and air quality. If these undetectable data points are not handled properly, they can seriously affect the effectiveness of environmental risk assessments and pollution control strategies.

[0003] Currently, the common processing methods used in environmental data analysis mainly include replacement and deletion methods. Replacement methods typically involve replacing undetected values ​​with a preset threshold, such as half the detection limit or the detection limit itself. While this method is simple to operate, it often leads to the loss of important information in datasets with a high proportion of undetected values, resulting in biased statistical analysis and inaccurate results. Furthermore, deletion methods, by simply discarding all undetected values, while simplifying the statistical processing, also cause significant data loss, especially when the proportion of undetected values ​​is high. This method significantly reduces the representativeness of the dataset and the reliability of the analysis results.

[0004] To overcome the limitations of these methods and achieve better results in parameter estimation, researchers have developed various statistical models, including parametric, semi-parametric, and non-parametric models. These methods attempt to preserve potentially useful information in undetected values ​​through more sophisticated statistical techniques, leading to more accurate data interpretation and result inference. Parametric models estimate population parameters by assuming the population data follows a known distribution, using sample information, such as maximum likelihood estimation, least squares estimation, Bayesian estimation, and the EM algorithm. Non-parametric models do not presuppose a basic distribution of the data but rely on information from the sample data itself for estimation; common methods include kernel functions and nearest neighbor functions. Semi-parametric models combine the advantages of parametric and non-parametric estimation, providing a flexible solution suitable for environmental data analysis with unclear or complex data distributions.

[0005] In agricultural environmental research, pollutant data and related indicators in soil, water, and air often exhibit skewed characteristics such as gamma distribution. The prevalence of this skewed distribution means that traditional parameter estimation methods relying on the assumption of normal distribution often fail to yield accurate results. However, current technologies lack effective models for handling complex data structures between two related pollutants, especially when the data is non-negative and skewed. For environmental data with nonlinear dependencies, such as pollutant concentration distributions in soil or water, existing technologies have not disclosed methods for more accurately modeling pollutant concentration distributions, particularly when the data contains a large number of undetected values. No technical solutions have been found that can deeply reveal the complex dependencies between pollutants. Maximum likelihood estimation (MLE) is particularly suitable for skewed environmental data, effectively handling data containing undetected values ​​given a known probability distribution.

[0006] Therefore, this invention is proposed. Summary of the Invention

[0007] The purpose of this invention is to overcome the above-mentioned technical deficiencies and provide a method and system for estimating the correlation of pollutant data with skewed distribution and undetected values, so as to solve the technical problem of lacking accurate estimation of the correlation of pollutant data with skewed distribution and undetected values ​​in related technologies.

[0008] To achieve the above-mentioned technical objectives, the present invention adopts the following technical solution:

[0009] According to one aspect of the present invention, a method for estimating the correlation of pollutant data with skewed distribution and containing undetected values ​​is provided, comprising:

[0010] Step S101: Determine the distribution characteristics of pollutants;

[0011] Step S102: Use a t-test to assess the differences in pollutant shape parameters;

[0012] Step S103: Determine whether the difference has reached a preset significance level. If yes, then use a semi-parametric two-dimensional gamma distribution model for estimation; otherwise, use a parametric two-dimensional gamma distribution model.

[0013] Furthermore, determining the distribution characteristics of pollutants specifically includes:

[0014] Use descriptive statistical tools and visualization methods to assess whether the data conforms to a known probability distribution.

[0015] Furthermore, the use of the t-test to assess the differences in pollutant shape parameters specifically includes:

[0016] Calculate the p-value for the t-test;

[0017] If the p-value is less than the preset significance threshold, then in step S103, it is determined that the difference has reached the preset significance level, confirming that the difference between the shape parameters of the two pollutants is significant.

[0018] If the p-value is greater than or equal to the preset significance threshold, then in step S103 it is determined that the difference has not reached the preset significance level, confirming that the difference between the shape parameters of the two pollutants is not significant.

[0019] According to another aspect of the present invention, a correlation estimation system for skewed pollutant data containing undetected values ​​is also provided, comprising:

[0020] A pollutant distribution characteristics determination unit, used to determine the distribution characteristics of pollutants;

[0021] A unit for calculating the difference in pollutant shape parameters, used to assess the difference in pollutant shape parameters using t-tests;

[0022] The model selection unit is used to determine whether the difference has reached a preset significance level. If so, a semi-parametric two-dimensional gamma distribution model is used for estimation; otherwise, a parametric two-dimensional gamma distribution model is used.

[0023] Furthermore, determining the distribution characteristics of pollutants specifically includes:

[0024] Use descriptive statistical tools and visualization methods to assess whether the data conforms to a known probability distribution.

[0025] Furthermore, the use of the t-test to assess the differences in pollutant shape parameters specifically includes:

[0026] Calculate the p-value for the t-test;

[0027] If the p-value is less than the preset significance threshold, then in step S103, it is determined that the difference has reached the preset significance level, confirming that the difference between the shape parameters of the two pollutants is significant.

[0028] If the p-value is greater than or equal to the preset significance threshold, then in step S103 it is determined that the difference has not reached the preset significance level, confirming that the difference between the shape parameters of the two pollutants is not significant.

[0029] According to another aspect of the present invention, an electronic device is also provided, comprising: a processor and a memory;

[0030] The memory stores a computer-readable program that can be executed by the processor;

[0031] When the processor executes the computer-readable program, it performs the steps of the method described above.

[0032] According to another aspect of the present invention, a computer-readable storage medium is also provided, the computer-readable storage medium storing one or more programs, the one or more programs being executable by one or more processors to implement the steps in the method described above.

[0033] This invention provides a method and system for estimating the correlation of pollutant data with skewed distributions and undetected values, based on the development and application of semi-parametric and parametric two-dimensional gamma models using maximum likelihood estimation. The two-dimensional gamma model is chosen because it can effectively handle the complex data structure between two related pollutants, especially when the data is non-negative and skewed. This model is particularly suitable for describing common nonlinear dependencies in environmental data, such as pollutant concentration distributions in soil or water. By fitting the model using two-dimensional gamma maximum likelihood estimation, pollutant concentration distributions can be modeled more accurately, especially when the data contains a large number of undetected values. These models can deeply reveal the complex dependencies between pollutants. Therefore, this invention not only expands the application scope of existing statistical models but also provides a more scientific analytical tool for environmental monitoring and risk assessment, possessing significant theoretical and practical application value. Attached Figure Description

[0034] Figure 1 This is a flowchart illustrating a method for estimating the correlation of pollutant data with skewed distribution and undetected values, as provided in an embodiment of the present invention.

[0035] Figure 2 This diagram illustrates the impact of sample size on the accuracy of the likelihood estimate of the correlation coefficient.

[0036] Figure 3 This diagram illustrates the impact of the non-detection rate on the accuracy of the likelihood estimate of the correlation coefficient.

[0037] Figure 4 This diagram illustrates the impact of the overall correlation coefficient on the accuracy of the likelihood estimate.

[0038] Figure 5 A diagram comparing the accuracy of correlation coefficient estimates from different methods;

[0039] Figure 6 A schematic diagram illustrating the impact of sample size on the accuracy of the likelihood estimate of the correlation coefficient (semi-parametric type);

[0040] Figure 7 A schematic diagram illustrating the impact of the non-detection rate on the accuracy of the likelihood estimate of the correlation coefficient (semi-parametric type);

[0041] Figure 8 A schematic diagram illustrating the impact of the overall correlation coefficient on the accuracy of the likelihood estimate (semi-parametric type);

[0042] Figure 9 A diagram illustrating the comparison of the accuracy of correlation coefficient estimates from different methods (semi-parametric type);

[0043] Figure 10 This is a histogram of gamma distribution for pollutants containing undetected values.

[0044] Figure 11 A schematic diagram illustrating the concentration correlation between two variables of air pollutants;

[0045] Figure 12 A schematic diagram illustrating the rank correlation between two variables of air pollutants;

[0046] Figure 13 This is a schematic diagram of the structure of an estimation system for the correlation of pollutant data with skewed distribution and undetected values, provided in an embodiment of the present invention.

[0047] Figure 14 This is a schematic diagram of an electronic device used to implement a method for estimating the correlation of pollutant data with skewed distribution and undetected values, as provided in an embodiment of the present invention. Detailed Implementation

[0048] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.

[0049] Example 1

[0050] According to embodiments of the present invention, a method for estimating the correlation of pollutant data with skewed distribution and containing undetected values ​​is provided, combining... Figure 1 The method includes:

[0051] Step S101: Determine the distribution characteristics of pollutants.

[0052] In step S101, preliminary data analysis is performed to verify the data distribution type of each pollutant. This involves using descriptive statistical tools and visualization methods such as histograms or QQ plots to assess whether the data conforms to a known probability distribution.

[0053] Step S102: Use a t-test to assess the differences in pollutant shape parameters.

[0054] In step S102, a t-test is applied to determine whether there is a significant difference in the shape parameters of the two pollutants. The p-value is used to determine whether the difference in shape parameters reaches a preset significance level (e.g., p < 0.05). This step is crucial because the similarity or difference in shape parameters directly affects which statistical model to use to process the data. The t-test provides sufficient evidence to support the conclusion that there is a statistically significant difference between the two groups of data.

[0055] Step S103: Determine whether the difference has reached a preset significance level. If yes, then use a semi-parametric two-dimensional gamma distribution model for estimation; otherwise, use a parametric two-dimensional gamma distribution model.

[0056] In step S103, as described above, a preset significance threshold can be used to compare the p-value of the t-test to determine whether the difference between the two pollutants is significant. For example, the specific judgment process may include:

[0057] Calculate the p-value for the t-test;

[0058] If the p-value is less than the preset significance threshold, then in step S103, it is determined that the difference has reached the preset significance level, confirming that the difference between the shape parameters of the two pollutants is significant.

[0059] If the p-value is greater than or equal to the preset significance threshold, then in step S103 it is determined that the difference has not reached the preset significance level, confirming that the difference between the shape parameters of the two pollutants is not significant.

[0060] Further, in step S103, when the t-test results show significant differences between shape parameters, a semi-parametric two-dimensional gamma distribution model is used for estimation. The semi-parametric model does not rely on consistent shape parameters and can more flexibly adapt to the characteristics of the data. When the differences between shape parameters are not significant, a parametric two-dimensional gamma distribution model is selected. This model relies on a uniform setting of shape parameters and is suitable for situations where parameter differences are small. By effectively selecting and applying the most appropriate statistical model based on the specific characteristics of the pollutant data, the best estimation results can be achieved.

[0061] To verify the accuracy of the parametric two-dimensional gamma distribution estimation model and the semi-parametric two-dimensional gamma distribution estimation model in the above steps, the method of this embodiment of the invention further includes a process of generating simulation data to verify the accuracy of the model, specifically including:

[0062] Step S201, Generation of simulation data:

[0063] Using the R language, the present invention generates simulated pollutant data that conforms to a two-dimensional gamma distribution. In addition, a detection limit (LOD) is set, and data below this limit is marked as "<LOD", thereby constructing a simulated data set that includes non-detected values.

[0064] Step S202, setting of influencing factors;

[0065] Parametric gamma estimation:

[0066] Sample size (two-dimensional gamma distribution data): 50, 100, 200, 500, 800, 1000 respectively;

[0067] Population correlation coefficient (Pearson correlation): 0.2, 0.5, 0.8;

[0068] Non-detection rate (<LOD%): By setting the LOD values respectively, the non-detection rate is made to be 0% to 100%, with an interval of 10%, so that data combinations with different non-detection rates can be obtained;

[0069] Semiparametric gamma estimation:

[0070] Sample size (two-dimensional gamma distribution data): 10, 50, 100, 500, 1000, etc. respectively;

[0071] Population correlation coefficient (Spearman correlation): 0.2, 0.5, 0.8, -0.2, -0.4, -0.9;

[0072] Non-detection rate (<LOD%): By setting different detection limit (LOD) values respectively, the non-detection rate is made to be 0% to 100%, with an interval of 10%, so that data combinations with different non-detection rates can be obtained;

[0073] Step S203, accuracy of the parametric two-dimensional gamma distribution estimation model;

[0074] Using the generated simulated data, the accuracy of the parametric two-dimensional gamma distribution estimation model in estimating the correlation of pollutant data with non-detected values is studied under different factors. In the simulation, fixed shape parameters and scale parameters are selected to generate random samples (alpha1 = 4, alpha2 = 4, beta1 = 3, beta2 = 2). By adjusting different LOD values, various data combinations with the required non-detection rates can be obtained. The effects of sample size, non-detection rate, and population correlation coefficient on the accuracy of correlation estimation are analyzed (combined with Figure 2-5This invention demonstrates that as the volume of simulated data increases to over 500, the convergence of correlation coefficient estimation improves, the distribution of estimated values ​​tends to stabilize, the median is closer to the assumed true value of 0.5, and the number of outliers and extreme values ​​is significantly reduced. Furthermore, although the non-detection rate has some impact on the estimation results, the MLE method provides relatively accurate estimates at medium to high correlation levels when dealing with data with high non-detection rates, exhibiting excellent stability and reliability. Especially when the non-detection rate exceeds 80%, although the accuracy of methods such as DL / 2 replacement and deletion decreases, the estimates obtained by the MLE method still maintain small bias and high concentration. This finding is one of the important achievements of this research, highlighting the advantages of the MLE method in ensuring the stability and accuracy of the estimates.

[0075] Step S204, the accuracy of the semi-parametric two-dimensional gamma distribution estimation model;

[0076] Combination Figure 6-9 The influence of various factors on the estimation accuracy is relatively small, and the absolute value of the estimation bias (the difference between the estimated value and the true value) generally remains within 0.1. This finding not only confirms the high accuracy of the semi-parametric two-dimensional gamma distribution estimation model in processing correlated data, but also demonstrates that both semi-parametric and parametric models can effectively control estimation errors and ensure the accuracy of data analysis. Through detailed simulations and analyses, this invention not only reveals the performance of each model in different scenarios, but also verifies the reliability and efficiency of these models in practical applications.

[0077] Step S205. Description of Gamma Distribution Data;

[0078] Combination Figure 10 This invention utilizes data conforming to a gamma distribution to validate the proposed statistical models, particularly in handling complex data containing undetected values. This validation involves using semi-parametric and parametric two-dimensional gamma distribution models to assess their applicability and accuracy in practical applications. The performance of these models under different undetected rate conditions is systematically analyzed to determine their effectiveness in estimating pollutant concentrations. Figure 10 In the middle, the purple part represents the undetected data portion, and these histograms confirm the gamma distribution hypothesis followed by the pollutant data.

[0079] This invention presents a method for estimating the correlation of pollutant data containing undetected values ​​using the maximum likelihood method (MLE) based on simulated two-dimensional gamma data. Even when the undetection rate increases (even to 80%), the stability of the correlation coefficient obtained by MLE is not significantly affected. This indicates that although the undetection rate of the data can affect the accuracy of the estimation to some extent, the maximum likelihood estimation model based on the two-dimensional gamma distribution proposed in this invention can effectively mitigate this impact, ensuring the consistency and stability of the method, and is a reliable estimation strategy. The calculation process of this invention is not limited to the gamma distribution; in particular, the semi-parametric two-dimensional gamma distribution estimation model can be extended to other distributions.

[0080] The two estimation models proposed in this invention demonstrate significant advantages in practical applications, providing more accurate estimation results under the same conditions compared to traditional substitution and deletion methods. Furthermore, by estimating actual pollutant data, not only concentration correlations but also rank correlations can be obtained. Combining the analysis of these two correlations enables a more comprehensive understanding of the complex interactions between environmental data, contributing to the accurate assessment of pollutant behavior and thus enhancing the model's application value and practicality in environmental science. This method is also applicable to analyses in other fields (such as medicine and biology), especially when processing data below the detection limit.

[0081] <Application Example>

[0082] Given the chemical stability and bioaccumulation of compounds such as organochlorine pesticides (OCPs) and polychlorinated biphenyls (PCBs), their persistence in the environment and potential risks to ecosystems and human health make them important subjects of study. These compounds not only persist in the environment for extended periods but also amplify through the food chain, posing threats to wildlife and human health. Studying their environmental behavior, distribution patterns, and impacts on organisms can help develop more effective environmental monitoring and pollution control strategies.

[0083] This invention references a dataset provided by Wang Xiaoping, which records in detail the concentrations of persistent organic pollutants (POPs) in suspended particulate matter in Namtso Lake from 2012 to 2014. This data covers 15 sampling points (S1-S15) in Namtso Lake, reflecting the distribution and environmental concentrations of POPs in the region, including data on POP concentrations in the atmosphere, lake water, and fish bodies. This provides valuable empirical data for this invention (Wang Xiaoping, Dataset of Persistent Organic Pollutant Concentrations in the Atmosphere and Lake Water of Namtso Lake, National Tibetan Plateau Scientific Data Center). Furthermore, the eight atmospheric pollutants follow a gamma distribution; this invention uses these data (with varying degrees of undetected rates) to evaluate the effectiveness of the proposed method.

[0084] Parametric two-dimensional gamma distribution fitting model

[0085] Concentration correlation analysis allows for the quantification and comparison of the coexistence relationships of different chemical substances in environmental samples, revealing potential common sources or similar environmental behavioral patterns among pollutants. Concentration correlation is particularly important for understanding the interactions of pollutants in the atmosphere and their possible common emission sources. Figure 11 This study demonstrates the concentration correlations among pollutants such as β-HCH in atmospheric pollutants. These pollutants were selected as research subjects because their shape parameters did not differ significantly (confirmed by a t-test). In each chart, data points are distinguished by color-coded markers: light purple rectangles indicate the portion of the pollutant concentration below the detection limit on the X-axis, while light green rectangles indicate the portion of the pollutant concentration below the detection limit on the Y-axis, thus visually demonstrating the concentration correlations among pollutants in different scenarios.

[0086] Semi-parametric two-dimensional gamma distribution fitting model

[0087] Combination Figure 12 Rank correlation is particularly helpful in revealing potential relationships between data points below the detection limit because it depends not directly on actual concentration values, but on their relative positions within the dataset. Through this analysis, researchers can explore the correlation of relative order or rank among various pollutants, such as α-HCH, in different environmental samples. This approach provides powerful insights into which pollutants tend to co-occur in the environment or accumulate in organisms, especially when faced with undetectable values.

[0088] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods according to the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0089] Example 2

[0090] According to embodiments of the present invention, a correlation estimation system for pollutant data with skewed distribution and containing undetected values ​​is provided, combining... Figure 13 The system includes:

[0091] The pollutant distribution characteristics determination unit 21 is used to determine the distribution characteristics of pollutants;

[0092] The pollutant shape parameter difference calculation unit 22 is used to evaluate the difference in pollutant shape parameters using the t-test.

[0093] The model selection unit 23 is used to determine whether the difference has reached a preset significance level. If so, a semi-parametric two-dimensional gamma distribution model is used for estimation; otherwise, a parametric two-dimensional gamma distribution model is used.

[0094] Optionally, specific examples in this embodiment can refer to the examples described in the above embodiments, and will not be repeated here.

[0095] It should be noted that the examples and application scenarios implemented by the above modules and corresponding steps are the same, but are not limited to the content disclosed in the above embodiments. It should also be noted that the above modules, as part of the device, can run in a corresponding hardware environment, and can be implemented through software or hardware, wherein the hardware environment includes a network environment.

[0096] Figure 14 This is a structural block diagram of a terminal according to an embodiment of this application, such as... Figure 14 As shown, the terminal may include: one or more (only one is shown) processors 101, memory 103, and transmission devices 105, such as... Figure 14 As shown, the terminal may also include input / output devices 107.

[0097] The memory 103 can be used to store software programs and modules, such as the program instructions / modules corresponding to the methods and apparatus in this embodiment. The processor 101 executes various functional applications and data processing by running the software programs and modules stored in the memory 103, thereby implementing the above-described methods. The memory 103 may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 103 may further include memory remotely located relative to the processor 101, and these remote memories can be connected to the terminal via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0098] The aforementioned transmission device 105 is used to receive or send data via a network, and can also be used for data transfer between the processor and memory. Specific examples of the network described above may include wired networks and wireless networks. In one example, the transmission device 105 includes a Network Interface Controller (NIC), which can be connected to other network devices and routers via a network cable to communicate with the Internet or a local area network. In another example, the transmission device 105 is a radio frequency (RF) module, used for wireless communication with the Internet.

[0099] Specifically, memory 103 is used to store application programs.

[0100] The processor 101 can call the application stored in the memory 103 through the transmission device 105 to perform the following steps: Step S101: Determine the distribution characteristics of pollutants; Step S102: Use a t-test to evaluate the differences in pollutant shape parameters; Step S103: Determine whether the differences have reached a preset significance level. If so, use a semi-parametric two-dimensional gamma distribution model for estimation; if not, use a parametric two-dimensional gamma distribution model.

[0101] Optionally, specific examples in this embodiment can refer to the examples described in the above embodiments, and will not be repeated here.

[0102] Those skilled in the art will understand that the structure of the terminal described above is merely illustrative, and the terminal can be a smartphone (such as an Android phone, an iOS phone, etc.), a tablet computer, a PDA, a mobile internet device (MID), a PAD, or other terminal devices. Figure 11 This does not limit the structure of the aforementioned electronic device. For example, the terminal may also include components that are more... Figure 11 The more or fewer components shown (such as network interfaces, display devices, etc.), or having the same Figure 11 The different configurations shown.

[0103] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be implemented by a program instructing the hardware related to the terminal device. The program can be stored in a computer-readable storage medium, which may include: flash drive, read-only memory (ROM), random access memory (RAM), disk or optical disk, etc.

[0104] Embodiments of this application also provide a storage medium. Optionally, in this embodiment, the storage medium can be used to execute program code for the above-described method.

[0105] Optionally, in this embodiment, the storage medium may be located on at least one of the network devices in the network shown in the above embodiment.

[0106] Optionally, specific examples in this embodiment can refer to the examples described in the above embodiments, and will not be repeated here.

[0107] Optionally, in this embodiment, the storage medium may include, but is not limited to, various media capable of storing program code, such as USB flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, or optical disks.

[0108] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0109] If the integrated units in the above embodiments are implemented as software functional units and sold or used as independent products, they can be stored in the aforementioned computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause one or more computer devices (which may be personal computers, servers, or network devices, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application.

[0110] In the above embodiments of this application, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0111] The above description is only a preferred embodiment of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of this application, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. A method for estimating the correlation of pollutant data with skewed distribution and containing undetected values, characterized in that, include: Step S101: Determine the distribution characteristics of pollutants; Step S102: Use a t-test to assess the differences in pollutant shape parameters; Step S103: Determine whether the difference has reached a preset significance level. If yes, use a semi-parametric two-dimensional gamma distribution model for estimation; otherwise, use a parametric two-dimensional gamma distribution model. The determination of the distribution characteristics of pollutants specifically includes: Use descriptive statistical tools and visualization methods to assess whether the data conforms to a known probability distribution; The use of t-tests to assess differences in pollutant shape parameters specifically includes: Calculate the p-value for the t-test; If the p-value is less than the preset significance threshold, then in step S103, it is determined that the difference has reached the preset significance level, confirming that the difference between the shape parameters of the two pollutants is significant. If the p-value is greater than or equal to the preset significance threshold, then in step S103 it is determined that the difference has not reached the preset significance level, confirming that the difference between the shape parameters of the two pollutants is not significant.

2. A correlation estimation system for pollutant data with skewed distribution and containing undetected values, characterized in that, include: A pollutant distribution characteristics determination unit, used to determine the distribution characteristics of pollutants; A unit for calculating the difference in pollutant shape parameters, used to assess the difference in pollutant shape parameters using t-tests; The model selection unit is used to determine whether the difference has reached a preset significance level. If so, a semi-parametric two-dimensional gamma distribution model is used for estimation; otherwise, a parametric two-dimensional gamma distribution model is used. The determination of the distribution characteristics of pollutants specifically includes: Use descriptive statistical tools and visualization methods to assess whether the data conforms to a known probability distribution; The use of t-tests to assess differences in pollutant shape parameters specifically includes: Calculate the p-value for the t-test; If the p-value is less than the preset significance threshold, then in step S103, it is determined that the difference has reached the preset significance level, confirming that the difference between the shape parameters of the two pollutants is significant. If the p-value is greater than or equal to the preset significance threshold, then in step S103 it is determined that the difference has not reached the preset significance level, confirming that the difference between the shape parameters of the two pollutants is not significant.

3. An electronic device, characterized in that, include: Processor and memory; The memory stores a computer-readable program that can be executed by the processor; When the processor executes the computer-readable program, it implements the steps of the method as described in claim 1.

4. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores one or more programs that can be executed by one or more processors to perform the steps of the method as described in claim 1.

Images