Environment pollution monitoring method and system based on big data

Abstract

The invention discloses an environmental pollution monitoring method and system based on big data, relates to the technical field of big data, solves the problems of inconsistent data formats and data islands in different fields by constructing an environmental pollution data standard set D, and forms a unified data basis; under the support of a multi-dimensional statistical method, the relationship between pollution features is extracted and analyzed, and an associated feature set FSR is constructed, so that the defect of insufficient association caused by traditional single-field data analysis is overcome. The associated feature set FSR is screened, and the screened associated feature set SFSR is obtained, so that the prediction precision of the pollutant concentration is greatly improved, the problem of excessive invalid feature interference in the existing method is solved, the dynamic influence of time periods and different situations on the pollutant concentration is fully considered, and the prediction accuracy of the pollutant concentration is improved. The prediction result can reflect the actual pollution condition more accurately, and the defect that a static prediction model is insufficient in response to complex scenes is overcome.

Description

Technical Field

[0001] The present invention relates to the field of big data technology, and in particular to an environmental pollution monitoring method and system based on big data. Background Art

[0002] Environmental pollution monitoring is one of the core areas of ecological protection and sustainable development, involving multiple disciplines such as environmental science, meteorology, traffic management and public health. In these fields, environmental pollution monitoring and governance is a topic of continuous concern, especially in the process of modern urbanization, pollution problems are becoming increasingly complex and have a wide impact.

[0003] Specifically, environmental pollution monitoring is not limited to the monitoring of traditional pollutants such as air, water quality and soil, but also involves the correlation between various complex environmental factors and pollution sources. For example, various data such as traffic flow, industrial emissions and meteorological factors directly or indirectly affect the concentration and diffusion pattern of pollutants.

[0004] However, current environmental pollution monitoring methods mostly rely on data from a single field, usually focusing on the monitoring of a certain type of pollution source or pollutant. For example, most air quality monitoring systems rely only on meteorological data and pollutant concentration data, ignoring other factors that may affect air quality, such as traffic flow and industrial emissions. Although some systems also integrate multiple data sources, their processing capabilities and analysis depth are limited, making it difficult to effectively reveal the complex relationship between pollution sources and pollutants. Traditional monitoring often treats environmental pollution as a single factor, while ignoring the interaction between data from various fields. As a result, the analyzed pollution patterns and governance strategies fail to accurately reflect the actual situation, and there are large limitations and deviations. In addition, the lack of efficient analysis and dynamic response to real-time data makes traditional monitoring systems lack timeliness and accuracy in pollution warning and control, making it difficult to respond to sudden and long-term pollution events. Summary of the invention

[0005] In view of the deficiencies in the prior art, the present invention provides an environmental pollution monitoring method and system based on big data, which solves the problems mentioned in the background technology.

[0006] To achieve the above objectives, the present invention is implemented through the following technical solutions: an environmental pollution monitoring method based on big data, comprising the following steps:

[0007] S1. Collect data from multiple fields through the Internet, including environmental monitoring stations, traffic monitoring equipment, meteorological sensors and industrial emission records, and form a standard set of environmental pollution data D after preprocessing;

[0008] S2. Extract data from the standard set of environmental pollution data D to obtain traffic flow pollution characteristics, meteorological factor pollution characteristics and pollution concentration characteristics, and analyze the relationship between the characteristics through multi-dimensional statistical methods to obtain the associated feature set FSR;

[0009] S3, constructing a multi-dimensional prediction model based on the obtained associated feature set FSR, and obtaining the predicted pollution concentration Cp by training the prediction model, and screening the associated feature set FSR according to the predicted pollution concentration Cp to obtain the screened screened associated feature set SFSR;

[0010] S4. According to the obtained screening correlation feature set SFSR, the predicted pollution concentration Cp is corrected to obtain the dynamic pollution concentration Cd(t, s) in different time periods and different situations;

[0011] S5. By fitting the dynamic pollution concentration Cd(t, s) in different time periods and different situations, the comprehensive index Ctotal of pollutant concentration is obtained, and it is compared with the preset pollutant concentration warning threshold Cthe to obtain the environmental pollution detection result.

[0012] Preferably, said S1 includes S11 and S12;

[0013] S11. Collect data on factors affecting environmental pollution from multiple fields through the Internet, including environmental monitoring stations, traffic monitoring equipment, meteorological sensors, and industrial emission records, and mark the data on factors affecting environmental pollution collected from environmental monitoring stations, traffic monitoring equipment, meteorological sensors, and industrial emission records to obtain an environmental monitoring station pollution data set Dm, a traffic monitoring data set Dt, a meteorological sensor data set Dw, and an industrial emission data set Di, and integrate them to form a multidimensional big data set Draw = {Dm, Dt, Dw, Di};

[0014] Among them, the data on factors affecting environmental pollution collected by the environmental monitoring station include PM2.5 fine particle concentration Px, PM10 inhalable particle concentration Pk, carbon monoxide concentration CO, nitrogen dioxide concentration NO2, ozone concentration O3 and sulfur dioxide concentration SO2;

[0015] Traffic monitoring equipment collects data on factors that affect environmental pollution, including vehicle flow Ft, average vehicle speed Va, and road congestion index Ic;

[0016] The data collected by meteorological sensors on factors affecting environmental pollution include air temperature Te, humidity Hh, wind speed Ww, wind direction Dd, and air pressure Pp;

[0017] Industrial emission records collect data on factors that affect environmental pollution, including nitrogen oxide emissions ENOx, sulfur dioxide emissions ESO2, volatile organic compound emissions EVOC and particulate matter emissions EPM.

[0018] Preferably, S12, performing data preprocessing on the multidimensional large data set Draw, including data cleaning preprocessing and data standardization preprocessing, to obtain a preprocessed environmental pollution data standard set D;

[0019] Among them, data cleaning preprocessing is to process missing values, outliers and duplicate data of the environmental monitoring station pollution data set Dm, traffic monitoring data set Dt, meteorological sensor data set Dw and industrial emission data set Di in the multidimensional big data set Draw, including using interpolation and mean methods to process missing values, and using statistical rules to remove outliers. The statistical rules include using triple standard deviation statistics;

[0020] Data standardization preprocessing eliminates the differences between different dimensions by standardizing the multidimensional big data set Draw after data cleaning preprocessing, including using Z-score standardization, Min-Max standardization and decimal calibration standardization for data standardization preprocessing.

[0021] Preferably, S2 includes S21 and S22;

[0022] S21, extracting data from the standard set of environmental pollution data D, obtaining the environmental monitoring station pollution data set Dm, traffic monitoring data set Dt, meteorological sensor data set Dw and industrial emission data set Di after data preprocessing, obtaining traffic flow pollution features TDm, traffic flow pollution features TDt, meteorological factor pollution features TDw and pollution concentration features TDi through feature marking, and reorganizing and integrating them to obtain feature set F;

[0023] The reorganized and integrated feature set F = {Px, Pk, CO, NO2, O3, SO2, Ft, Va, Ic, Te, Hh, Ww, Dd, Pp, ENOx, ESO2, EVOC, EPM};

[0024] S22. Use a multi-dimensional statistical method to analyze the relationship between features in the feature set F, evaluate the linear relationship between the features and the pollutant concentrations, obtain the correlation coefficient R (Fx, Fy) between the feature Fx and the feature Fy by performing correlation analysis on the x-th feature Fx and the y-th feature Fy in the feature set F, and substitute it into the feature set F to mark the association relationship between the feature Fx and the feature Fy. Simultaneously, eliminate the features in the feature set F that have no association relationship with the correlation coefficient R (Fx, Fy) to form the associated feature set FSR.

[0025] Preferably, the correlation coefficient R(Fx, Fy) is obtained by the following calculation formula:

[0026]

[0027] Where Cov(Fx, Fy) represents the covariance of feature Fx and feature Fy, S(Fx) and S(Fy) represent the standard deviation of feature Fx and feature Fy respectively;

[0028] The correlation coefficient R(Fx, Fy) reflects the linear relationship between the feature Fx and the feature Fy, and distinguishes the strong correlation, weak correlation and no correlation between the feature Fx and the feature Fy;

[0029] When the correlation coefficient R(Fx, Fy) = 1, a completely positive correlation result is obtained, indicating that the features Fx and Fy change completely synchronously, and it is determined that the features Fx and Fy are strongly correlated;

[0030] When the correlation coefficient R(Fx, Fy) = -1, a completely negative correlation result is obtained, indicating that the feature Fx and the feature Fy have completely opposite changes, and it is determined that the feature Fx and the feature Fy are weakly correlated;

[0031] When the correlation coefficient R(Fx, Fy) = 0, a non-correlated result is obtained, indicating that there is no linear relationship between the feature Fx and the feature Fy;

[0032] Among them, strong correlation includes strong positive correlation and strong negative correlation; weak correlation includes weak positive correlation and weak negative correlation;

[0033] When the correlation coefficient R(Fx, Fy) ≥ Qthe, it means that the change between the feature Fx and the feature Fy is a strong linear synchronous change, and the feature Fx and the feature Fy are strongly positively correlated;

[0034] When the correlation coefficient R(Fx, Fy)≥-Qthe, it means that the change between the feature Fx and the feature Fy is a strong linear reverse change, and the feature Fx and the feature Fy are strongly negatively correlated;

[0035] When the correlation coefficient R(Fx, Fy)<Rthe, it means that the change between the feature Fx and the feature Fy is a weak linear synchronous change, and the feature Fx and the feature Fy are weakly positively correlated;

[0036] When the correlation coefficient R(Fx, Fy) < -Rthe, it means that the change between the feature Fx and the feature Fy is a weak linear reverse change, and the feature Fx and the feature Fy are weakly negatively correlated;

[0037] Among them, Qthe and Rthe represent the strong correlation threshold and the weak correlation threshold respectively, and 0<weak correlation threshold Rthe<strong correlation threshold Qthe<1, and the specific value is set by the user.

[0038] Preferably, said S3 includes S31 and S32;

[0039] S31, constructing a multi-dimensional prediction model according to the obtained correlation feature set FSR, including using a decision tree regression model and a linear regression model to establish a prediction model, and obtaining a predicted pollution concentration Cp by training the prediction model;

[0040] The predicted pollution concentration Cp is obtained by the following calculation formula:

[0041]

[0042] In the formula, n represents the total number of features in the associated feature set FSR, FSR(i) represents the i-th feature in the associated feature set FSR, and c(i) represents the i-th feature weight value.

[0043] Preferably, S32, screening the associated feature set FSR according to the obtained predicted pollution concentration Cp and the correlation coefficient R (Fx, Fy), and obtaining the screened screened associated feature set SFSR;

[0044] The screening is performed by S321 and S322;

[0045] S321, by fitting the predicted pollution concentration Cp with the correlation coefficient R(Fx, Fy), obtain the pollution concentration correlation coefficient R(FSR(i), Cp) between the i-th feature in the associated feature set FSR and the predicted pollution concentration Cp;

[0046] The pollution concentration correlation coefficient R (FSR (i), Cp) is obtained by the following calculation formula:

[0047]

[0048] Where COV(FSR(i), Cp) represents the covariance between the i-th feature in the associated feature set FSR and the predicted pollution concentration Cp, S(FSR(i)) and S(Cp) represent the standard deviation between the i-th feature in the associated feature set FSR and the predicted pollution concentration Cp;

[0049] S322, compare the pollution concentration correlation coefficient R (FSR (i), Cp) with the preset rejection threshold TCthe to obtain a rejection screening mark result, perform secondary rejection processing on the associated feature set FSR according to the rejection screening mark result, and obtain a filtered associated feature set SFSR;

[0050] The elimination screening marker results are obtained by the following comparison method:

[0051] When |pollution concentration correlation coefficient R(FSR(i), Cp)|≥removal threshold TCthe, the removal screening mark result is obtained as the retention result, and the i-th feature in the associated feature set FSR is retained;

[0052] When -removal threshold TCthe<pollution concentration correlation coefficient R(FSR(i), Cp)<removal threshold TCthe, the removal screening mark result is obtained as a non-retained result, and the i-th feature in the associated feature set FSR is removed.

[0053] Preferably, the S4 includes S41;

[0054] S41, according to the obtained screening correlation feature set SFSR, the predicted pollution concentration Cp is corrected to obtain the dynamic pollution concentration Cd(t, s) in different time periods and different situations;

[0055] The dynamic pollution concentration Cd(t, s) is obtained by the following calculation formula:

[0056]

[0057] In the formula, Cd(t, s) represents the dynamic pollution concentration at time t and situation q, w(i(q)) represents the corrected weight value of the i-th feature in the associated feature set FSR under situation q, and △FSR(i(t, q)) represents the change of the i-th feature in the associated feature set FSR under time t and situation q.

[0058] Preferably, the S5 includes S51;

[0059] S51. By fitting the dynamic pollution concentration Cd(t, s) in different time periods and different situations, a comprehensive index Ctotal of pollutant concentration is obtained, and the index is compared with the preset pollutant concentration warning threshold Cthe to obtain the environmental pollution detection result;

[0060] The comprehensive index Ctotal is obtained by the following calculation formula:

[0061]

[0062] In the formula, m represents the comprehensive total number, which is obtained by the total number of time periods, T represents the total number of time periods, the total number of specific time periods = the comprehensive total number m, and Q represents the total number of situations;

[0063] The environmental pollution detection results are obtained by the following comparison method:

[0064] When the comprehensive index Ctotal is less than the pollutant concentration warning threshold Cthe, the environmental pollution detection result is qualified;

[0065] When the comprehensive index Ctotal ≥ the pollutant concentration warning threshold Cthe, the environmental pollution detection result is unqualified, indicating that the dynamic pollution concentration Cd(t, s) exceeds the standard in time period t and situation q.

[0066] The environmental pollution monitoring system based on big data includes a multi-dimensional data acquisition module, a multi-dimensional data extraction module, a data analysis module, a dynamic correction module and an evaluation and decision-making module;

[0067] The multi-dimensional data acquisition module collects data from multiple fields through the Internet, including environmental monitoring stations, traffic monitoring equipment, meteorological sensors and industrial emission records, and forms a standard set of environmental pollution data D after preprocessing;

[0068] The multidimensional data extraction module extracts data from the environmental pollution data standard set D, obtains traffic flow pollution characteristics, meteorological factor pollution characteristics and pollution concentration characteristics, and analyzes the relationship between the characteristics through a multidimensional statistical method to obtain a correlation feature set FSR;

[0069] The data analysis module constructs a multi-dimensional prediction model according to the obtained associated feature set FSR, obtains the predicted pollution concentration Cp by training the prediction model, and screens the associated feature set FSR according to the predicted pollution concentration Cp to obtain the screened screened associated feature set SFSR;

[0070] The dynamic correction module corrects the predicted pollution concentration Cp according to the obtained screening correlation feature set SFSR to obtain the dynamic pollution concentration Cd(t, s) in different time periods and different situations;

[0071] The evaluation and decision-making module obtains the comprehensive index Ctotal of pollutant concentration by fitting the dynamic pollution concentration Cd(t, s) in different time periods and different situations, and compares it with the preset pollutant concentration warning threshold Cthe to obtain the environmental pollution detection result.

[0072] The present invention provides an environmental pollution monitoring method and system based on big data, which has the following beneficial effects:

[0073] (1) By constructing a standard set of environmental pollution data D, the problems of inconsistent data formats and data islands in different fields are solved, forming a unified data foundation; with the support of multi-dimensional statistical methods, the relationship between pollution features is extracted and analyzed, and the associated feature set FSR is constructed, thus overcoming the defect of insufficient correlation caused by traditional single-field data analysis. The associated feature set FSR is screened and the screened associated feature set SFSR is obtained, which greatly improves the prediction accuracy of pollutant concentrations, solves the problem of excessive invalid feature interference in existing methods, and fully considers the dynamic impact of time periods and different situations on pollutant concentrations, so that the prediction results can more accurately reflect the actual pollution situation, which makes up for the defect that the static prediction model is insufficiently responsive to complex scenarios.

[0074] (2) A multi-dimensional prediction model based on decision tree regression and linear regression model was constructed through the associated feature set FSR, and the predicted pollution concentration Cp was accurately calculated, providing a reliable basis for the quantification of pollution conditions. On this basis, by calculating the pollution concentration correlation coefficient R (FSR (i), Cp) and comparing it with the elimination threshold TCthe, the features in the associated feature set FSR were screened for the second time, forming a more refined screening associated feature set SFSR. This staged screening mechanism can effectively eliminate features that have little or no effect on the prediction of pollutant concentration, significantly reduce the complexity of the model, and improve the prediction accuracy. At the same time, the screening strategy based on fitting ensures the scientific nature of feature selection and avoids the problem of reduced model efficiency due to feature redundancy or noise interference in traditional methods.

[0075] (3) By dynamically correcting the screening correlation feature set SFSR, combined with the total number of time periods T and the total number of situations Q, the dynamic pollution concentration Cd(t, s) is effectively calculated, and based on the fitting results of the dynamic pollution concentration, the comprehensive index of pollutant concentration Ctotal is obtained. Compared with traditional static monitoring methods, this method fully considers the dynamic impact of specific time periods and situations on pollutant concentrations, and accurately quantifies the contribution of dynamic changes by adjusting the correction weight value w(i(q)), significantly improving the accuracy and response flexibility of pollution concentration monitoring. In addition, by comparing with the pollutant concentration warning threshold Cthe, this method can quickly and scientifically determine whether the environmental pollution detection results are qualified, and accurately locate the time periods and situations where pollution exceeds the standard, providing a more targeted and operational basis for pollution control decisions. BRIEF DESCRIPTION OF THE DRAWINGS

[0076] Figure 1 A schematic diagram of the steps of the environmental pollution monitoring method based on big data of the present invention;

[0077] Figure 2 It is a schematic diagram of the block diagram of the environmental pollution monitoring system based on big data of the present invention. DETAILED DESCRIPTION

[0078] The technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0079] Example 1

[0080] The present invention provides an environmental pollution monitoring method based on big data, please refer to Figure 1 , including the following steps:

[0081] S1. Collect data from multiple fields through the Internet, including environmental monitoring stations, traffic monitoring equipment, meteorological sensors and industrial emission records, and form a standard set of environmental pollution data D after preprocessing;

[0082] S2. Extract data from the standard set of environmental pollution data D to obtain traffic flow pollution characteristics, meteorological factor pollution characteristics and pollution concentration characteristics, and analyze the relationship between the characteristics through multi-dimensional statistical methods to obtain the associated feature set FSR;

[0083] S3, constructing a multi-dimensional prediction model based on the obtained associated feature set FSR, and obtaining the predicted pollution concentration Cp by training the prediction model, and screening the associated feature set FSR according to the predicted pollution concentration Cp to obtain the screened screened associated feature set SFSR;

[0084] S4. According to the obtained screening correlation feature set SFSR, the predicted pollution concentration Cp is corrected to obtain the dynamic pollution concentration Cd(t, s) in different time periods and different situations;

[0085] S5. By fitting the dynamic pollution concentration Cd(t, s) in different time periods and different situations, the comprehensive index Ctotal of pollutant concentration is obtained, and it is compared with the preset pollutant concentration warning threshold Cthe to obtain the environmental pollution detection result.

[0086] In this embodiment, by comprehensively utilizing the data collected from multiple fields on the Internet, a full-process system from data standardization, feature extraction, prediction modeling to dynamic correction and comprehensive evaluation is established. By constructing a standard set D of environmental pollution data, the problems of inconsistent data formats and data islands in different fields are solved, forming a unified data foundation; with the support of multi-dimensional statistical methods, the relationship between pollution features is extracted and analyzed, and the associated feature set FSR is constructed, thereby overcoming the defect of insufficient correlation caused by traditional single-field data analysis. Further, through a multi-dimensional prediction model, the associated feature set FSR is screened and the screened associated feature set SFSR is obtained, which greatly improves the prediction accuracy of pollutant concentration, solves the problem of excessive invalid feature interference in the existing method, and fully considers the dynamic impact of time periods and different situations on pollutant concentration, so that the prediction results can more accurately reflect the actual pollution situation, which makes up for the defect of insufficient response of static prediction models to complex scenarios. Finally, by fitting the dynamic pollution concentration, the comprehensive index of pollutant concentration Ctotal is calculated, and compared with the pollutant concentration warning threshold Cthe, the environmental pollution situation is comprehensively evaluated, which can provide a scientific quantitative basis for environmental governance. Compared with traditional monitoring methods, this method has the advantages of more comprehensive data processing, more accurate prediction results, and higher monitoring timeliness, which helps to achieve precise and intelligent pollution warning, governance decision-making and public health protection.

[0087] Example 2

[0088] This embodiment is explained in Example 1, please refer to Figure 1 , specifically: the S1 includes S11 and S12;

[0089] S11. Collect data on factors affecting environmental pollution from multiple fields through the Internet, including environmental monitoring stations, traffic monitoring equipment, meteorological sensors, and industrial emission records, and mark the data on factors affecting environmental pollution collected from environmental monitoring stations, traffic monitoring equipment, meteorological sensors, and industrial emission records to obtain an environmental monitoring station pollution data set Dm, a traffic monitoring data set Dt, a meteorological sensor data set Dw, and an industrial emission data set Di, and integrate them to form a multidimensional big data set Draw = {Dm, Dt, Dw, Di};

[0090] Among them, the data on factors affecting environmental pollution collected by the environmental monitoring station include PM2.5 fine particle concentration Px, PM10 inhalable particle concentration Pk, carbon monoxide concentration CO, nitrogen dioxide concentration NO2, ozone concentration O3 and sulfur dioxide concentration SO2;

[0091] Traffic monitoring equipment collects data on factors that affect environmental pollution, including vehicle flow Ft, average vehicle speed Va, and road congestion index Ic;

[0092] The data collected by meteorological sensors on factors affecting environmental pollution include air temperature Te, humidity Hh, wind speed Ww, wind direction Dd, and air pressure Pp;

[0093] Industrial emission records collect data on factors that affect environmental pollution, including nitrogen oxide emissions ENOx, sulfur dioxide emissions ESO2, volatile organic compound emissions EVOC and particulate matter emissions EPM.

[0094] S12, performing data preprocessing on the multidimensional big data set Draw, including data cleaning preprocessing and data standardization preprocessing, to obtain a preprocessed environmental pollution data standard set D;

[0095] Among them, data cleaning preprocessing is to process missing values, outliers and duplicate data of the environmental monitoring station pollution data set Dm, traffic monitoring data set Dt, meteorological sensor data set Dw and industrial emission data set Di in the multidimensional big data set Draw, including using interpolation and mean methods to process missing values, and using statistical rules to remove outliers. The statistical rules include using triple standard deviation statistics;

[0096] Data standardization preprocessing eliminates the differences between different dimensions by standardizing the multidimensional big data set Draw after data cleaning preprocessing, including using Z-score standardization, Min-Max standardization and decimal calibration standardization for data standardization preprocessing.

[0097] The S2 includes S21 and S22;

[0098] S21, extracting data from the standard set of environmental pollution data D, obtaining the environmental monitoring station pollution data set Dm, traffic monitoring data set Dt, meteorological sensor data set Dw and industrial emission data set Di after data preprocessing, obtaining traffic flow pollution features TDm, traffic flow pollution features TDt, meteorological factor pollution features TDw and pollution concentration features TDi through feature marking, and reorganizing and integrating them to obtain feature set F;

[0099] The reorganized and integrated feature set F = {Px, Pk, CO, NO2, O3, SO2, Ft, Va, Ic, Te, Hh, Ww, Dd, Pp, ENOx, ESO2, EVOC, EPM};

[0100] S22. Use a multi-dimensional statistical method to analyze the relationship between features in the feature set F, evaluate the linear relationship between the features and the pollutant concentrations, obtain the correlation coefficient R (Fx, Fy) between the feature Fx and the feature Fy by performing correlation analysis on the x-th feature Fx and the y-th feature Fy in the feature set F, and substitute it into the feature set F to mark the association relationship between the feature Fx and the feature Fy. Simultaneously, eliminate the features in the feature set F that have no association relationship with the correlation coefficient R (Fx, Fy) to form the associated feature set FSR.

[0101] Among them, the correlation coefficient R(Fx, Fy) is obtained by the following calculation formula:

[0102]

[0103] Where Cov(Fx, Fy) represents the covariance of feature Fx and feature Fy, which is used to measure the degree of joint variation of feature Fx and feature Fy. S(Fx) and S(Fy) represent the standard deviation of feature Fx and feature Fy, respectively, which are used to measure the degree of discreteness of feature Fx and feature Fy.

[0104] The correlation coefficient R(Fx, Fy) reflects the linear relationship between the feature Fx and the feature Fy, and distinguishes the strong correlation, weak correlation and no correlation between the feature Fx and the feature Fy;

[0105] When the correlation coefficient R(Fx, Fy) = 1, a completely positive correlation result is obtained, indicating that the features Fx and Fy change completely synchronously, and it is determined that the features Fx and Fy are strongly correlated;

[0106] When the correlation coefficient R(Fx, Fy) = -1, a completely negative correlation result is obtained, indicating that the feature Fx and the feature Fy have completely opposite changes, and it is determined that the feature Fx and the feature Fy are weakly correlated;

[0107] When the correlation coefficient R(Fx, Fy) = 0, a non-correlated result is obtained, indicating that there is no linear relationship between the feature Fx and the feature Fy;

[0108] Among them, strong correlation includes strong positive correlation and strong negative correlation; weak correlation includes weak positive correlation and weak negative correlation;

[0109] When the correlation coefficient R(Fx, Fy) ≥ Qthe, it means that the change between the feature Fx and the feature Fy is a strong linear synchronous change, and the feature Fx and the feature Fy are strongly positively correlated;

[0110] When the correlation coefficient R(Fx, Fy)≥-Qthe, it means that the change between the feature Fx and the feature Fy is a strong linear reverse change, and the feature Fx and the feature Fy are strongly negatively correlated;

[0111] When the correlation coefficient R(Fx, Fy)<Rthe, it means that the change between the feature Fx and the feature Fy is a weak linear synchronous change, and the feature Fx and the feature Fy are weakly positively correlated;

[0112] When the correlation coefficient R(Fx, Fy) < -Rthe, it means that the change between the feature Fx and the feature Fy is a weak linear reverse change, and the feature Fx and the feature Fy are weakly negatively correlated;

[0113] Among them, Qthe and Rthe represent the strong correlation threshold and the weak correlation threshold respectively, and 0<weak correlation threshold Rthe<strong correlation threshold Qthe<1, and the specific value is set by the user.

[0114] In this embodiment, a large amount of data on environmental pollution factors in multiple fields is collected through the Internet, including pollution data sets of environmental monitoring stations, traffic monitoring data sets, meteorological sensor data sets and industrial emission data sets, and a unified environmental pollution data standard set D is formed through a multidimensional data preprocessing method, which effectively solves the problems of complex data sources and inconsistent dimensions; at the same time, through multi-dimensional feature extraction and statistical analysis, a feature set F is obtained, and through the calculation of the correlation coefficient R (Fx, Fy) and the division of the strong and weak correlation thresholds Qthe and Rthe, highly correlated features are screened out, and finally a correlation feature set FSR is formed, which overcomes the defect of reducing the model prediction accuracy due to redundant features and irrelevant features in traditional methods. The characteristic of this method is that, through the refined division of the correlation relationship of the features, it not only improves the scientific nature of data analysis, but also significantly reduces the interference of invalid features on the pollutant prediction model, so that the model can more accurately characterize the complex dynamic characteristics of environmental pollution, and provide a reliable data basis and theoretical support for the efficient monitoring and precise governance of environmental pollution.

[0115] Example 3

[0116] This embodiment is explained in Example 2. Please refer to Figure 1 , specifically: S3 includes S31 and S32;

[0117] S31, constructing a multi-dimensional prediction model according to the obtained correlation feature set FSR, including using a decision tree regression model and a linear regression model to establish a prediction model, and obtaining a predicted pollution concentration Cp by training the prediction model;

[0118] The predicted pollution concentration Cp is obtained by the following calculation formula:

[0119]

[0120] In the formula, n represents the total number of features in the associated feature set FSR, FSR(i) represents the i-th feature in the associated feature set FSR, and c(i) represents the i-th feature weight value.

[0121] S32, screening the associated feature set FSR according to the obtained predicted pollution concentration Cp and the correlation coefficient R (Fx, Fy), and obtaining the screened screened associated feature set SFSR;

[0122] The screening is performed by S321 and S322;

[0123] S321, by fitting the predicted pollution concentration Cp with the correlation coefficient R(Fx, Fy), obtain the pollution concentration correlation coefficient R(FSR(i), Cp) between the i-th feature in the associated feature set FSR and the predicted pollution concentration Cp;

[0124] The pollution concentration correlation coefficient R (FSR (i), Cp) is obtained by the following calculation formula:

[0125]

[0126] Where COV(FSR(i), Cp) represents the covariance between the i-th feature in the associated feature set FSR and the predicted pollution concentration Cp, S(FSR(i)) and S(Cp) represent the standard deviation between the i-th feature in the associated feature set FSR and the predicted pollution concentration Cp;

[0127] S322, compare the pollution concentration correlation coefficient R (FSR (i), Cp) with the preset rejection threshold TCthe to obtain a rejection screening mark result, perform secondary rejection processing on the associated feature set FSR according to the rejection screening mark result, and obtain a filtered associated feature set SFSR;

[0128] The elimination screening marker results are obtained by the following comparison method:

[0129] When |pollution concentration correlation coefficient R(FSR(i), Cp)|≥removal threshold TCthe, the removal screening mark result is obtained as the retention result, and the i-th feature in the associated feature set FSR is retained;

[0130] When -removal threshold TCthe<pollution concentration correlation coefficient R(FSR(i), Cp)<removal threshold TCthe, the removal screening mark result is obtained as a non-retained result, and the i-th feature in the associated feature set FSR is removed.

[0131] In this embodiment, a multi-dimensional prediction model based on decision tree regression and linear regression model is constructed through the associated feature set FSR, and the predicted pollution concentration Cp is accurately calculated, providing a reliable basis for the quantification of pollution conditions. On this basis, by calculating the pollution concentration correlation coefficient R (FSR (i), Cp) and comparing it with the elimination threshold TCthe, the features in the associated feature set FSR are screened twice to form a more refined screening associated feature set SFSR. This staged screening mechanism can effectively eliminate features that have little or no effect on the prediction of pollutant concentration, significantly reduce the complexity of the model, and improve the prediction accuracy. At the same time, the screening strategy based on fitting ensures the scientific nature of feature selection and avoids the problem of reduced model efficiency due to feature redundancy or noise interference in traditional methods. The advantage of this method is that it not only improves the generalization ability and robustness of the prediction model, but also provides more efficient and accurate data support for subsequent dynamic correction and comprehensive evaluation links, thereby achieving comprehensive optimization of the environmental pollution monitoring process.

[0132] Example 4

[0133] This embodiment is explained in Example 3, please refer to Figure 1 Specifically: S4 includes S41;

[0134] S41, according to the obtained screening correlation feature set SFSR, the predicted pollution concentration Cp is corrected to obtain the dynamic pollution concentration Cd(t, s) in different time periods and different situations;

[0135] The dynamic pollution concentration Cd(t, s) is obtained by the following calculation formula:

[0136]

[0137] In the formula, Cd(t, s) represents the dynamic pollution concentration at time t and situation q, w(i(q)) represents the corrected weight value of the i-th feature in the associated feature set FSR under situation q, for example: when situation q = peak and situation q = rainfall, the proportion of situation q increases, otherwise it decreases, △FSR(i(t, q)) represents the change of the i-th feature in the associated feature set FSR at time t and situation q, which is obtained by comparing the actual amount of the i-th feature in the associated feature set FSR at time t and situation q with the historical mean or the preset basic value.

[0138] The S5 includes S51;

[0139] S51. By fitting the dynamic pollution concentration Cd(t, s) in different time periods and different situations, a comprehensive index Ctotal of pollutant concentration is obtained, and the index is compared with the preset pollutant concentration warning threshold Cthe to obtain the environmental pollution detection result;

[0140] The comprehensive index Ctotal is obtained by the following calculation formula:

[0141]

[0142] In the formula, m represents the comprehensive total number, which is obtained by the total number of time periods, T represents the total number of time periods, the total number of specific time periods = the comprehensive total number m, and Q represents the total number of situations;

[0143] The environmental pollution detection results are obtained by the following comparison method:

[0144] When the comprehensive index Ctotal is less than the pollutant concentration warning threshold Cthe, the environmental pollution detection result is qualified;

[0145] When the comprehensive index Ctotal ≥ the pollutant concentration warning threshold Cthe, the environmental pollution detection result is unqualified, indicating that the dynamic pollution concentration Cd(t, s) exceeds the standard in time period t and situation q.

[0146] In this embodiment, by dynamically correcting the screening association feature set SFSR, combined with the total number of time periods T and the total number of situations Q, the dynamic pollution concentration Cd(t, s) is effectively calculated, and based on the fitting results of the dynamic pollution concentration, the comprehensive index Ctotal of the pollutant concentration is obtained. Compared with the traditional static monitoring method, this method fully considers the dynamic impact of specific time periods and situations on pollutant concentrations, and accurately quantifies the contribution of dynamic changes by adjusting the correction weight value w(i(q)), significantly improving the accuracy and response flexibility of pollution concentration monitoring. In addition, by comparing the comprehensive index Ctotal with the pollutant concentration warning threshold Cthe, this method can quickly and scientifically determine whether the environmental pollution detection results are qualified, and accurately locate the time period and situation where the pollution exceeds the standard, providing a more targeted and operational basis for pollution control decisions. Its biggest feature is that the combination of dynamic correction and comprehensive evaluation makes pollution monitoring not only global, but also has higher situational adaptation ability and dynamic sensitivity, thereby effectively making up for the defect of insufficient response to complex environmental changes in traditional methods.

[0147] Example 5

[0148] Environmental pollution monitoring system based on big data, please refer to Figure 2,Specifically: including multi-dimensional data acquisition module, multi-dimensional data extraction module, data analysis module, dynamic correction module and evaluation decision module;

[0149] The multi-dimensional data acquisition module collects data from multiple fields through the Internet, including environmental monitoring stations, traffic monitoring equipment, meteorological sensors and industrial emission records, and forms a standard set of environmental pollution data D after preprocessing;

[0150] The multidimensional data extraction module extracts data from the environmental pollution data standard set D, obtains traffic flow pollution characteristics, meteorological factor pollution characteristics and pollution concentration characteristics, and analyzes the relationship between the characteristics through a multidimensional statistical method to obtain a correlation feature set FSR;

[0151] The data analysis module constructs a multi-dimensional prediction model according to the obtained associated feature set FSR, obtains the predicted pollution concentration Cp by training the prediction model, and screens the associated feature set FSR according to the predicted pollution concentration Cp to obtain the screened screened associated feature set SFSR;

[0152] The dynamic correction module corrects the predicted pollution concentration Cp according to the obtained screening correlation feature set SFSR to obtain the dynamic pollution concentration Cd(t, s) in different time periods and different situations;

[0153] The evaluation and decision-making module obtains the comprehensive index Ctotal of pollutant concentration by fitting the dynamic pollution concentration Cd(t, s) in different time periods and different situations, and compares it with the preset pollutant concentration warning threshold Cthe to obtain the environmental pollution detection result.

[0154] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. An environmental pollution monitoring method based on big data, characterized in that: The following steps are involved: S1. Collect data from multiple fields through the Internet, including environmental monitoring stations, traffic monitoring equipment, meteorological sensors and industrial emission records, and form a standard set of environmental pollution data D after preprocessing; S2. Extract data from the standard set of environmental pollution data D to obtain traffic flow pollution characteristics, meteorological factor pollution characteristics and pollution concentration characteristics, and analyze the relationship between the characteristics through multi-dimensional statistical methods to obtain the associated feature set FSR; S3, constructing a multi-dimensional prediction model based on the obtained associated feature set FSR, and obtaining the predicted pollution concentration Cp by training the prediction model, and screening the associated feature set FSR according to the predicted pollution concentration Cp to obtain the screened screened associated feature set SFSR; S4. According to the obtained screening correlation feature set SFSR, the predicted pollution concentration Cp is corrected to obtain the dynamic pollution concentration Cd(t, s) in different time periods and different situations; S5. By fitting the dynamic pollution concentration Cd(t, s) in different time periods and different situations, the comprehensive index Ctotal of pollutant concentration is obtained, and it is compared with the preset pollutant concentration warning threshold Cthe to obtain the environmental pollution detection result.

2. The environmental pollution monitoring method based on big data according to claim 1 is characterized in that: Said S1 includes S11 and S12; S11. Collect data on factors affecting environmental pollution from multiple fields through the Internet, including environmental monitoring stations, traffic monitoring equipment, meteorological sensors, and industrial emission records, and mark the data on factors affecting environmental pollution collected from environmental monitoring stations, traffic monitoring equipment, meteorological sensors, and industrial emission records to obtain an environmental monitoring station pollution data set Dm, a traffic monitoring data set Dt, a meteorological sensor data set Dw, and an industrial emission data set Di, and integrate them to form a multidimensional big data set Draw = {Dm, Dt, Dw, Di}; Among them, the data on factors affecting environmental pollution collected by the environmental monitoring station include PM2.5 fine particle concentration Px, PM10 inhalable particle concentration Pk, carbon monoxide concentration CO, nitrogen dioxide concentration NO2, ozone concentration O3 and sulfur dioxide concentration SO2; Traffic monitoring equipment collects data on factors that affect environmental pollution, including vehicle flow Ft, average vehicle speed Va, and road congestion index Ic; The data collected by meteorological sensors on factors affecting environmental pollution include air temperature Te, humidity Hh, wind speed Ww, wind direction Dd, and air pressure Pp; Industrial emission records collect data on factors that affect environmental pollution, including nitrogen oxide emissions ENOx, sulfur dioxide emissions ESO2, volatile organic compound emissions EVOC and particulate matter emissions EPM.

3. The environmental pollution monitoring method based on big data according to claim 2 is characterized in that: S12, performing data preprocessing on the multidimensional big data set Draw, including data cleaning preprocessing and data standardization preprocessing, to obtain a preprocessed environmental pollution data standard set D; Among them, data cleaning preprocessing is to process missing values, outliers and duplicate data of the environmental monitoring station pollution data set Dm, traffic monitoring data set Dt, meteorological sensor data set Dw and industrial emission data set Di in the multidimensional big data set Draw, including using interpolation and mean methods to process missing values, and using statistical rules to remove outliers. The statistical rules include using triple standard deviation statistics; Data standardization preprocessing eliminates the differences between different dimensions by standardizing the multidimensional big data set Draw after data cleaning preprocessing, including using Z-score standardization, Min-Max standardization and decimal calibration standardization for data standardization preprocessing.

4. The environmental pollution monitoring method based on big data according to claim 1 is characterized in that: The S2 includes S21 and S22; S21, extracting data from the standard set of environmental pollution data D, obtaining the environmental monitoring station pollution data set Dm, traffic monitoring data set Dt, meteorological sensor data set Dw and industrial emission data set Di after data preprocessing, obtaining traffic flow pollution features TDm, traffic flow pollution features TDt, meteorological factor pollution features TDw and pollution concentration features TDi through feature marking, and reorganizing and integrating them to obtain feature set F; The reorganized and integrated feature set F = {Px, Pk, CO, NO2, O3, SO2, Ft, Va, Ic, Te, Hh, Ww, Dd, Pp, ENOx, ESO2, EVOC, EPM}; S22. Use a multi-dimensional statistical method to analyze the relationship between features in the feature set F, evaluate the linear relationship between the features and the pollutant concentrations, obtain the correlation coefficient R (Fx, Fy) between the feature Fx and the feature Fy by performing correlation analysis on the x-th feature Fx and the y-th feature Fy in the feature set F, and substitute it into the feature set F to mark the association relationship between the feature Fx and the feature Fy. Simultaneously, eliminate the features in the feature set F that have no association relationship with the correlation coefficient R (Fx, Fy) to form the associated feature set FSR.

5. The environmental pollution monitoring method based on big data according to claim 1 is characterized in that: The correlation coefficient R(Fx, Fy) is obtained by the following calculation formula: Where Cov(Fx, Fy) represents the covariance of feature Fx and feature Fy, S(Fx) and S(Fy) represent the standard deviation of feature Fx and feature Fy respectively; The correlation coefficient R(Fx, Fy) reflects the linear relationship between the feature Fx and the feature Fy, and distinguishes the strong correlation, weak correlation and no correlation between the feature Fx and the feature Fy; When the correlation coefficient R(Fx, Fy) = 1, a completely positive correlation result is obtained, indicating that the features Fx and Fy change completely synchronously, and it is determined that the features Fx and Fy are strongly correlated; When the correlation coefficient R(Fx, Fy) = -1, a completely negative correlation result is obtained, indicating that the feature Fx and the feature Fy have completely opposite changes, and it is determined that the feature Fx and the feature Fy are weakly correlated; When the correlation coefficient R(Fx, Fy) = 0, a non-correlated result is obtained, indicating that there is no linear relationship between the feature Fx and the feature Fy; Among them, strong correlation includes strong positive correlation and strong negative correlation; weak correlation includes weak positive correlation and weak negative correlation; When the correlation coefficient R(Fx, Fy) ≥ Qthe, it means that the change between the feature Fx and the feature Fy is a strong linear synchronous change, and the feature Fx and the feature Fy are strongly positively correlated; When the correlation coefficient R(Fx, Fy)≥-Qthe, it means that the change between the feature Fx and the feature Fy is a strong linear reverse change, and the feature Fx and the feature Fy are strongly negatively correlated; When the correlation coefficient R(Fx, Fy)<Rthe, it means that the change between the feature Fx and the feature Fy is a weak linear synchronous change, and the feature Fx and the feature Fy are weakly positively correlated; When the correlation coefficient R(Fx, Fy) < -Rthe, it means that the change between the feature Fx and the feature Fy is a weak linear reverse change, and the feature Fx and the feature Fy are weakly negatively correlated; Among them, Qthe and Rthe represent the strong correlation threshold and the weak correlation threshold respectively, and 0<weak correlation threshold Rthe<strong correlation threshold Qthe<1, and the specific value is set by the user.

6. The environmental pollution monitoring method based on big data according to claim 5 is characterized in that: The S3 includes S31 and S32; S31, constructing a multi-dimensional prediction model according to the obtained correlation feature set FSR, including using a decision tree regression model and a linear regression model to establish a prediction model, and obtaining a predicted pollution concentration Cp by training the prediction model; The predicted pollution concentration Cp is obtained by the following calculation formula: In the formula, n represents the total number of features in the associated feature set FSR, FSR(i) represents the i-th feature in the associated feature set FSR, and c(i) represents the i-th feature weight value.

7. The environmental pollution monitoring method based on big data according to claim 6 is characterized by: S32, screening the associated feature set FSR according to the obtained predicted pollution concentration Cp and the correlation coefficient R (Fx, Fy), and obtaining the screened screened associated feature set SFSR; The screening is performed by S321 and S322; S321, by fitting the predicted pollution concentration Cp with the correlation coefficient R(Fx, Fy), obtain the pollution concentration correlation coefficient R(FSR(i), Cp) between the i-th feature in the associated feature set FSR and the predicted pollution concentration Cp; The pollution concentration correlation coefficient R (FSR (i), Cp) is obtained by the following calculation formula: Where COV(FSR(i), Cp) represents the covariance between the i-th feature in the associated feature set FSR and the predicted pollution concentration Cp, S(FSR(i)) and S(Cp) represent the standard deviation between the i-th feature in the associated feature set FSR and the predicted pollution concentration Cp; S322, compare the pollution concentration correlation coefficient R (FSR (i), Cp) with the preset rejection threshold TCthe to obtain a rejection screening mark result, perform secondary rejection processing on the associated feature set FSR according to the rejection screening mark result, and obtain a filtered associated feature set SFSR; The elimination screening marker results are obtained by the following comparison method: When |pollution concentration correlation coefficient R(FSR(i), Cp)|≥removal threshold TCthe, the removal screening mark result is obtained as the retention result, and the i-th feature in the associated feature set FSR is retained; When -removal threshold TCthe<pollution concentration correlation coefficient R(FSR(i), Cp)<removal threshold TCthe, the removal screening mark result is obtained as a non-retained result, and the i-th feature in the associated feature set FSR is removed.

8. The environmental pollution monitoring method based on big data according to claim 7 is characterized in that: The S4 includes S41; S41, according to the obtained screening correlation feature set SFSR, the predicted pollution concentration Cp is corrected to obtain the dynamic pollution concentration Cd(t, s) in different time periods and different situations; The dynamic pollution concentration Cd(t, s) is obtained by the following calculation formula: In the formula, Cd(t, s) represents the dynamic pollution concentration at time t and situation q, w(i(q)) represents the corrected weight value of the i-th feature in the associated feature set FSR under situation q, and △FSR(i(t, q)) represents the change of the i-th feature in the associated feature set FSR under time t and situation q.

9. The environmental pollution monitoring method based on big data according to claim 1 is characterized in that: The S5 includes S51; S51. By fitting the dynamic pollution concentration Cd(t, s) in different time periods and different situations, a comprehensive index Ctotal of pollutant concentration is obtained, and the index is compared with the preset pollutant concentration warning threshold Cthe to obtain the environmental pollution detection result; The comprehensive index Ctotal is obtained by the following calculation formula: In the formula, m represents the comprehensive total number, which is obtained by the total number of time periods, T represents the total number of time periods, the total number of specific time periods = the comprehensive total number m, and Q represents the total number of situations; The environmental pollution detection results are obtained by the following comparison method: When the comprehensive index Ctotal is less than the pollutant concentration warning threshold Cthe, the environmental pollution detection result is qualified; When the comprehensive index Ctotal ≥ the pollutant concentration warning threshold Cthe, the environmental pollution detection result is unqualified, indicating that the dynamic pollution concentration Cd(t, s) exceeds the standard in time period t and situation q.

10. An environmental pollution monitoring system based on big data, applied to the environmental pollution monitoring method based on big data according to any one of claims 1 to 9, characterized in that: It includes multi-dimensional data acquisition module, multi-dimensional data extraction module, data analysis module, dynamic correction module and evaluation decision module; The multi-dimensional data acquisition module collects data from multiple fields through the Internet, including environmental monitoring stations, traffic monitoring equipment, meteorological sensors and industrial emission records, and forms a standard set of environmental pollution data D after preprocessing; The multidimensional data extraction module extracts data from the environmental pollution data standard set D, obtains traffic flow pollution characteristics, meteorological factor pollution characteristics and pollution concentration characteristics, and analyzes the relationship between the characteristics through a multidimensional statistical method to obtain a correlation feature set FSR; The data analysis module constructs a multi-dimensional prediction model according to the obtained associated feature set FSR, obtains the predicted pollution concentration Cp by training the prediction model, and screens the associated feature set FSR according to the predicted pollution concentration Cp to obtain the screened screened associated feature set SFSR; The dynamic correction module corrects the predicted pollution concentration Cp according to the obtained screening correlation feature set SFSR to obtain the dynamic pollution concentration Cd(t, s) in different time periods and different situations; The evaluation and decision-making module obtains the comprehensive index Ctotal of pollutant concentration by fitting the dynamic pollution concentration Cd(t, s) in different time periods and different situations, and compares it with the preset pollutant concentration warning threshold Cthe to obtain the environmental pollution detection result.

Images