Intelligent environment monitoring method and system based on big data

Abstract

The invention relates to the technical field of environment monitoring, in particular to an intelligent environment monitoring method and system based on big data, and the method comprises the following steps: collecting real-time environment monitoring data, recording the indexes of air temperature, humidity and pollutant concentration from differential sensors, uploading and updating the data, and carrying out the data verification and format standardization. And obtaining a processed environment data set. According to the method, the volatility of each environmental index is quantified by calculating and analyzing the information entropy, abnormal and rare events in the data are effectively identified, the positioning precision of the environmental abnormality is further improved by recording the abnormal indexes and labeling the place and time, the response can be quickly made, and the accuracy of positioning the environmental abnormality is improved. According to the method, the environment change trend can be accurately identified and predicted, the real-time performance, the accuracy and the prediction capability of environment monitoring are greatly improved through deep processing and analysis of environment monitoring data, and environment management and policy decision are better supported.

Description

Technical Field

[0001] The present invention relates to the technical field of environmental monitoring, and in particular to an intelligent environmental monitoring method and system based on big data. Background Art

[0002] The field of environmental monitoring technology aims to track and evaluate changes in various parameters in the environment, covering the entire process from basic data collection to complex analytical processing, focusing on air quality, water quality, soil conditions, noise levels and radiation. The development of environmental monitoring technology can obtain data in real time and predict future environmental changes through advanced algorithms to support the decision-making process of environmental protection and sustainable development. Emerging intelligent environmental monitoring technologies, such as the use of Internet of Things (IoT) devices and big data analysis, are improving the frequency and accuracy of data collection, making environmental monitoring more efficient and comprehensive.

[0003] Among them, the intelligent environmental monitoring method of big data refers to the use of big data technology to optimize and improve the efficiency and accuracy of environmental monitoring. By integrating the data collected by environmental monitoring equipment and applying machine learning and data analysis technology, it can not only monitor the environmental conditions in real time, but also predict and simulate future environmental trends. The main uses include improving the response speed to environmental pollution, more accurately assessing environmental risks, optimizing resource allocation and environmental policy formulation, and effectively supporting environmental management and policy decision-making.

[0004] Although existing environmental monitoring technologies cover the entire process from data collection to analysis and processing, they show deficiencies in processing speed and real-time response when processing large-scale and diverse data. In particular, in terms of data verification and format standardization, there is a lack of sufficient flexibility and automation, which leads to data processing delays, affecting timeliness and the timeliness of decision-making. Existing technologies rely on traditional statistical methods when predicting future environmental changes, which require assumptions about data distribution, limiting the accuracy of predictions and the breadth of applications. In terms of anomaly detection, there is a lack of effective tools to quantitatively analyze and mark anomalies, making the response to environmental anomalies not timely or accurate enough. For example, pollution incidents that are not identified and responded to in a timely manner lead to increased risks to the environment and public health. Therefore, existing technologies have obvious deficiencies in the real-time, accuracy and predictive capabilities of data processing, which directly affect the efficiency of environmental monitoring and the formulation of environmental policies. Summary of the invention

[0005] The purpose of the present invention is to solve the shortcomings of the prior art and to propose an intelligent environment monitoring method and system based on big data.

[0006] In order to achieve the above object, the present invention adopts the following technical solution, a big data-based intelligent environment monitoring method, comprising the following steps: S1: Collect real-time environmental monitoring data, record indicators of temperature, humidity, and pollutant concentration from differentiated sensors, upload and update the data, and obtain processed environmental data sets through data verification and format standardization; S2: performing information entropy calculation on the indicators in the processed environmental data set, including analyzing the frequency distribution of each environmental indicator and calculating volatility, and obtaining information entropy analysis results; S3: Identify indicators with abnormal entropy values ​​through the information entropy analysis results, mark the indicators with corresponding time and location, analyze areas of environmental anomalies and rare events, and generate abnormal indicator records; S4: performing non-parametric estimation of probability distribution on the data in the abnormal indicator record, counting the frequency of each marked data point and estimating the distribution shape, to obtain a probability distribution estimation result; S5: using the probability distribution estimation result, analyzing the change trend of the environmental indicators in the time series according to the big data, determining the change of each indicator through trend testing, and obtaining the trend detection result; S6: Based on the trend detection results, the cumulative deviation value of each environmental indicator is calculated, and the deviation and abnormal changes in the long-term trend are identified by comparing the accumulation and average of long-term data, and an environmental monitoring log is generated.

[0007] As a further scheme of the present invention, the processed environmental data set includes standardized and verified data of temperature, humidity and pollutant concentration indicators collected from multiple differentiated sensors, the information entropy analysis results include volatility analysis results of the frequency distribution of temperature, humidity and pollutant concentration, and information entropy calculation results of each indicator, the abnormal indicator records include environmental indicators with abnormal information entropy values, associated time and place, and areas of environmental anomalies and rare events, the probability distribution estimation results include the estimated frequency distribution and distribution shape of abnormal indicator record data points, the trend detection results include test results of the determined change trend of each environmental indicator, the stability of the trend and the change points, and the environmental monitoring log includes the calculated cumulative deviation value of each environmental indicator, deviations and abnormal changes in long-term trends identified by long-term data comparison.

[0008] As a further solution of the present invention, the steps of collecting real-time environmental monitoring data, recording the indicators of temperature, humidity, and pollutant concentration from differential sensors, uploading and updating the data, and obtaining the processed environmental data set through data verification and format standardization are as follows: S101: Deploy a variety of environmental sensors to monitor the temperature, humidity and pollutant concentration in the target area. The sensors automatically collect data every minute, and the data is transmitted to the data center in real time via the wireless network to obtain real-time environmental data; S102: Using the real-time environmental data, perform data verification, including checking the data integrity, range and type, removing abnormal values, and performing data update once an hour to obtain verified environmental data; S103: Based on the verified environmental data, standardize the data format, including unifying the timestamp format, numerical units and data structure, check whether the data meets the standard requirements for analysis and storage, and obtain a processed environmental data set.

[0009] As a further solution of the present invention, information entropy calculation is performed on the indicators in the processed environmental data set, including analyzing the frequency distribution of each environmental indicator and calculating the volatility. The steps of obtaining the information entropy analysis results are specifically as follows: S201: extracting the data frequency of each environmental indicator from the processed environmental data set, including temperature, humidity and pollutant concentration, calculating the frequency of occurrence of statistical points in the data set, processing each indicator separately, and obtaining an indicator frequency distribution diagram; S202: Based on the indicator frequency distribution diagram, the volatility of each environmental indicator is calculated, and the fluctuation range of the indicator data is determined by analyzing the standard deviation and coefficient of variation of the data points. The volatility calculation is performed once a week to monitor the stability of the environmental conditions and obtain the indicator volatility analysis results; S203: Utilize the indicator volatility analysis result to perform information entropy calculation, use the information entropy formula to calculate the volatility of each environmental indicator, identify the randomness and information content of the data, and obtain the information entropy analysis result.

[0010] As a further solution of the present invention, the information entropy analysis results are used to identify indicators with abnormal entropy values, mark the indicators with corresponding time and place, analyze the areas of environmental anomalies and rare events, and generate abnormal indicator records in the following specific steps: S301: Using the information entropy analysis result and a decision tree algorithm, screen environmental indicators whose entropy values ​​deviate from the normal range, including temperature, humidity, and pollutant concentration, calculate the degree of entropy deviation of each indicator, and obtain a list of abnormal entropy value indicators; S302: Based on the abnormal entropy value indicator list, each abnormal indicator is associated with the corresponding timestamp and geographic location information, and the time and location of each abnormal event are verified to be recorded, and the environmental monitoring data is updated in real time according to the abnormal indicator to obtain abnormal indicator time and location association data; S303: Analyze the environmental anomalies and rare events in the target area, including climate change and sudden environmental events, based on the abnormal indicator time and location association data, evaluate the environmental stability of each area, and obtain abnormal indicator records.

[0011] As a further solution of the present invention, the formula of the decision tree algorithm is as follows: ; in, is the weighted average information entropy, Represents the split attribute Next The probability of a subset, It is The information entropy of the subsets is Representation attributes The number of values ​​of is the adjustment factor, represents the average size of the subsets, It is The size of the subset, is the total number of subsets.

[0012] As a further solution of the present invention, the steps of performing non-parametric estimation of probability distribution on the data in the abnormal index record, counting the frequency of each marked data point and estimating the distribution shape, and obtaining the probability distribution estimation result are specifically as follows: S401: extracting data points marked as abnormal based on the abnormal index record, performing frequency statistics on each data point, recording the number of occurrences in the entire data set, and obtaining an abnormal data frequency statistics table; S402: using the abnormal data frequency statistics table, estimating the probability distribution of each abnormal data point, without relying on the prior setting of the data distribution, analyzing the distribution shape of the data, and obtaining an approximate shape of the probability distribution; S403: Using the probability distribution approximate shape, integrating the distribution estimation results of the abnormal data points, identifying and analyzing the distribution characteristics of the abnormal environment indicators, including the peak, skewness and tail characteristics, and obtaining the probability distribution estimation results.

[0013] As a further solution of the present invention, the probability distribution estimation result is used to analyze the change trend of environmental indicators in the time series according to big data, and the change of each indicator is determined by trend testing. The steps of obtaining the trend detection result are specifically as follows: S501: extracting distribution data of environmental indicators based on the probability distribution estimation result, using the data to form a time series, arranging the time series data points of each indicator in the order of real-time occurrence, verifying the continuity and integrity of the time series, and obtaining the environmental indicator time series; S502: According to the environmental indicator time series, each time series is analyzed using a trend test, where the test does not require the data to follow a target distribution, and the trend change in the data set is detected to obtain an initial trend analysis result; S503: Using the initialization trend analysis result, determine the change trend of each environmental indicator, distinguish indicators with stable trends, evaluate the degree of change in environmental quality, and obtain trend detection results.

[0014] As a further solution of the present invention, based on the trend detection results, the cumulative deviation value of each environmental indicator is calculated, and the deviation and abnormal changes in the long-term trend are identified by comparing the accumulation and average of long-term data. The steps of generating an environmental monitoring log are specifically as follows: S601: based on the trend detection result, extract the time series data of each environmental indicator, calculate the difference between each data point and the long-term average value, and continuously track the deviation degree of the environmental indicator to obtain a deviation data table; S602: using the deviation data table, accumulating the deviation values ​​of the environmental indicators in time series, integrating the accumulated values ​​and arranging them in time sequence, and generating a cumulative deviation graph; S603: Compare the cumulative anomaly values ​​at multiple time points with the long-term average cumulative anomaly value through the cumulative anomaly graph, select the time period that deviates from the norm, and calibrate the abnormal changes according to the deviation to obtain the environmental monitoring log.

[0015] An intelligent environment monitoring system based on big data, the intelligent environment monitoring system based on big data is used to execute the above-mentioned intelligent environment monitoring method based on big data, the system comprises: The data collection and processing module collects data on temperature, humidity, and pollutant concentration from a variety of environmental sensors, performs data upload, verifies and standardizes the format of the raw data, and obtains a standardized environmental data set; The information entropy calculation module performs frequency distribution analysis on the environmental indicators in the standardized environmental data set, calculates the volatility of the indicators to determine the information entropy, and generates information entropy analysis results; The anomaly identification module identifies environmental indicators with abnormal information entropy values ​​based on the information entropy analysis results, marks abnormal indicators associated with target time and location, and obtains an abnormal indicator record set through regional analysis of abnormal and rare events; The data point frequency statistics module performs non-parametric estimation of the probability distribution of the data in the abnormal indicator record set, calculates the frequency of the data points, estimates the distribution shape, and generates a probability distribution estimation result; The trend analysis module uses the probability distribution estimation results to analyze the changing trends of environmental indicators in the big data time series, conducts trend tests to verify the changes in indicators, calculates the cumulative deviations of environmental indicators, identifies deviations in long-term trends, and generates environmental monitoring logs.

[0016] Compared with the prior art, the advantages and positive effects of the present invention are: In the present invention, by collecting real-time environmental monitoring data and uploading and updating it, the data is verified and the format is standardized to ensure that the processed data set has high accuracy and reliability. The calculation and analysis of information entropy quantifies the volatility of each environmental indicator, and the quantitative analysis method can effectively identify anomalies and rare events in the data. The recording of abnormal indicators and the marking of location and time further improve the positioning accuracy of environmental anomalies, which is convenient for rapid response. Non-parametric probability distribution estimation makes it unnecessary to assume the specific distribution form of the data, accurately count the frequency and distribution shape of each data point, and provide a deeper understanding of the fluctuation pattern of environmental indicators. The introduction of trend testing, combined with the comparison of long-term data accumulation values, helps to accurately identify and predict the trend of environmental changes, and provides data support for long-term environmental protection and policy making. Through in-depth processing and analysis of environmental monitoring data, the real-time, accuracy and predictive ability of environmental monitoring are greatly improved, and environmental management and policy decision-making are better supported. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 It is a schematic diagram of the workflow of the present invention; Figure 2 This is a detailed flow chart of S1 of the present invention; Figure 3 This is a detailed flow chart of S2 of the present invention; Figure 4 This is a detailed flow chart of S3 of the present invention; Figure 5 This is a detailed flow chart of S4 of the present invention; Figure 6 This is a detailed flow chart of S5 of the present invention; Figure 7 This is a detailed flow chart of S6 of the present invention; Figure 8 It is a system flow chart of the present invention. DETAILED DESCRIPTION

[0018] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0019] In the description of the present invention, it should be understood that the terms "length", "width", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside" and the like indicate positions or positional relationships based on the positions or positional relationships shown in the drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as limiting the present invention. In addition, in the description of the present invention, "multiple" means two or more, unless otherwise clearly and specifically defined.

[0020] See also Figure 1 The present invention provides a technical solution, an intelligent environment monitoring method based on big data, comprising the following steps: S1: Collect real-time environmental monitoring data, record indicators of temperature, humidity, and pollutant concentration from differentiated sensors, upload and update the data, and obtain processed environmental data sets through data verification and format standardization; S2: Perform information entropy calculation on the indicators in the processed environmental data set, including analyzing the frequency distribution of each environmental indicator and calculating the volatility, and obtaining the information entropy analysis results; S3: Identify indicators with abnormal entropy values ​​through information entropy analysis results, mark the indicators with corresponding time and location, analyze areas of environmental anomalies and rare events, and generate abnormal indicator records; S4: Perform non-parametric estimation of the probability distribution of the data in the abnormal indicator record, count the frequency of each marked data point and estimate the distribution shape to obtain the probability distribution estimation result; S5: Using the probability distribution estimation results, according to the big data analysis of the change trend of environmental indicators in the time series, the change of each indicator is determined through trend testing to obtain the trend detection results; S6: Based on the trend detection results, calculate the cumulative deviation value of each environmental indicator, identify deviations and abnormal changes in the long-term trend by comparing the accumulation and average of long-term data, and generate an environmental monitoring log.

[0021] The processed environmental data set includes standardized and verified data of temperature, humidity and pollutant concentration indicators collected from multiple differentiated sensors. The information entropy analysis results include the volatility analysis results of the frequency distribution of temperature, humidity and pollutant concentration, and the information entropy calculation results of each indicator. The abnormal indicator records include environmental indicators with abnormal information entropy values, the associated time and place, and the areas of environmental anomalies and rare events. The probability distribution estimation results include the estimated frequency distribution and distribution shape of the abnormal indicator record data points. The trend detection results include the test results of the determined change trend of each environmental indicator, the stability of the trend and the change points. The environmental monitoring log includes the calculated cumulative deviation value of each environmental indicator, the deviations and abnormal changes in the long-term trend identified by long-term data comparison.

[0022] See also Figure 2 , collect real-time environmental monitoring data, record the indicators of temperature, humidity, and pollutant concentration from differentiated sensors, upload and update the data, and obtain the processed environmental data set through data verification and format standardization. The specific steps are: S101: deploy a variety of environmental sensors to monitor the temperature, humidity and pollutant concentration of the target area. The sensors automatically collect data every minute, and the data is transmitted to the data center in real time via the wireless network. The execution process of obtaining real-time environmental data is as follows; Deploy a variety of environmental sensors in the target area to monitor temperature, humidity and pollutant concentrations. The sensor settings need to accurately match the environmental characteristics to ensure the validity and accuracy of the data. The sensor selection includes temperature and humidity sensors and air quality monitoring equipment. The equipment automatically collects data every minute. The data collection involves the timed activation of the sensor and the data reading process to ensure that the data every minute can reflect the environmental conditions at that moment. The collected data is transmitted to the data center in real time via the wireless network. During the data transmission process, the stability and transmission speed of the network need to be ensured to prevent data loss or errors during the transmission process. After the data is successfully transmitted to the data center, real-time environmental data is obtained.

[0023] S102: Use real-time environmental data to perform data verification, including checking data integrity, scope and type, removing abnormal values, and performing data updates every hour. The execution process of obtaining verified environmental data is as follows; Real-time environmental data is used to perform data verification, which involves checks on data integrity, scope, and type. During the verification process, the data is systematically screened to ensure that all data is within a predetermined reasonable range. The data verification process includes automatic identification and elimination of outliers. For data that exceeds the range, the system will mark and exclude it to ensure that it is not included in subsequent data analysis. The process is executed once an hour to ensure the accuracy and reliability of each batch of data. The data update process includes data cleaning, updating of error records, and refresh of the database to obtain verified environmental data.

[0024] S103: Based on the verified environmental data, the data format is standardized, including the unified timestamp format, numerical unit and data structure, and the data is checked to meet the standard requirements for analysis and storage. The execution flow of the processed environmental data set is as follows; Based on the verified environmental data, the data format is standardized according to the formula Calculate the average value of each environmental data. Represents the average value of the formatted data. represents a single data point, Represents the total number of data points. Formula explanation and formula calculation derivation process: Consider a simple example, if there are five data points with temperature values ​​of 20°C, 22°C, 21°C, 23°C, and 19°C. The average of the values ​​can be obtained by inserting it into the formula The specific steps are as follows: The calculation process shows how to convert the actual collected data values ​​into average values, which are then used for data standardization. The process not only involves data averaging, but also includes the unification of timestamp formats and the organization of data structures to ensure that the data meets the standard requirements for analysis and storage.

[0025] See also Figure 3 , perform information entropy calculation on the indicators in the processed environmental data set, including analyzing the frequency distribution of each environmental indicator and calculating the volatility. The steps to obtain the information entropy analysis results are as follows: S201: Extract the data frequency of each environmental indicator, including temperature, humidity and pollutant concentration, from the processed environmental data set, calculate the frequency of occurrence of the statistical points in the data set, process each indicator separately, and obtain the indicator frequency distribution diagram. The execution flow is as follows; The data frequency of each environmental indicator is extracted through the processed environmental data set, which involves data processing of indicators such as temperature, humidity and pollutant concentration. The processing includes counting the frequency of occurrence of each indicator data point in the entire data set. The frequency is calculated based on the number of occurrences of each data point divided by the total number of data points. The frequency statistical method ensures that the processing of each indicator is independent, accurately reflects the normal distribution of each environmental factor, and obtains a frequency distribution diagram.

[0026] S202: Based on the indicator frequency distribution diagram, the volatility of each environmental indicator is calculated, and the volatility range of the indicator data is determined by analyzing the standard deviation and coefficient of variation of the data points. The volatility calculation is performed once a week to monitor the stability of the environmental conditions. The execution process of obtaining the indicator volatility analysis result is as follows; Based on the indicator frequency distribution diagram, the volatility of each environmental indicator is calculated according to the formula and Calculate the standard deviation and coefficient of variation. represents the standard deviation, represents a single data point, represents the average value, represents the total number of data points, Represents the coefficient of variation. Detailed explanation of the formula and the process of formula calculation: Suppose there is a set of data points of an environmental indicator as [18, 20, 22, 20, 19], and calculate the average value of the data: , plug the value of each data point into the formula for standard deviation: , calculate the coefficient of variation: ,The calculation process shows that the volatility of the indicator is quantified through the standard deviation and ,coefficient of variation. The calculation of volatility helps to determine the stability of ,environmental conditions. The volatility calculation performed weekly ,provides data support for monitoring environmental conditions.

[0027] S203: Using the indicator volatility analysis results, information entropy calculation is performed, and the volatility of each environmental indicator is calculated using the information entropy formula, and the randomness and information content of the data are identified. The execution process of obtaining the information entropy analysis results is as follows; The information entropy is calculated using the results of the indicator volatility analysis. Information entropy is a method to measure the randomness and information content of data. The information entropy formula is used to calculate the volatility of each environmental indicator, which involves substituting the probability distribution of each indicator into the information entropy formula. During the calculation process, the information entropy calculation formula is: ,in, The probability of occurrence of the representative indicator value. The information entropy calculation of each environmental indicator is carried out independently and is not affected by the remaining indicators. The calculation result of the information entropy can identify and quantify the volatility and complexity of the data, provide important reference data for the environmental monitoring and early warning system, and obtain the information entropy analysis result.

[0028] See also Figure 4 ,Through the information entropy analysis results, we identify the indicators with abnormal entropy values, mark the indicators with the corresponding time and place, analyze the areas of environmental anomalies and rare events, and generate abnormal indicator records in the following steps: S301: Using the information entropy analysis results and the decision tree algorithm, environmental indicators whose entropy values ​​deviate from the normal range are screened, including temperature, humidity and pollutant concentration, and the degree of entropy deviation of each indicator is calculated to obtain the abnormal entropy value indicator list. The execution process is as follows; The formula for the decision tree algorithm is as follows: ; in, is the weighted average information entropy, Represents the split attribute Next The probability of a subset, It is The information entropy of the subsets is Representation attributes The number of values ​​of is the adjustment factor, represents the average size of the subsets, It is The size of the subset, is the total number of subsets.

[0029] To demonstrate the specific application and calculation process of the formula, consider a simple environmental monitoring data set, including air temperature, humidity and pollutant concentration, divided into three subsets. The specific parameters and data come from field measurements.

[0030] Probability of subset : Indicates the proportion of each subset in the total data set. For example, if the three subsets are evenly distributed, the probability of each subset is .

[0031] Information Entropy :The information entropy of each subset is calculated based on the volatility of the content. For example, suppose the information entropy of the subsets calculated is 0.8, 0.5, and 0.9 respectively.

[0032] property The number of values : The number of possible values ​​of an attribute, such as temperature, humidity, and pollutant concentration each have 3 different states, so .

[0033] Adjustment factor : Determined through the volatility study of actual data, it is set to 0.5 to balance information gain and data volatility.

[0034] Subset size and the average subset size : If each subset has an equal number of data points, for example, the size of each subset is 100, then .

[0035] Substitute the actual values ​​for calculation: Set the information entropy of three subsets They are 0.8, 0.5 and 0.9 respectively.

[0036] ; The results show that under the current environmental monitoring settings, the attributes The weighted average information entropy of the data (i.e., temperature, humidity, and pollutant concentration) is 0.7333. The value reflects the overall volatility of the data. The correlation lies in determining the overall entropy of the data set under given environmental conditions, which can be used for the classification and evaluation of environmental quality.

[0037] S302: Based on the abnormal entropy value indicator list, each abnormal indicator is associated with the corresponding timestamp and geographic location information, and the time and location of each abnormal event are verified to be recorded. The environmental monitoring data is updated in real time according to the abnormal indicator, and the execution process of obtaining the abnormal indicator time and location associated data is as follows; Based on the list of abnormal entropy indicators, verify the time and location of each abnormal event according to the formula Associate each anomaly indicator with a timestamp and geographic location. Represents the comprehensive entropy value of abnormal events, The timestamp representing the event, The coded value representing the geographic location information, Represents the number of events. Detailed explanation of the formula and the process of formula calculation: Consider three abnormal events with timestamps of (unit is minutes from the start of the day), the geographic location is encoded as . Substitute the values ​​into the formula to calculate: =(158· )+(162· )+(170· ) 158·1.099+162·1.609+170·1.386 173.662+260.658+235.62=669.94. This result shows that by associating timestamp and geographic location information, a comprehensive entropy value can be calculated, which represents the complexity and volatility of the spatial and temporal distribution of abnormal events, providing a quantitative basis for further data analysis.

[0038] S303: Analyze the environmental anomalies and rare events in the target area, including climate change and sudden environmental events, based on the abnormal indicator time and location association data, evaluate the environmental stability of each area, and obtain the execution process of the abnormal indicator record as follows; According to the time and place correlation data of abnormal indicators, the environmental anomalies and rare events in the target area are analyzed, including the assessment of anomalies caused by climate change and sudden environmental events. The analysis process involves comparing the time and place correlation data with known environmental stability records, identifying data points that are significantly different from the normal state, and updating the environmental monitoring data in real time to ensure that the data in each area reflects the latest environmental status. The evaluation process includes a comprehensive analysis of the volatility and frequency distribution of environmental indicators to obtain abnormal indicator records.

[0039] See also Figure 5 , perform non-parametric estimation of the probability distribution of the data in the abnormal indicator record, count the frequency of each marked data point and estimate the distribution shape, and the specific steps to obtain the probability distribution estimation result are: S401: Based on the abnormal index record, extract the data points marked as abnormal, perform frequency statistics on each data point, record the number of occurrences in the entire data set, and obtain the abnormal data frequency statistics table. The execution process is as follows; Based on the abnormal indicator records, data points marked as abnormal are extracted, and frequency statistics of the data points are performed, which involves calculating the number of occurrences of each data point in the entire data set. The frequency statistical method records the points marked as abnormal by checking the status of each data point in the data set one by one. The creation of the statistical table not only depends on the identification and marking process of the data points, but also requires accurate calculation of the number of occurrences of each abnormal point. The process ensures the accuracy and reliability of the statistical results and provides basic data for subsequent analysis. The frequency of data points reveals the frequency of occurrence of abnormal events, which provides a basis for in-depth analysis of the causes and impacts of abnormalities, and obtains a statistical table of the frequency of abnormal data.

[0040] S402: using the abnormal data frequency statistics table, estimating the probability distribution of each abnormal data point, not relying on the prior setting of the data distribution, analyzing the distribution shape of the data, and obtaining the approximate shape of the probability distribution. The execution flow is as follows; Use the abnormal data frequency statistics table to estimate the probability distribution of each abnormal data point, according to the formula Calculate the probability of a data point appearing. In the formula, Representative data points The probability of occurrence, Representative data points The number of occurrences of Represents the total number of data points. Detailed explanation of the formula and formula calculation derivation process: Consider a simple example, assuming that the abnormal data point A appears 15 times in the data set, and the total number of data points in the data set is 500. Substitute the value into the probability formula: The calculation shows that the data points The probability of occurrence is 3%, and the probability estimate is based on the actual number of occurrences of the data point, rather than relying on any prior assumptions about the data distribution. This provides a basis for further statistical analysis, allowing the actual distribution shape of the data to be analyzed.

[0041] S403: Using the probability distribution approximate shape, integrating the distribution estimation results of the abnormal data points, identifying and analyzing the distribution characteristics of the abnormal environment indicators, including the peak, skewness and tail characteristics, the execution flow of obtaining the probability distribution estimation results is as follows; The probability distribution approximate shape is adopted to integrate the distribution estimation results of abnormal data points. The process includes identifying and analyzing the distribution characteristics of abnormal environmental indicators. The analysis of distribution characteristics focuses on the peak, skewness and tail characteristics of abnormal data. The characteristics are obtained through statistical analysis and are based on the actual distribution of data points. It is not only a theoretical model. By observing the actual distribution of data, we can more accurately understand the nature and environmental impact of abnormal events and obtain probability distribution estimation results.

[0042] See also Figure 6 , using the probability distribution estimation results, according to the big data analysis of the changing trends of environmental indicators in the time series, and determining the changes of each indicator through trend testing, the specific steps to obtain the trend detection results are: S501: Based on the probability distribution estimation result, the distribution data of the environmental indicators are extracted, and the data is used to form a time series. The time series data points of each indicator are arranged in the order of real-time occurrence, and the continuity and integrity of the time series are verified. The execution process of obtaining the environmental indicator time series is as follows; Based on the probability distribution estimation results, the distribution data of environmental indicators are extracted and the data are used to form a time series. The creation of the time series involves arranging the data points of each indicator in the order of real-time occurrence. The process ensures the continuity and integrity of the time series. The continuity of the time series is verified by checking whether there are missing or inconsistent intervals between timestamps. The verification of continuity is a key step to ensure the accuracy of time series analysis. The integrity of the time series is ensured by fully recording all data points from the beginning of observation to the present, and the time series of environmental indicators is obtained.

[0043] S502: According to the environmental indicator time series, each time series is analyzed using a trend test. The test does not require the data to follow the target distribution. The trend changes in the data set are detected. The execution process of obtaining the initial trend analysis result is as follows; According to the time series of environmental indicators, trend test is used to analyze each time series, according to the formula Test the trend change in the data set. Where T represents the trend value, represents a data point in a time series, Represents the corresponding time stamp, Represents the number of data points. Formula explanation and formula calculation derivation process: Set a time series data point set to [10,12,13,14,15], associated with the time mark [1,2,3,4,5]. Substitute the value into the trend test formula: The calculation shows the sum of the products of the time series data points and the time markers divided by the square root of the sum of the squares of the time markers. This value provides a measure of trend change in the data set and can determine whether the data is growing or declining.

[0044] S503: Using the initialization trend analysis results, determine the change trend of each environmental indicator, distinguish the indicators with stable trends, evaluate the degree of change in environmental quality, and obtain the trend detection results. The execution process is as follows; Using the results of the initialized trend analysis, the changing trend of each environmental indicator is determined. In the process, indicators with stable trends are distinguished. The analysis is based on the results of time series data and trend tests. Indicators with stable trends are evaluated by the size and direction of trend values. Assessing the degree of change in environmental quality involves explaining the relationship between trend test results and actual changes in environmental indicators, revealing the stability and trend of each environmental indicator over time, providing important information for environmental management and policy making, helping to guide future environmental protection measures, and obtaining trend detection results.

[0045] See also Figure 7 Based on the trend detection results, the cumulative deviation value of each environmental indicator is calculated. By comparing the accumulation and average of long-term data, the deviation and abnormal changes in the long-term trend are identified. The specific steps for generating environmental monitoring logs are as follows: S601: Based on the trend detection results, extract the time series data of each environmental indicator, calculate the difference between each data point and the long-term average value, and continue to track the deviation degree of the environmental indicator to obtain the execution process of the deviation data table as follows; Based on the trend detection results, extract the time series data of each environmental indicator, calculate the difference between each data point and the long-term average value, and use the formula Continuously track the deviation of environmental indicators.

[0046] In the formula, Representative data points The long-term average Detailed explanation of the formula and the process of formula calculation: Set the time series data points to [15, 18, 20, 14, 19] and the long-term average to 17. Insert the data and calculate the deviation value of each point: , , , , This calculation process shows that the degree of deviation of each data point is calculated and recorded. This continuous tracking helps identify abnormal changes in environmental indicators and generates deviation data tables, which provide important basic data for further analysis.

[0047] S602: Using the deviation data table, the deviation values ​​of the environmental indicators are accumulated in time series, the accumulated values ​​are integrated and arranged in time sequence, and the execution process of generating a cumulative deviation map is as follows; The deviation data table is used to accumulate the deviation values ​​of environmental indicators in time series. In the process, the accumulated values ​​are integrated and arranged in chronological order. The accumulation process involves adding up the continuous deviation values, and the deviation value of each new time point is added to the previous cumulative total value. Such accumulation shows the dynamic change of deviation over time and can clearly show the cumulative deviation trend of data points. It is particularly important for environmental monitoring and trend analysis, and provides a visual tool to observe and compare long-term changes in environmental data and generate cumulative deviation maps.

[0048] S603: Compare the cumulative anomaly values ​​of multiple time points with the long-term average cumulative anomaly value through the cumulative anomaly graph, screen the time period that deviates from the norm, and calibrate the abnormal changes according to the deviations. The execution process of the environmental monitoring log is as follows; Through the cumulative anomaly chart, the cumulative anomaly values ​​of multiple time points are compared with the long-term average cumulative anomaly value. The comparison process involves screening out the time periods that deviate from the norm. For each time period, the abnormal changes are calibrated according to the degree of deviation. This not only helps to identify when the data points deviate from the norm, but also reveals the trend and severity of the deviation. By recording the changes in environmental quality in detail, it provides important information for understanding the patterns and causes of environmental changes, helps to formulate response measures and long-term environmental protection strategies, and obtains an environmental monitoring log.

[0049] See also Figure 8 , an intelligent environment monitoring system based on big data, the intelligent environment monitoring system based on big data is used to execute the above-mentioned intelligent environment monitoring method based on big data, the system includes: The data collection and processing module collects data on temperature, humidity, and pollutant concentration from a variety of environmental sensors, performs data upload, verifies and standardizes the format of the raw data, and obtains a standardized environmental data set; The information entropy calculation module performs frequency distribution analysis on environmental indicators in the standardized environmental data set, calculates the volatility of the indicators to determine the information entropy, and generates information entropy analysis results; The anomaly identification module identifies environmental indicators with abnormal information entropy values ​​based on the information entropy analysis results, marks abnormal indicators associated with the target time and location, and obtains an abnormal indicator record set through regional analysis of abnormal and rare events; The data point frequency statistics module performs non-parametric estimation of the probability distribution of the data in the abnormal indicator record set, calculates the frequency of the data points, estimates the distribution shape, and generates the probability distribution estimation result; The trend analysis module uses probability distribution estimation results to analyze the changing trends of environmental indicators in big data time series, conducts trend tests to verify the changes in indicators, calculates the cumulative deviations of environmental indicators, identifies deviations in long-term trends, and generates environmental monitoring logs.

[0050] The above are only preferred embodiments of the present invention and are not intended to limit the present invention in other forms. Any technician familiar with the profession may use the technical contents disclosed above to change or modify them into equivalent embodiments with equivalent changes and apply them to other fields. However, any simple modification, equivalent change and modification made to the above embodiments based on the technical essence of the present invention without departing from the technical solution of the present invention still falls within the protection scope of the technical solution of the present invention.

Claims

1. An intelligent environmental monitoring method based on big data, characterized in that: The following steps are involved: Collect real-time environmental monitoring data, record indicators of temperature, humidity, and pollutant concentration from differentiated sensors, upload and update the data, and obtain processed environmental data sets through data verification and format standardization; Performing information entropy calculation on the indicators in the processed environmental data set, including analyzing the frequency distribution of each environmental indicator and calculating volatility, and obtaining information entropy analysis results; Through the information entropy analysis results, identify indicators with abnormal entropy values, mark the indicators with corresponding time and place, analyze the areas of environmental anomalies and rare events, and generate abnormal indicator records; Performing non-parametric estimation of probability distribution on the data in the abnormal indicator record, counting the frequency of each marked data point and estimating the distribution shape, to obtain a probability distribution estimation result; Using the probability distribution estimation result, analyzing the change trend of environmental indicators in the time series according to big data, determining the change of each indicator through trend testing, and obtaining the trend detection result; Based on the trend detection results, the cumulative deviation value of each environmental indicator is calculated, and by comparing the accumulation and average of long-term data, deviations and abnormal changes in long-term trends are identified to generate an environmental monitoring log.

2. The intelligent environment monitoring method based on big data according to claim 1 is characterized in that: The processed environmental data set includes standardized and verified data of air temperature, humidity and pollutant concentration indicators collected from multiple differentiated sensors, the information entropy analysis results include volatility analysis results of the frequency distribution of air temperature, humidity and pollutant concentration, and information entropy calculation results of each indicator, the abnormal indicator records include environmental indicators with abnormal information entropy values, associated time and place, and areas of environmental anomalies and rare events, the probability distribution estimation results include the estimated frequency distribution and distribution shape of abnormal indicator record data points, the trend detection results include the test results of the determined change trend of each environmental indicator, the stability of the trend and the change points, and the environmental monitoring log includes the calculated cumulative deviation value of each environmental indicator, deviations and abnormal changes in long-term trends identified by long-term data comparison.

3. The intelligent environment monitoring method based on big data according to claim 1 is characterized in that: The steps to collect real-time environmental monitoring data, record the indicators of temperature, humidity, and pollutant concentration from differentiated sensors, upload and update the data, and obtain the processed environmental data set through data verification and format standardization are as follows: Deploy a variety of environmental sensors to monitor the temperature, humidity and pollutant concentration in the target area. The sensors automatically collect data every minute, and the data is transmitted to the data center in real time via wireless networks to obtain real-time environmental data; Using the real-time environmental data, perform data verification, including checking the data integrity, scope and type, removing abnormal values, and performing data updates once an hour to obtain verified environmental data; Based on the verified environmental data, data format standardization is performed, including unified timestamp format, numerical units and data structure, and the data is checked to meet standard requirements for analysis and storage to obtain a processed environmental data set.

4. The intelligent environment monitoring method based on big data according to claim 1 is characterized in that: Information entropy calculation is performed on the indicators in the processed environmental data set, including analyzing the frequency distribution of each environmental indicator and calculating the volatility. The steps of obtaining the information entropy analysis results are specifically as follows: Extract the data frequency of each environmental indicator from the processed environmental data set, including temperature, humidity and pollutant concentration, calculate the frequency of occurrence of statistical points in the data set, process each indicator separately, and obtain an indicator frequency distribution diagram; Based on the indicator frequency distribution diagram, the volatility of each environmental indicator is calculated, and the fluctuation range of the indicator data is determined by analyzing the standard deviation and coefficient of variation of the data points. The volatility calculation is performed once a week to monitor the stability of the environmental conditions and obtain the indicator volatility analysis results; The information entropy is calculated using the indicator volatility analysis results, and the volatility of each environmental indicator is calculated using the information entropy formula to identify the randomness and information content of the data to obtain the information entropy analysis results.

5. The intelligent environment monitoring method based on big data according to claim 1 is characterized in that: Through the information entropy analysis results, the indicators with abnormal entropy values ​​are identified, the indicators are marked with corresponding time and place, the areas of environmental anomalies and rare events are analyzed, and the steps of generating abnormal indicator records are specifically as follows: Using the information entropy analysis results and a decision tree algorithm, environmental indicators whose entropy values ​​deviate from the normal range are screened, including temperature, humidity, and pollutant concentration, and the degree of entropy deviation of each indicator is calculated to obtain a list of abnormal entropy value indicators; Based on the abnormal entropy value indicator list, each abnormal indicator is associated with the corresponding timestamp and geographic location information, and the time and location of each abnormal event are verified to be recorded. The environmental monitoring data is updated in real time according to the abnormal indicator to obtain the abnormal indicator time and location association data; According to the time and place correlation data of the abnormal indicators, the environmental anomalies and rare events in the target area, including climate change and sudden environmental events, are analyzed, the environmental stability of each area is evaluated, and the abnormal indicator records are obtained.

6. The intelligent environment monitoring method based on big data according to claim 5 is characterized in that: The formula of the decision tree algorithm is as follows: ； in, is the weighted average information entropy, Represents the split attribute Next The probability of a subset, It is The information entropy of the subsets is Representation attributes The number of values ​​of is the adjustment factor, represents the average size of the subsets, It is The size of the subset, is the total number of subsets.

7. The intelligent environment monitoring method based on big data according to claim 1 is characterized in that: The steps of performing non-parametric estimation of probability distribution on the data in the abnormal indicator record, counting the frequency of each marked data point and estimating the distribution shape, and obtaining the probability distribution estimation result are specifically as follows: Based on the abnormal index record, extract the data points marked as abnormal, perform frequency statistics on each data point, record the number of occurrences in the entire data set, and obtain an abnormal data frequency statistics table; The abnormal data frequency statistics table is used to estimate the probability distribution of each abnormal data point, without relying on the prior setting of the data distribution, and the distribution shape of the data is analyzed to obtain the approximate shape of the probability distribution; The probability distribution approximate shape is adopted, the distribution estimation results of abnormal data points are integrated, the distribution characteristics of abnormal environmental indicators are identified and analyzed, including peak, skewness and tail characteristics, and the probability distribution estimation results are obtained.

8. The intelligent environment monitoring method based on big data according to claim 1 is characterized in that: Using the probability distribution estimation result, according to the big data analysis time series, the change trend of environmental indicators is determined by trend testing, and the steps of obtaining the trend detection result are as follows: Based on the probability distribution estimation result, the distribution data of the environmental indicators are extracted, and the data are used to form a time series. The time series data points of each indicator are arranged in the order of real-time occurrence, and the continuity and integrity of the time series are verified to obtain the environmental indicator time series; According to the environmental indicator time series, each time series is analyzed using a trend test. The test does not require the data to follow the target distribution. The trend changes in the data set are detected to obtain an initial trend analysis result. The initialization trend analysis result is used to determine the change trend of each environmental indicator, distinguish indicators with stable trends, evaluate the degree of change in environmental quality, and obtain trend detection results.

9. The intelligent environment monitoring method based on big data according to claim 1 is characterized in that: Based on the trend detection results, the cumulative deviation value of each environmental indicator is calculated, and the deviation and abnormal changes in the long-term trend are identified by comparing the accumulation and average of long-term data. The steps of generating an environmental monitoring log are specifically as follows: Based on the trend detection results, extract the time series data of each environmental indicator, calculate the difference between each data point and the long-term average value, and continuously track the deviation degree of the environmental indicator to obtain a deviation data table; Using the deviation data table, the deviation values ​​of the environmental indicators are accumulated in time series, the accumulated values ​​are integrated and arranged in time sequence, and a cumulative deviation graph is generated; By comparing the cumulative anomaly values ​​at multiple time points with the long-term average cumulative anomaly value through the cumulative anomaly diagram, the time periods that deviate from the norm are screened, and the abnormal changes are calibrated according to the deviations to obtain the environmental monitoring log.

10. An intelligent environment monitoring system based on big data, characterized in that: According to any one of claims 1 to 9, the intelligent environment monitoring method based on big data comprises: The data collection and processing module collects data on temperature, humidity, and pollutant concentration from a variety of environmental sensors, performs data upload, verifies and standardizes the format of the raw data, and obtains a standardized environmental data set; The information entropy calculation module performs frequency distribution analysis on the environmental indicators in the standardized environmental data set, calculates the volatility of the indicators to determine the information entropy, and generates information entropy analysis results; The anomaly identification module identifies environmental indicators with abnormal information entropy values ​​based on the information entropy analysis results, marks abnormal indicators associated with target time and location, and obtains an abnormal indicator record set through regional analysis of abnormal and rare events; The data point frequency statistics module performs non-parametric estimation of the probability distribution of the data in the abnormal indicator record set, calculates the frequency of the data points, estimates the distribution shape, and generates a probability distribution estimation result; The trend analysis module uses the probability distribution estimation results to analyze the changing trends of environmental indicators in the big data time series, conducts trend tests to verify the changes in indicators, calculates the cumulative deviations of environmental indicators, identifies deviations in long-term trends, and generates environmental monitoring logs.

Images