Data analysis system and method based on environment detection cloud platform

Through the data analysis system based on the environmental detection cloud platform, the problem of environmental detection data being prone to abnormalities in the outdoor environment is solved, and more accurate and complete data analysis results are achieved.

CN119557295AActive Publication Date: 2025-03-04GUANGZHOU DELONG ENVIRONMENTAL TESTING TECH CO LTD
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202510119439.4
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2025-01-24
Publication Date
2025-03-04
Estimated Expiration
2045-01-24

AI Technical Summary

Technical Problem

Existing environmental testing technologies are susceptible to natural or biological factors in outdoor environments, resulting in abnormalities in the detection data and affecting the detection results.

Method used

A data analysis system based on an environmental detection cloud platform is adopted, which includes an environmental detection module, an environmental detection cloud platform and a data transmission module. Through the environmental detection cloud platform, the environmental detection data of the target monitoring site is analyzed in correlation with the data of other monitoring sites, the reference monitoring site is determined, and the data is converted into the data of the reference monitoring site through the neural network model, and abnormal data is removed and filled.

Benefits of technology

It improves the accuracy of environmental detection data analysis results, enhances the persuasiveness of the analysis results, and improves the integrity of the data.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119557295A_ABST
    Figure CN119557295A_ABST
Patent Text Reader

Abstract

The invention discloses a data analysis system and method based on an environment detection cloud platform, and relates to the technical field of environment detection, and the method comprises the steps: determining a reference monitoring station according to the correlation between environment detection data of all monitoring stations; performing conversion to obtain first environment detection data and second environment detection data of the reference monitoring station, and analyzing data distribution of the first environment detection data and the second environment detection data; removing and filling the abnormal data; the environment detection data is converted to the same reference monitoring station to be analyzed, data distribution at the reference monitoring station is analyzed, the influence of model precision is small, and the accuracy of the analysis result of the environment detection data is further improved; under the condition of determining that the abnormity exists, the data distribution of the target monitoring station is analyzed, so that the analysis result is more persuasive; and the abnormal data is filled, so that the integrity of the data is improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of environmental detection, and in particular to a data analysis system and method based on an environmental detection cloud platform. Background Art

[0002] Environmental testing is one of the important means of environmental protection. Through continuous environmental testing, we can timely understand the environmental quality status, discover problems of environmental pollution and environmental damage, and provide a scientific basis for environmental protection. At present, environmental testing has received widespread attention and implementation around the world. Many countries and regions have established a complete environmental testing system to continuously monitor and analyze environmental factors such as atmosphere, water, soil, and organisms. At the same time, with the development of science and technology, the technology and methods of environmental testing are constantly updated and improved, which improves the accuracy and efficiency of testing; however, due to the complexity of the outdoor environment, when testing the environment, it is easy to be affected by natural or biological factors, resulting in abnormal test data and affecting the test results. Summary of the invention

[0003] The purpose of the present invention is to provide a data analysis system and method based on an environmental detection cloud platform to solve the problems raised in the prior art.

[0004] To achieve the above-mentioned purpose, the present invention provides the following technical solutions: a data analysis system based on an environmental detection cloud platform, comprising an environmental detection module, an environmental detection cloud platform and a data transmission module; the output end of the environmental detection module is connected to the input end of the environmental monitoring cloud platform, so as to obtain the environmental detection data of the monitoring site and send it to the environmental monitoring cloud platform; the environmental monitoring cloud platform is used to convert the environmental detection data collected by the target monitoring site and other monitoring sites into the environmental detection data of the reference monitoring site, judge the abnormal data of the distribution of the environmental detection data converted by the target monitoring site, analyze the concentration trend of the target monitoring site under the condition of the existence of abnormal data, determine the starting point of the abnormal data, and remove and fill the abnormal data; the data transmission module is used to connect the environmental detection module and the environmental detection cloud platform, and support data exchange and sharing between the environmental detection module and the environmental detection cloud platform.

[0005] The environmental detection cloud platform also includes a data storage unit, a correlation analysis unit, a conversion unit, a standardization unit, a first analysis unit, a second analysis unit and a filling unit; the correlation analysis unit is used for the correlation between the environmental detection data of all monitoring sites, and determines the reference monitoring site based on the correlation; the conversion unit is used to convert the environmental detection data collected by the target monitoring site and other monitoring sites into the first environmental detection data and the second environmental detection data of the reference monitoring site; the standardization unit is used to standardize the first environmental detection data and the second environmental detection data; the first analysis unit is used to analyze the data distribution of the first environmental detection data and the second environmental detection data; based on the data distribution, the abnormal data of the target monitoring site is judged; the second analysis unit, under the condition that it is determined that the target monitoring site has abnormal data, determines the starting point of the abnormal data according to the mean and variance analysis central trend of the environmental detection data of the target monitoring site; the filling unit is used to remove the abnormal data and fill the removed abnormal data through simulation data.

[0006] The standardized unit is standardized by the following formula: Z=(z-az) / sz, where Z is the data after standardization, z is the data before standardization, az is the average value of the data before standardization, and sz is the standard deviation of the data before standardization.

[0007] The conversion unit inputs the environmental detection data collected by the target monitoring site into the first neural network model through the first neural network model to obtain the first environmental detection data of the reference monitoring site; obtains the currently collected environmental detection data from other monitoring stations except the reference monitoring site, and inputs it into the second neural network model to obtain the second environmental detection data simulation value of the reference monitoring site; directly obtains the currently collected environmental detection data observation value from the reference monitoring site, and performs weighted summation on the second environmental detection data simulation value and observation value of the reference monitoring site to obtain the second environmental detection data of the reference monitoring site.

[0008] To achieve the above object, the present invention provides the following technical solution: a data analysis method based on an environmental detection cloud platform, comprising the following steps: S11, the environment detection module obtains the environment detection data of the monitoring site, and the environment detection module sends the security data to the environment detection cloud platform, and the monitoring site is divided into a target monitoring site and other monitoring sites; S12, the environmental monitoring cloud platform analyzes the correlation between the environmental monitoring data of all monitoring sites, and determines the reference monitoring site based on the correlation; S13, the environmental detection cloud platform converts the environmental detection data of the target monitoring site and the environmental detection data of other monitoring sites into the first environmental detection data and the second environmental detection data of the reference monitoring site, and analyzes the data distribution of the first environmental detection data and the second environmental detection data; based on the data distribution, it is determined whether there is abnormal data at the target monitoring site. If there is abnormal data, the starting point of the abnormal data at the target monitoring site is determined, the abnormal data is removed and filled, and the filled environmental detection data and the environmental detection data before the abnormality occurs are marked as safe data; if there is no abnormal data, the environmental detection data is marked as safe data.

[0009] There is a situation where environmental detection data is first marked as safe data and then detected as abnormal. This is because the abnormal data must occupy a certain proportion of the total data to detect anomalies. Therefore, the environmental detection data marking adopts the last-in-first principle, that is, the new judgment result overwrites the old judgment result.

[0010] In step S12, analyzing the correlation between the environmental detection data of all monitoring sites and determining the reference monitoring site based on the correlation further includes the following steps: Obtain the historical environmental detection data of all monitoring sites, select n consecutive environmental detection data from the environmental detection data of two monitoring sites, and calculate the correlation T between the environmental detection data of the two monitoring sites. ab , , where a i and b i are the observed values ​​of the environmental monitoring data of the two monitoring stations, and is the average value of the environmental monitoring data observations of the two monitoring stations, n is the sample size, and n is the sample number; Add all other monitoring sites to the set U and obtain the correlation T between the target monitoring site and the jth other monitoring site. j , j is the monitoring site number, remove the jth other monitoring site from the set U, and calculate the average value TU of the correlation between the remaining monitoring sites in the set U and the jth other monitoring site j , T j and TU j Add up to get the estimated scores of the jth other monitoring station; take the other monitoring station with the highest estimated scores as the reference monitoring station.

[0011] Analyze the correlation between monitoring sites. The higher the correlation with the reference monitoring site, the more the environmental detection data of the target monitoring site can reflect the environmental detection data of the reference monitoring site. The same is true for other monitoring sites. The purpose of analyzing the correlation is to obtain the data change trend of the reference monitoring site, rather than directly performing error analysis on the converted data. Since the reference monitoring site is also affected by natural or biological activity factors, the environmental detection data of the reference monitoring site, in addition to itself, is also converted using data from other monitoring sites. The positive and negative signs of the correlation only affect the direction. The closer the value is to 0, the weaker the correlation. For this reason, all correlations can be taken as absolute values.

[0012] In step S13, the step of converting the environmental detection data of the target monitoring site and the environmental detection data of other monitoring sites into the first environmental detection data and the second environmental detection data of the reference monitoring site further includes the following steps: S31, obtaining historical environmental detection data of the target monitoring site and the reference monitoring site with the same collection time, dividing the obtained data into a training set and a test set, taking the historical environmental detection data of m1 target monitoring sites in the training set as input, and the historical environmental detection data of m reference monitoring sites as output, and training the first neural network model of the reference monitoring site; wherein m1 is the model order, and m is the output dimension; verifying the first neural network model through the test set; obtaining the m1 environmental detection data currently collected by the target monitoring site, inputting them into the first neural network model, and obtaining the first environmental detection data of the reference monitoring site; during the operation of the target monitoring site, keep executing step S31; S32, obtain historical environmental detection data of the reference monitoring station and other monitoring stations except the reference monitoring station with the same collection time, divide the obtained data into a training set and a test set, use the historical environmental detection data of m2 monitoring stations except the reference monitoring station in the training set as input, and the historical environmental detection data of m reference monitoring stations as output, and train the second neural network model of the reference monitoring station; wherein m2 is the input feature, which is an integer multiple of the model order; verify the second neural network model through the test set; obtain the m2 environmental detection data currently collected from other monitoring stations except the reference monitoring station, input them into the second neural network model, and obtain the simulated value of the second environmental detection data of the reference monitoring station; directly obtain the observed values ​​of the m2 environmental detection data currently collected from the reference monitoring station, and perform weighted summation on the simulated value and observed value of the second environmental detection data of the reference monitoring station to obtain the second environmental detection data of the reference monitoring station; during the working process of other monitoring stations, keep executing step S32.

[0013] There must be errors in the environmental monitoring data of the reference monitoring sites obtained through transformation. For this reason, the errors are not analyzed, but the trends after transformation are analyzed after eliminating the influence of the mean and variance. If the trend obtained after the transformation of the target monitoring site is different from the trends obtained after the transformation of other monitoring sites, it means that the data of the target monitoring site is less representative, that is, the target monitoring site is interfered with.

[0014] Obtain the mean and variance of the first environment detection data and the mean and variance of the second environment detection data, and perform standardization on the first environment detection data and the second environment detection data. The standardization formula is as follows: Z=(z-az) / sz, where Z is the data after standardization, z is the data before standardization, az is the average value of the data before standardization, and sz is the standard deviation of the data before standardization; The standardized first environmental detection data and the second environmental detection data are fitted to obtain the first regression curve equation and the second regression curve equation, and the root mean square error between the first regression curve equation and the second regression curve equation is calculated. If the root mean square error does not exceed the threshold, it is judged that there is no abnormal data at the target monitoring site. If the root mean square error exceeds the threshold, it is judged that there is abnormal data at the target monitoring site.

[0015] In step S13, the step of determining the starting point of abnormal data of the target monitoring site further includes the following steps: S100, obtaining the time point t at which the target monitoring site is determined to have abnormal data; obtaining historical data when no abnormality occurs from the target monitoring site, and calculating the mean mean1 and standard deviation v1 based on the historical data when no abnormality occurs; obtaining historical data when abnormality occurs from the target monitoring site, and calculating the mean mean2 and standard deviation v2 based on the historical data when abnormality occurs; generating the normal interval [mean1-k×v1,mean1+k×v1] and the abnormal interval [mean2-k×v2,mean2+k×v2] based on mean1 and v1, mean2 and v2, respectively, where k is a coefficient; S200, starting from time point t, add the environmental detection data collected by the target monitoring station at time point t to the set W, add one environmental detection data collected by the target monitoring station in the set before all time points in W to the set W, calculate the mean value M and standard deviation V of the set W, generate the detection interval [Mk×V,M+k×V] according to M and V, calculate the overlap p1 and p2 between the detection interval and the normal interval and the abnormal interval, p1=c1 / ct1, p1=c2 / ct2, where c1 is the interval length of the detection interval repeated with the normal interval, c2 is the interval length of the detection interval repeated with the abnormal interval, ct1 and ct1 are the interval lengths of the normal interval and the abnormal interval; if p1 is greater than p2, obtain the next minimum value of the collection time corresponding to all environmental detection data in the set W to obtain the starting point of the abnormal data; if p1 is not greater than p2, return to step S200.

[0016] When there is less abnormal data, it is difficult to affect the overall data distribution. When abnormal data is detected, it means that a lot of abnormal data has been generated, so it is necessary to determine when the abnormal data began to be generated. Since the detection data of the target monitoring station comes from the target monitoring station itself, it is difficult to judge the authenticity and accuracy of the detection data. It is difficult to explain whether the target monitoring station has an abnormality based on the distribution and numerical changes of the target monitoring station data, because the data of the target monitoring station itself will also fluctuate; if it is determined that the target monitoring station has abnormal data, it will be more credible to judge the starting point of the abnormality based on the data distribution of the monitoring station.

[0017] In step S13, the removal and filling of abnormal data further includes the following steps: The historical data of the target monitoring site when no abnormality occurs are obtained, and the historical data of the target monitoring site when no abnormality occurs are fitted to obtain a regression curve equation, a data baseline is generated according to the regression curve equation, and the removed abnormal data is filled with the data of the data baseline.

[0018] Compared with the prior art, the beneficial effects of the present invention are: the environmental detection data is converted to the same reference monitoring site for analysis, and the data distribution at the reference monitoring site is analyzed, which is less affected by the model accuracy, thereby improving the accuracy of the environmental detection data analysis results; when it is determined that anomalies exist, the data distribution of the target monitoring site itself is analyzed, making the analysis results more convincing; and the abnormal data is filled in to improve the integrity of the data. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 The present invention is a structural schematic diagram of a data analysis system based on an environmental detection cloud platform. DETAILED DESCRIPTION

[0020] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0021] Example: Figure 1 As shown, the present invention provides a technical solution, a data analysis system based on an environment detection cloud platform, comprising an environment detection module, an environment detection cloud platform and a data transmission module; the output end of the environment detection module is connected to the input end of the environment monitoring cloud platform, so as to obtain the environment detection data of the monitoring site and send it to the environment monitoring cloud platform; the environment monitoring cloud platform is used to convert the environment detection data collected by the target monitoring site and other monitoring sites into the environment detection data of the reference monitoring site, judge the abnormal data of the distribution of the environmental detection data converted by the target monitoring site, analyze the concentration trend of the target monitoring site under the condition of the existence of abnormal data, determine the starting point of the abnormal data, and remove and fill the abnormal data; the data transmission module is used to connect the environment detection module and the environment detection cloud platform, and support data exchange and sharing between the environment detection module and the environment detection cloud platform.

[0022] The environmental detection cloud platform also includes a data storage unit, a correlation analysis unit, a conversion unit, a standardization unit, a first analysis unit, a second analysis unit and a filling unit; the correlation analysis unit is used for the correlation between the environmental detection data of all monitoring sites, and determines the reference monitoring site based on the correlation; the conversion unit is used to convert the environmental detection data collected by the target monitoring site and other monitoring sites into the first environmental detection data and the second environmental detection data of the reference monitoring site; the standardization unit is used to standardize the first environmental detection data and the second environmental detection data; the first analysis unit is used to analyze the data distribution of the first environmental detection data and the second environmental detection data; based on the data distribution, the abnormal data of the target monitoring site is judged; the second analysis unit, under the condition that it is determined that the target monitoring site has abnormal data, determines the starting point of the abnormal data according to the mean and variance analysis central trend of the environmental detection data of the target monitoring site; the filling unit is used to remove the abnormal data and fill the removed abnormal data through simulation data.

[0023] The standardized unit is standardized by the following formula: Z=(z-az) / sz, where Z is the data after standardization, z is the data before standardization, az is the average value of the data before standardization, and sz is the standard deviation of the data before standardization.

[0024] The conversion unit inputs the environmental detection data collected by the target monitoring site into the first neural network model through the first neural network model to obtain the first environmental detection data of the reference monitoring site; obtains the currently collected environmental detection data from other monitoring stations except the reference monitoring site, and inputs it into the second neural network model to obtain the second environmental detection data simulation value of the reference monitoring site; directly obtains the currently collected environmental detection data observation value from the reference monitoring site, and performs weighted summation on the second environmental detection data simulation value and observation value of the reference monitoring site to obtain the second environmental detection data of the reference monitoring site.

[0025] Embodiment: The present invention provides a technical solution, a data analysis method based on an environment detection cloud platform, comprising the following steps: S11, the environment detection module obtains the environment detection data of the monitoring site, and the environment detection module sends the security data to the environment detection cloud platform, and the monitoring site is divided into a target monitoring site and other monitoring sites; S12, the environmental monitoring cloud platform analyzes the correlation between the environmental monitoring data of all monitoring sites and determines the reference monitoring site based on the correlation: Obtain the historical environmental detection data of all monitoring sites, select n consecutive environmental detection data from the environmental detection data of two monitoring sites, and calculate the correlation T between the environmental detection data of the two monitoring sites. ab , , where a i and b i are the observed values ​​of the environmental monitoring data of the two monitoring stations, and is the average value of the environmental monitoring data observations of the two monitoring stations, n is the sample size, and n is the sample number; the absolute value of all correlations is taken; Add all other monitoring sites to the set U and obtain the correlation T between the target monitoring site and the jth other monitoring site. j , j is the monitoring site number, remove the jth other monitoring site from the set U, and calculate the average value TU of the correlation between the remaining monitoring sites in the set U and the jth other monitoring site j , T j and TU j Add up to get the estimated scores of the jth other monitoring station; take the other monitoring station with the highest estimated scores as the reference monitoring station.

[0026] S13, the environmental detection cloud platform converts the environmental detection data of the target monitoring site and the environmental detection data of other monitoring sites into the first environmental detection data and the second environmental detection data of the reference monitoring site: S31, obtaining historical environmental detection data of the target monitoring site and the reference monitoring site with the same collection time, dividing the obtained data into a training set and a test set, taking the historical environmental detection data of m1 target monitoring sites in the training set as input, and the historical environmental detection data of m reference monitoring sites as output, and training the first neural network model of the reference monitoring site; wherein m1 is the model order, and m is the output dimension; verifying the first neural network model through the test set; obtaining the m1 environmental detection data currently collected by the target monitoring site, inputting them into the first neural network model, and obtaining the first environmental detection data of the reference monitoring site; during the operation of the target monitoring site, keep executing step S31; S32, obtain historical environmental detection data of the reference monitoring station and other monitoring stations except the reference monitoring station with the same collection time, divide the obtained data into a training set and a test set, use the historical environmental detection data of m2 monitoring stations except the reference monitoring station in the training set as input, and the historical environmental detection data of m reference monitoring stations as output, and train the second neural network model of the reference monitoring station; wherein m2 is the input feature, which is an integer multiple of the model order; verify the second neural network model through the test set; obtain the m2 environmental detection data currently collected from other monitoring stations except the reference monitoring station, input them into the second neural network model, and obtain the simulated value of the second environmental detection data of the reference monitoring station; directly obtain the observed values ​​of the m2 environmental detection data currently collected from the reference monitoring station, and perform weighted summation on the simulated value and observed value of the second environmental detection data of the reference monitoring station to obtain the second environmental detection data of the reference monitoring station; during the working process of other monitoring stations, keep executing step S32.

[0027] For the first neural network model, the output dimension can be optionally set to 1, and the optimal model order m1 can be determined by detecting the effect of the first neural network model under different model orders. For example, the monitoring station collects environmental detection data once a minute. When the historical time is 8:00, the target monitoring station data at 7:41, 7:42, ..., 7:59, and 8:00 are used as input, and the data of the reference monitoring station at 8:00 is used as output to train the model. The effect of the first neural network model when the model order is 20 can be obtained. The target monitoring station data at 7:41, 7:42, ..., 7:59, and 8:00 are used as input, and the data of the reference monitoring station at 8:00 is used as output. The target monitoring site data at 7:59 and 8:00 are used as input, and the data of the reference monitoring site at 8:00 is used as output to train the model, and the effect of the first neural network model when the model order is 21 can be obtained, and the model order with the best effect is selected through testing; after obtaining the first neural network model, for example, the current time is 9:00, the first environmental detection data at 9:00 is obtained through the data before 9:00, and at 9:01, the input window is slid back one grid to obtain the first environmental detection data at 9:01, and repeated execution can obtain all the first environmental detection data.

[0028] For the second neural network model, data from multiple other monitoring sites can be selected as input, so m2 is an integer multiple of the model order.

[0029] Analyze the data distribution of the first environment detection data and the second environment detection data; determine whether there is abnormal data at the target monitoring site based on the data distribution: Obtain the mean and variance of the first environment detection data and the mean and variance of the second environment detection data, and perform standardization on the first environment detection data and the second environment detection data. The standardization formula is as follows: Z=(z-az) / sz, where Z is the data after standardization, z is the data before standardization, az is the average value of the data before standardization, and sz is the standard deviation of the data before standardization; The standardized first environmental detection data and the second environmental detection data are fitted to obtain the first regression curve equation and the second regression curve equation, and the root mean square error between the first regression curve equation and the second regression curve equation is calculated. If the root mean square error does not exceed the threshold, it is judged that there is no abnormal data at the target monitoring site. If the root mean square error exceeds the threshold, it is judged that there is abnormal data at the target monitoring site.

[0030] The threshold value can be determined based on the historical data of the target monitoring site, that is, in the historical data of the target monitoring site where abnormal data is clearly present, the root mean square error of the first regression curve equation and the second regression curve equation under the presence of abnormal data can be obtained; the threshold value is set based on the obtained root mean square error.

[0031] If there is abnormal data, determine the starting point of the abnormal data at the target monitoring site: S100, obtaining the time point t at which the target monitoring site is determined to have abnormal data; obtaining historical data when no abnormality occurs from the target monitoring site, and calculating the mean mean1 and standard deviation v1 based on the historical data when no abnormality occurs; obtaining historical data when abnormality occurs from the target monitoring site, and calculating the mean mean2 and standard deviation v2 based on the historical data when abnormality occurs; generating the normal interval [mean1-k×v1,mean1+k×v1] and the abnormal interval [mean2-k×v2,mean2+k×v2] based on mean1 and v1, mean2 and v2, respectively, where k is a coefficient; S200, starting from time point t, add the environmental detection data collected by the target monitoring station at time point t to the set W, add one environmental detection data collected by the target monitoring station in the set before all time points in W to the set W, calculate the mean value M and standard deviation V of the set W, generate the detection interval [Mk×V,M+k×V] according to M and V, calculate the overlap p1 and p2 between the detection interval and the normal interval and the abnormal interval, p1=c1 / ct1, p1=c2 / ct2, where c1 is the interval length of the detection interval repeated with the normal interval, c2 is the interval length of the detection interval repeated with the abnormal interval, ct1 and ct1 are the interval lengths of the normal interval and the abnormal interval; if p1 is greater than p2, obtain the next minimum value of the collection time corresponding to all environmental detection data in the set W to obtain the starting point of the abnormal data; if p1 is not greater than p2, return to step S200.

[0032] When the current time is 9:00, the mean and variance of the environmental detection data of the target monitoring site at 8:59 and 9:00 are calculated to obtain the detection interval. If the detection interval has a higher degree of overlap with the abnormal interval, the data at 8:58 is added to the set W and calculated again; if the detection interval has a higher degree of overlap with the normal interval, the starting point of the abnormal data is 9:00, 8:59 is the minimum value, and 9:00 is the second minimum value.

[0033] Remove and fill in the abnormal data, and mark the filled environmental detection data and the environmental detection data before the abnormality occurs as safe data: The historical data of the target monitoring site when no abnormality occurs are obtained, and the historical data of the target monitoring site when no abnormality occurs are fitted to obtain a regression curve equation, a data baseline is generated according to the regression curve equation, and the removed abnormal data is filled with the data of the data baseline.

[0034] If there is no abnormal data, the environmental detection data is marked as safe data.

[0035] It will be apparent to those skilled in the art that the invention is not limited to the details of the exemplary embodiments described above and that the invention can be implemented in other specific forms without departing from the spirit or essential features of the invention. Therefore, the embodiments should be considered exemplary and non-limiting in all respects, and the scope of the invention is defined by the appended claims rather than the foregoing description, and it is intended that all variations falling within the meaning and scope of the equivalent elements of the claims be included in the invention. Any reference numeral in a claim should not be considered as limiting the claim to which it relates.

Claims

1. A data analysis method based on an environmental detection cloud platform, characterized in that: The following steps are involved: S11, the environment detection module obtains the environment detection data of the monitoring site, and the environment detection module sends the security data to the environment detection cloud platform, and the monitoring site is divided into a target monitoring site and other monitoring sites; S12, the environmental monitoring cloud platform analyzes the correlation between the environmental monitoring data of all monitoring sites, and determines the reference monitoring site based on the correlation; S13, the environmental detection cloud platform converts the environmental detection data of the target monitoring site and the environmental detection data of other monitoring sites into the first environmental detection data and the second environmental detection data of the reference monitoring site, and analyzes the data distribution of the first environmental detection data and the second environmental detection data; based on the data distribution, it is determined whether there is abnormal data at the target monitoring site, and if there is abnormal data, the starting point of the abnormal data at the target monitoring site is determined, the abnormal data is removed and filled, and the filled environmental detection data and the environmental detection data before the abnormality occurs are marked as safe data; If there is no abnormal data, the environmental detection data is marked as safe data.

2. According to the data analysis method based on the environment detection cloud platform of claim 1, it is characterized in that: In step S12, analyzing the correlation between the environmental detection data of all monitoring sites and determining the reference monitoring site based on the correlation further includes the following steps: Obtain the historical environmental detection data of all monitoring sites, select n consecutive environmental detection data from the environmental detection data of two monitoring sites, and calculate the correlation T between the environmental detection data of the two monitoring sites. ab , , where a i and b i are the observed values ​​of the environmental monitoring data of the two monitoring stations, and is the average value of the environmental monitoring data observations of the two monitoring stations, n is the sample size, and n is the sample number; Add all other monitoring sites to the set U and obtain the correlation T between the target monitoring site and the jth other monitoring site. j , j is the monitoring site number, remove the jth other monitoring site from the set U, and calculate the average value TU of the correlation between the remaining monitoring sites in the set U and the jth other monitoring site j , T j and TU j Add up to get the estimated scores of the jth other monitoring station; take the other monitoring station with the highest estimated scores as the reference monitoring station.

3. A data analysis method based on an environmental detection cloud platform according to claim 2, characterized in that: In step S13, the step of converting the environmental detection data of the target monitoring site and the environmental detection data of other monitoring sites into the first environmental detection data and the second environmental detection data of the reference monitoring site further includes the following steps: S31, obtaining historical environmental detection data of the target monitoring site and the reference monitoring site with the same collection time, dividing the obtained data into a training set and a test set, taking the historical environmental detection data of m1 target monitoring sites in the training set as input, and the historical environmental detection data of m reference monitoring sites as output, and training the first neural network model of the reference monitoring site; wherein m1 is the model order, and m is the output dimension; verifying the first neural network model through the test set; obtaining the m1 environmental detection data currently collected by the target monitoring site, inputting them into the first neural network model, and obtaining the first environmental detection data of the reference monitoring site; during the operation of the target monitoring site, keep executing step S31; S32, obtain historical environmental detection data of the reference monitoring site and other monitoring sites except the reference monitoring site with the same collection time, divide the obtained data into a training set and a test set, use the historical environmental detection data of m2 other monitoring sites except the reference monitoring site in the training set as input, and the historical environmental detection data of m reference monitoring sites as output, and train the second neural network model of the reference monitoring site; wherein m2 is the input feature, which is an integer multiple of the model order; verify the second neural network model through the test set; obtain the m2 environmental detection data currently collected from other monitoring sites except the reference monitoring site, input them into the second neural network model, and obtain the simulated value of the second environmental detection data of the reference monitoring site; directly obtain the observed values ​​of the m2 environmental detection data currently collected from the reference monitoring site, and perform weighted summation on the simulated value and observed value of the second environmental detection data of the reference monitoring site to obtain the second environmental detection data of the reference monitoring site; during the working process of other monitoring stations, keep executing step S32.

4. A data analysis method based on an environmental detection cloud platform according to claim 3, characterized in that: In step S13, the analyzing the data distribution of the first environment detection data and the second environment detection data further includes the following steps: Obtain the mean and variance of the first environment detection data and the mean and variance of the second environment detection data, and perform standardization on the first environment detection data and the second environment detection data. The standardization formula is as follows: Z=(z-az) / sz, where Z is the data after standardization, z is the data before standardization, az is the average value of the data before standardization, and sz is the standard deviation of the data before standardization; The standardized first environmental detection data and the second environmental detection data are fitted to obtain the first regression curve equation and the second regression curve equation, and the root mean square error between the first regression curve equation and the second regression curve equation is calculated. If the root mean square error does not exceed the threshold, it is judged that there is no abnormal data at the target monitoring site. If the root mean square error exceeds the threshold, it is judged that there is abnormal data at the target monitoring site.

5. The data analysis method based on the environment detection cloud platform according to claim 4 is characterized in that: In step S13, the step of determining the starting point of abnormal data of the target monitoring site further includes the following steps: S100, obtaining the time point t at which the target monitoring site is determined to have abnormal data; obtaining historical data when no abnormality occurs from the target monitoring site, and calculating the mean mean1 and standard deviation v1 based on the historical data when no abnormality occurs; obtaining historical data when abnormality occurs from the target monitoring site, and calculating the mean mean2 and standard deviation v2 based on the historical data when abnormality occurs; generating the normal interval [mean1-k×v1,mean1+k×v1] and the abnormal interval [mean2-k×v2,mean2+k×v2] based on mean1 and v1, mean2 and v2, respectively, where k is a coefficient; S200, starting from time point t, add the environmental detection data collected by the target monitoring station at time point t to the set W, add one environmental detection data collected by the target monitoring station in the set before all time points in W to the set W, calculate the mean value M and standard deviation V of the set W, generate the detection interval [Mk×V,M+k×V] according to M and V, calculate the overlap p1 and p2 between the detection interval and the normal interval and the abnormal interval, p1=c1 / ct1, p1=c2 / ct2, where c1 is the interval length of the detection interval repeated with the normal interval, c2 is the interval length of the detection interval repeated with the abnormal interval, ct1 and ct1 are the interval lengths of the normal interval and the abnormal interval; if p1 is greater than p2, obtain the next minimum value of the collection time corresponding to all environmental detection data in the set W to obtain the starting point of the abnormal data; if p1 is not greater than p2, return to step S200.

6. A data analysis method based on an environmental detection cloud platform according to claim 4, characterized in that: In step S13, the removal and filling of abnormal data further includes the following steps: The historical data of the target monitoring site when no abnormality occurs is obtained, and the historical data of the target monitoring site when no abnormality occurs is fitted to obtain a regression curve equation, a data baseline is generated according to the regression curve equation, and the data of the data baseline is used to fill in the removed abnormal data.

7. A data analysis system based on an environmental detection cloud platform, characterized in that: It includes an environment detection module, an environment detection cloud platform and a data transmission module; the output end of the environment detection module is connected to the input end of the environment monitoring cloud platform, and is used to obtain the environment detection data of the monitoring site and send it to the environment monitoring cloud platform; the environment monitoring cloud platform is used to convert the environment detection data collected by the target monitoring site and other monitoring sites into the environment detection data of the reference monitoring site, judge the abnormal data of the distribution of the environmental detection data converted by the target monitoring site, analyze the concentration trend of the target monitoring site under the condition of the existence of abnormal data, determine the starting point of the abnormal data, and remove and fill the abnormal data; The data transmission module is used to connect the environment detection module and the environment detection cloud platform, and supports data exchange and sharing between the environment detection module and the environment detection cloud platform.

8. The data analysis system based on the environment detection cloud platform according to claim 7 is characterized in that: The environmental detection cloud platform also includes a data storage unit, a correlation analysis unit, a conversion unit, a standardization unit, a first analysis unit, a second analysis unit and a filling unit; the correlation analysis unit is used for the correlation between the environmental detection data of all monitoring sites, and determines the reference monitoring site based on the correlation; the conversion unit is used to convert the environmental detection data collected by the target monitoring site and other monitoring sites into the first environmental detection data and the second environmental detection data of the environmental detection data of the reference monitoring site; the standardization unit is used to standardize the first environmental detection data and the second environmental detection data; The first analysis unit is used to analyze the data distribution of the first environment detection data and the second environment detection data; and judge the abnormal data of the target monitoring site based on the data distribution; The second analysis unit determines the starting point of the abnormal data based on the mean and variance analysis central trend of the environmental detection data of the target monitoring site under the condition that the target monitoring site has abnormal data; The filling unit is used to remove abnormal data and fill the removed abnormal data with simulated data.

9. The data analysis system based on the environment detection cloud platform according to claim 7 is characterized in that: The standardized unit is standardized by the following formula: Z=(z-az) / sz, where Z is the data after standardization, z is the data before standardization, az is the average value of the data before standardization, and sz is the standard deviation of the data before standardization.

10. The data analysis system based on the environment detection cloud platform according to claim 7, characterized in that: The conversion unit inputs the environmental detection data collected by the target monitoring site into the first neural network model through the first neural network model to obtain the first environmental detection data of the reference monitoring site; obtains the currently collected environmental detection data from other monitoring stations except the reference monitoring site, and inputs it into the second neural network model to obtain the second environmental detection data simulation value of the reference monitoring site; directly obtains the currently collected environmental detection data observation value from the reference monitoring site, and performs weighted summation on the second environmental detection data simulation value and observation value of the reference monitoring site to obtain the second environmental detection data of the reference monitoring site.

Citation Information

Patent Citations

  • Pollution visual data processing system and method applying cloud computing technology

    CN119202063A

  • Water quality monitoring data analysis method and apparatus, device, and storage medium

    WO2022160682A1