An online environment data collection and analysis system and method

Abstract

An online environmental data acquisition and analysis system and method, relating to the field of environmental monitoring data processing technology, includes: a data acquisition module, a first spectral analysis module, an evaluation module, and an analysis model optimization module. The data acquisition module automatically identifies and parses the data, while the first spectral analysis module, evaluation module, and second spectral analysis module analyze the data and output the analysis results. This solves the problems of difficulty in simultaneously processing data from multiple different GCMS devices due to their different data formats, and the need for manual review of detected substances in existing detection methods, which consumes significant manpower when dealing with large amounts of data.

Description

Technical Field

[0001] This invention relates to the field of environmental monitoring data processing technology, specifically to an online environmental data acquisition and analysis system and method. Background Technology

[0002] Currently, most traditional environmental monitoring companies use gas chromatography-mass spectrometry (GC-MS) to detect VOCs in ambient air. Ambient air is collected by a sampling system and then enters a concentration system. Under ultra-low temperature conditions, volatile organic compounds in the atmosphere are frozen and captured in an empty capillary trap column; then, rapid heating and desorption allow the compounds to enter the GCMS analysis system. After separation by the chromatographic column, they are detected by FID and MS detectors. Real-time acquisition and analysis of VOCs and characteristic pollutant emission data, along with a data validity review mechanism to verify the monitoring data, provide accurate and valid monitoring data for environmental monitoring departments, enabling real-time monitoring of pollution emissions and providing data support and decision support for pollution reduction and total emission control. However, the data acquisition and analysis process using different GCMS devices is challenging because different devices have different data formats, making it difficult to process data from multiple different devices simultaneously. Furthermore, the current detection method requires manual verification of detected substances, which consumes significant manpower when dealing with large volumes of data. Summary of the Invention

[0003] This invention provides an online environmental data acquisition and analysis system and method. By identifying, parsing and analyzing the data output by the devices, the system can obtain analysis results and process data output by multiple different devices simultaneously, thereby reducing labor costs.

[0004] An online environmental data acquisition and analysis system includes: a data acquisition module, a first spectral analysis module, an evaluation module, a second spectral analysis module, and an analysis model optimization module;

[0005] The data acquisition module is used to parse the input data to obtain spectral data;

[0006] The first spectral analysis module is used to analyze spectral data and obtain analysis results. The evaluation module evaluates the substances corresponding to the spectral data based on the analysis results. If the evaluation score is higher than a preset value, the second spectral analysis module is not triggered, and the obtained analysis results are directly output. If the evaluation score is lower than the preset value, the second spectral analysis module is triggered to verify the analysis results of the first spectral analysis module. If the verification result is still lower than the preset value, the current result is marked and output. If the verification result is higher than the preset value, the analysis results are directly output.

[0007] The analysis model optimization module is used to perform spectral data feature mining on the output analysis results, obtain the feature data of the spectral data, and verify the obtained data. After the verification is passed, the data is input into the first spectral analysis module and the second spectral analysis module to optimize the analysis model.

[0008] Furthermore, the data acquisition module includes a data format recognition unit and a parsing unit. The data format recognition unit is used to recognize the format of the input data, and the parsing unit retrieves the corresponding parsing protocol based on the recognition result to parse the input data and obtain spectral data.

[0009] Furthermore, the first spectral analysis module includes a first analysis model and a data caching unit. The data caching unit is used to synchronously cache the input spectral data, and the first analysis model is used to analyze the cached spectral data to obtain analysis results.

[0010] Furthermore, the second spectral analysis module includes a second analysis model and a data extraction unit. The data extraction unit is used to extract the corresponding spectral data from the data cache unit according to the time series and input it into the second analysis model for analysis to obtain the analysis results.

[0011] Furthermore, the analysis model optimization module includes a spectral data feature mining unit and a data verification unit. The spectral data feature mining unit is used to perform spectral data feature mining on the analysis results obtained from the first analysis model and the second analysis model to obtain the feature data of the spectral data. The data verification unit verifies the obtained data, and after successful verification, it is input into the first analysis model and the second analysis model to optimize the analysis model.

[0012] Furthermore, both the first analysis model and the second analysis model are trained neural network models.

[0013] Secondly, embodiments of the present invention provide an online environmental data acquisition and analysis method, comprising the following steps:

[0014] S1, Data Analysis: After identifying the input data, the corresponding analysis protocol is retrieved to analyze the data and obtain the spectral data of the substance.

[0015] S2, the first analysis, analyzes the spectral data of the substance to obtain the analysis results;

[0016] S3, Substance Evaluation: Evaluate the substances corresponding to the spectral data based on the analysis results;

[0017] S4, Second analysis: If the evaluated score is higher than the preset value, the analysis result is output directly. If the evaluated score is lower than the preset value, the result of the first analysis is verified. If the verification result is still lower than the preset value, the current result is marked and output. If the verification result is higher than the preset value, the analysis result is output directly.

[0018] S5, Analysis Model Optimization: Perform spectral data feature mining on the output analysis results to obtain the feature data of the spectral data, and verify the obtained data. After successful verification, use the feature data of the spectral data as samples to optimize the analysis model.

[0019] The beneficial effects of the above-described technical solutions provided in the embodiments of the present invention include at least the following:

[0020] This invention automatically identifies and parses the data output from GCMS devices through a data acquisition module. A first spectral analysis module, an evaluation module, and a second spectral analysis module analyze the data and output the results. An analysis model optimization module performs feature mining on the spectral data based on the output results, acquiring feature data from the spectral data and verifying the acquired data. After successful verification, the feature data is used as a sample to optimize the analysis model, improving the accuracy of data analysis by both the first and second spectral analysis modules. This invention solves the problems of difficulty in simultaneously processing data from multiple different GCMS devices due to their different data formats, and the need for manual review of detected substances in existing detection methods, which consumes significant manpower when dealing with large amounts of data.

[0021] Other features and advantages of the invention will be set forth in the following description, and will be apparent in part from the description, or may be learned by practicing the invention. The objects and other advantages of the invention may be realized and obtained by means of the structures particularly pointed out in the written description, claims, and drawings.

[0022] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description

[0023] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings:

[0024] Figure 1 This is a schematic diagram of the structure of the online environmental data acquisition and analysis system disclosed in an embodiment of the present invention;

[0025] Figure 2This is a schematic diagram of the structure of the online environmental data acquisition and analysis method disclosed in an embodiment of the present invention.

[0026] Figure label:

[0027] 1. Data acquisition module; 11. Data format recognition unit; 12. Parsing unit; 2. First spectrum analysis module; 21. First analysis model; 22. Data caching unit; 3. Evaluation module; 4. Second spectrum analysis module; 41. Second analysis model; 42. Data extraction unit; 5. Analysis model optimization module; 51. Spectrum data feature mining unit; 52. Data verification unit. Detailed Implementation

[0028] Exemplary embodiments of the present disclosure will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.

[0029] Example 1

[0030] like Figure 1 As shown, this embodiment of the invention provides an online environmental data acquisition and analysis system, including: a data acquisition module 1, a first spectrum analysis module 2, an evaluation module 3, a second spectrum analysis module 4, and an analysis model optimization module 5;

[0031] The data acquisition module 1 is used to parse the input data to obtain spectral data. The data acquisition module 1 includes a data format recognition unit 11 and a parsing unit 12. The data format recognition unit 11 is used to recognize the format of the input data, and the parsing unit 12 retrieves the corresponding parsing protocol according to the recognition result to parse the input data and obtain spectral data.

[0032] Specifically, the parsing unit 12 has a variety of parsing protocol libraries corresponding to GCMS devices. The data format recognition unit 11 recognizes the input data format. After recognizing the data format, the parsing unit 12 traverses the preset parsing protocol library according to the recognized data format to retrieve the corresponding parsing protocol to parse the data. The data acquisition module 1 is used to parse the data output by multiple GCMS devices to increase compatibility.

[0033] The first spectral analysis module 2 is used to analyze spectral data and obtain analysis results. The evaluation module 3 evaluates the substances corresponding to the spectral data based on the analysis results. If the evaluation score is higher than the preset value, the second spectral analysis module 4 is not triggered, and the obtained analysis results are directly output. If the evaluation score is lower than the preset value, the second spectral analysis module 4 is triggered to verify the analysis results of the first spectral analysis module 2. If the verification result is still lower than the preset value, the current result is marked and output. If the verification result is higher than the preset value, the analysis results are directly output. The first spectral analysis module 2 includes a first analysis model 21 and a data caching unit 22. The data caching unit 22 is used to synchronously cache the input spectral data. The first analysis model 21 is used to analyze the cached spectral data and obtain analysis results. The second spectral analysis module 4 includes a second analysis model 41 and a data extraction unit 42. The data extraction unit 42 is used to extract the corresponding spectral data from the data caching unit 22 according to the time series and input it into the second analysis model 41 for analysis to obtain analysis results.

[0034] Specifically, after the input data is parsed, the corresponding spectral data is output. The data caching unit 22 caches the spectral data according to the time series. The first analysis model 21 performs qualitative and quantitative analysis on the substance based on the cached spectral data to obtain the analysis results. The preset value in the evaluation module 3 is the standard value for judging the properties of the detected substance. When it is higher than the preset value, the detected substance has a low degree of environmental impact. When it is lower than the preset value, the detected substance has a high degree of environmental impact. When the score of the analysis result obtained by the first analysis model 21 after the first evaluation by the evaluation module 3 is higher than the preset value, the obtained analysis result is directly output. When the score of the analysis result obtained by the first analysis model 21 after the evaluation by the evaluation module 3 is lower than the preset value, the data extraction unit 42 extracts the corresponding spectral data from the data caching unit 22 according to the time series and inputs it into the second analysis model 41 for analysis to obtain the analysis results. The evaluation module 3 then performs a second evaluation. If the verification result is still lower than the preset value, the current result is marked and output. If the verification result is higher than the preset value, the analysis result is directly output. The marked analysis results are further processed by the reviewers.

[0035] The analysis model optimization module 5 is used to perform spectral data feature mining on the output analysis results, obtain the feature data of the spectral data, and verify the obtained data. After the verification is passed, the data is input into the first spectral analysis module 2 and the second spectral analysis module 4 to optimize the analysis model.

[0036] Specifically, the analysis model optimization module 5 includes a spectrum data feature mining unit 51 and a data verification unit 52. The spectrum data feature mining unit 51 is used to perform spectrum data feature mining on the analysis results obtained from the first analysis model 21 and the second analysis model 41 to obtain the feature data of the spectrum data. The data verification unit 52 verifies the obtained data. After the verification is passed, the data is input into the first analysis model 21 and the second analysis model 41 to optimize the analysis model.

[0037] This invention automatically identifies and parses the data output by the GCMS device through a data acquisition module 1. A first spectral analysis module 2, an evaluation module 3, and a second spectral analysis module 4 analyze the data and output the analysis results. An analysis model optimization module 5 performs spectral data feature mining based on the output analysis results, obtains the feature data of the spectral data, and verifies the obtained data. After successful verification, the feature data of the spectral data is used as a sample to optimize the analysis model, improving the accuracy of data analysis by the first spectral analysis module 2 and the second spectral analysis module 4. This invention solves the problems of difficulty in simultaneously processing data from multiple different GCMS devices due to their different data formats, and the need for manual review of detected substances in existing detection methods, which consumes significant manpower when dealing with large amounts of data.

[0038] Example 2

[0039] This invention also discloses an online environmental data acquisition and analysis method, such as... Figure 2 This includes the following steps:

[0040] S1, Data Analysis: After identifying the input data, the corresponding analysis protocol is retrieved to analyze the data and obtain the spectral data of the substance.

[0041] S2, the first analysis, analyzes the spectral data of the substance to obtain the analysis results;

[0042] S3, Substance Evaluation: Evaluate the substances corresponding to the spectral data based on the analysis results;

[0043] S4, Second analysis: If the evaluated score is higher than the preset value, the analysis result is output directly. If the evaluated score is lower than the preset value, the result of the first analysis is verified. If the verification result is still lower than the preset value, the current result is marked and output. If the verification result is higher than the preset value, the analysis result is output directly.

[0044] S5, Analysis Model Optimization: Perform spectral data feature mining on the output analysis results to obtain the feature data of the spectral data, and verify the obtained data. After successful verification, use the feature data of the spectral data as samples to optimize the analysis model.

[0045] This invention automatically identifies and parses the data output by the GCMS device through a data acquisition module 1. A first spectral analysis module 2, an evaluation module 3, and a second spectral analysis module 4 analyze the data and output the analysis results. An analysis model optimization module 5 performs spectral data feature mining based on the output analysis results, obtains the feature data of the spectral data, and verifies the obtained data. After successful verification, the feature data of the spectral data is used as a sample to optimize the analysis model, improving the accuracy of data analysis by the first spectral analysis module 2 and the second spectral analysis module 4. This invention solves the problems of difficulty in simultaneously processing data from multiple different GCMS devices due to their different data formats, and the need for manual review of detected substances in existing detection methods, which consumes significant manpower when dealing with large amounts of data.

[0046] It should be understood that the specific order or hierarchy of steps in the disclosed process is an example of an exemplary method. Based on design preferences, it should be understood that the specific order or hierarchy of steps in the process may be rearranged without departing from the scope of this disclosure. The appended method claims provide elements of various steps in an exemplary order and are not intended to limit the specific order or hierarchy described.

[0047] In the detailed description above, various features are combined together in a single embodiment to simplify this disclosure. This approach to disclosure should not be construed as reflecting an intention that embodiments of the claimed subject matter require more features than are explicitly stated in each claim. Rather, as reflected in the appended claims, the invention is presented with fewer features than all of the features in a single disclosed embodiment. Therefore, the appended claims are hereby explicitly incorporated into the detailed description, with each claim representing a separate preferred embodiment of the invention.

[0048] Those skilled in the art will also understand that the various illustrative logic blocks, modules, circuits, and algorithm steps described in conjunction with the embodiments herein can be implemented as electronic hardware, computer software, or a combination thereof. To clearly illustrate the interchangeability between hardware and software, the various illustrative components, blocks, modules, circuits, and steps described above are generally described in terms of their functionality. Whether such functionality is implemented as hardware or software depends on the specific application and the design constraints imposed on the overall system. Those skilled in the art can implement the described functionality in alternative ways for each specific application; however, such implementation decisions should not be construed as departing from the scope of this disclosure.

[0049] The steps of the methods or algorithms described in conjunction with the embodiments herein can be directly embodied in hardware, software modules executed by a processor, or a combination thereof. The software modules can reside in RAM memory, flash memory, ROM memory, EPROM memory, EEPROM memory, registers, hard disks, removable disks, CD-ROMs, or any other form of storage medium well known in the art. An exemplary storage medium is connected to the processor, enabling the processor to read information from and write information to the storage medium. Of course, the storage medium can also be a component of the processor. The processor and storage medium can reside in an ASIC. The ASIC can reside in a user terminal. Alternatively, the processor and storage medium can exist as discrete components in the user terminal.

[0050] For software implementation, the techniques described in this application can be implemented using modules (e.g., procedures, functions, etc.) that perform the functions described in this application. This software code can be stored in memory units and executed by a processor. The memory units can be implemented within the processor or outside the processor; in the latter case, they are communicatively coupled to the processor via various means, as is well known in the art.

[0051] The foregoing description includes examples of one or more embodiments. It is certainly impossible to describe all possible combinations of components or methods in order to describe the above embodiments, but those skilled in the art will recognize that further combinations and arrangements of the various embodiments are possible. Therefore, the embodiments described herein are intended to cover all such changes, modifications, and variations that fall within the scope of the appended claims. Furthermore, the term "comprising" as used in the specification or claims is interpreted in a manner similar to the term "including," as interpreted when used as a conjunction in the claims. Additionally, the use of any term "or" in the specification of the claims is intended to mean "non-exclusive or."

Claims

1. An online environment data collection and analysis system, characterized by, include: The system includes a data acquisition module, a first spectrum analysis module, an evaluation module, a second spectrum analysis module, and an analysis model optimization module. The data acquisition module is used to parse the input data to obtain spectral data. The data acquisition module includes a data format recognition unit and a parsing unit. The data format recognition unit is used to recognize the format of the input data. The parsing unit retrieves the corresponding parsing protocol according to the recognition result to parse the input data and obtain spectral data. The parsing unit has multiple parsing protocol libraries corresponding to GCMS devices. The data format recognition unit recognizes the format of the input data. After recognizing the data format, the parsing unit traverses the preset parsing protocol library according to the recognized data format to retrieve the corresponding parsing protocol to parse the data. The data acquisition module is used to parse the data output from multiple GCMS devices to increase compatibility. The first spectral analysis module is used to analyze spectral data and obtain analysis results. The first spectral analysis module includes a first analysis model and a data caching unit. The data caching unit caches the spectral data according to a time series. The first analysis model is used to perform qualitative and quantitative analysis on substances based on the cached spectral data and obtain analysis results. The evaluation module evaluates the substances corresponding to the spectral data based on the analysis results. If the evaluation score is higher than the preset value, the second spectral analysis module is not triggered, and the obtained analysis result is directly output. If the evaluation score is lower than the preset value, the second spectral analysis module is triggered to verify the analysis result of the first spectral analysis module. If the verification result is still lower than the preset value, the current result is marked and output. If the verification result is higher than the preset value, the analysis result is directly output. The preset value is the standard value for judging the properties of the detected substance. When it is higher than the preset value, the detected substance has a low degree of environmental impact. When it is lower than the preset value, the detected substance has a high degree of environmental impact. The second spectral analysis module includes a second analysis model and a data extraction unit. The data extraction unit is used to extract the corresponding spectral data from the data cache unit according to the time series and input it into the second analysis model for analysis to obtain the analysis results. The analysis model optimization module is used to perform spectral data feature mining on the output analysis results, obtain the feature data of the spectral data, and verify the obtained data. After the verification is passed, the data is input into the first spectral analysis module and the second spectral analysis module to optimize the analysis model.

2. An online environment data collection and analysis system as claimed in claim 1, wherein, The analysis model optimization module includes a spectral data feature mining unit and a data verification unit. The spectral data feature mining unit is used to perform spectral data feature mining on the analysis results obtained from the first analysis model and the second analysis model to obtain the feature data of the spectral data. The data verification unit verifies the obtained data, and after the verification is passed, it is input into the first analysis model and the second analysis model to optimize the analysis model.

3. The online environmental data acquisition and analysis system as described in claim 1, characterized in that, Both the first analysis model and the second analysis model are trained neural network models.

4. An online environmental data acquisition and analysis method, using the online environmental data acquisition and analysis system as described in claim 1, characterized in that, Includes the following steps: S1, Data Analysis: After identifying the input data, the corresponding analysis protocol is retrieved to analyze the data and obtain the spectral data of the substance. S2, the first analysis, involves qualitative and quantitative analysis of the spectral data of the substance to obtain the analytical results; S3, Substance Evaluation: Evaluate the substances corresponding to the spectral data based on the analysis results; S4, Second analysis: If the evaluated score is higher than the preset value, the analysis result is output directly. If the evaluated score is lower than the preset value, the result of the first analysis is verified. If the verification result is still lower than the preset value, the current result is marked and output. If the verification result is higher than the preset value, the analysis result is output directly. S5, Analysis Model Optimization: Perform spectral data feature mining on the output analysis results to obtain the feature data of the spectral data, and verify the obtained data. After successful verification, use the feature data of the spectral data as samples to optimize the analysis model.

Images