Regional real-time carbon dioxide monitoring instrument data processing system

Abstract

A regionalized real-time carbon dioxide monitoring instrument data processing system provided by the present invention comprises a monitoring instrument, the monitoring instrument is connected to a server, and the monitoring instrument comprises a system UI interface, data import and data analysis processing visualization. The monitoring instrument is provided with a data import column and a partition visualization display column which are arranged on the side edge and used for data import and data analysis processing visualization, mass monitoring data obtained by the high-time-resolution monitoring instrument can be efficiently and accurately processed, the problems of low time efficiency, high error rate and the like in the manual processing process are solved, and the data processing efficiency is improved. According to the real-time carbon dioxide monitoring instrument data analysis and processing system based on the R language shaft package, real-time maintenance can be achieved on the basis of an R language open source package library, and the problem that the cost of developing a specific data platform system is high is solved. The method and the device are used for overcoming the defect that in the prior art, an R language shiny package is not practiced in a regional carbon dioxide monitoring visualization system.

Description

Technical Field

[0001] The invention relates to the technical field of regional real-time high-resolution carbon dioxide monitoring data processing, and in particular to a regional real-time carbon dioxide monitoring instrument data processing system. Background Art

[0003] At present, many parks and enterprises are faced with the problems of cumbersome and delayed real-time carbon dioxide monitoring data processing and high cost of developing data platform systems. Traditional data processing methods often rely on complex manual data collection, cleaning and analysis processes, resulting in inefficiency and waste of resources. At the same time, establishing and maintaining a complete data platform system requires a lot of manpower, material and financial resources, which is a huge burden for many small parks or enterprises. In response to these challenges and to meet this demand, the present invention proposes a regional real-time carbon dioxide monitoring instrument data analysis and processing system based on R language and its powerful shiny package. Summary of the invention

[0004] In view of this, in order to solve the problems of cumbersome and delayed real-time carbon dioxide monitoring data processing and high cost of developing a data platform system in the technical background, the present invention proposes a regionalized real-time carbon dioxide monitoring instrument data processing system. The specific technical solution is as follows:

[0005] A regionalized real-time carbon dioxide monitoring instrument data processing system includes a monitoring instrument, the monitoring instrument is connected to a server, the monitoring instrument includes a system UI interface, data import, data analysis and processing visualization, and the monitoring instrument is provided with a data import bar and a partition visualization display bar on the side for data import and data analysis and processing visualization;

[0006] The data analysis and processing visualization mainly includes the cleaning of CO2 monitoring concentration data and the calculation of the mean of each time scale, the time series change characteristics of CO2 monitoring concentration data, the concentration change characteristics of specific time-varying patterns, meteorological data analysis, and the spatial distribution characteristics of multiple monitoring points; the data import, i.e. user operation, includes: the user operates in the system UI interface to upload and submit the recorded data of each parameter on the monitoring instrument;

[0007] After the user has uploaded the data, a prompt will be displayed indicating that the data has been successfully uploaded. When the user clicks on the time series change characteristics, the specific time-varying pattern concentration change characteristics, the meteorological data analysis, and the spatial distribution characteristics, the system UI interface will switch to the corresponding interface. The server receives the feedback information and calls the corresponding package library and function (tidyverse, ggplot2, idw(), etc.) to visualize the data in time series according to the format of the data and the module selected by the user, and displays the visualization results of the corresponding data period, so as to realize the automatic data processing and display of the visualization analysis results.

[0008] In one embodiment of the present invention, the data analysis and processing visualization uses the R language data reading package library (readxl(), openxlsx(), read.csv(), etc.) to import the data file, and calculates the number of missing values ​​in each column through the column missing value statistical function (for example: colSums(is.na(data)) function). The function returns a vector in which each element represents the number of missing values ​​in the corresponding column. The number of missing values ​​in each column is divided by the total number of rows and multiplied by 100 to obtain the missing value ratio of each column. The missing value deletion function (for example: na.omit() function) is used to delete the rows containing any missing values, thereby realizing the missing value screening processing of the original data.

[0009] Missing ratio = missing values / total number of rows × 100.

[0010] In one embodiment of the present invention, the data analysis and processing visualization converts the time column in the original data into the standard time format in R through the time date conversion function (for example, POSIXct() and other functions), and then uses the grouping aggregation function (such as aggregate() or the function in the dplyr package) to aggregate the data according to the required time scale, such as calculating the average or sum by hour, and performing daily, weekly and monthly average processing for subsequent analysis or visualization. In this way, the original data can be converted from fine-grained second or minute levels to a higher-level time scale data set to meet different analysis requirements, and finally displayed in the system UI interface through the interactive data table function (for example, DT::datatable(), etc.);

[0011] In one embodiment of the present invention, the visualization in the data analysis and processing visualization includes calling the corresponding data processing and visualization toolkit (e.g., tidyverse, ggplot2, etc.) to perform time series visualization processing on the data according to the format of the data and the module selected by the user, mainly including the time series change characteristics of CO2 monitoring concentration data (hourly time series change characteristics, daily change time series characteristics), specific time-varying mode concentration change characteristics (hourly change, weekly change, monthly change); calling the correlation visualization function (e.g., corrplot() function) and the wind rose visualization function (e.g., windRose() function) to perform meteorological data analysis (wind speed and direction characteristics, wind direction pollution characteristics, correlation characteristic analysis); calling the spatial data / object processing package library in R language (e.g., sf and sp) to process spatial data, interpret spatial geographic information, and convert the data into a spatial data frame (all_points_tro). Then, the longitude and latitude are set by the coordinate extraction function (e.g., sp::coordinates()), and the grid is generated using the expanded grid function (expand.grid()). The inverse distance interpolation function (e.g., idw() function) is used to perform inverse distance weighted interpolation to generate the interpolation result (idw_output), which includes the longitude, latitude, and result after interpolation. Finally, the ggplot2 library is used to plot the IDW interpolation results of the study area, thereby realizing real-time automatic visualization and display of the spatial distribution characteristics of multiple monitoring points.

[0012] The above technical solution has the following beneficial effects:

[0013] 1. The present invention can efficiently and accurately process the massive monitoring data acquired by the monitoring instrument with high time resolution, avoiding the problems of low time efficiency and high error rate in the manual processing process;

[0014] 2. The present invention can automatically perform visualization of various dimensions based on the data uploaded by the user, effectively reducing the labor and time costs of traditional visualization.

[0015] 3. The present invention is based on the R language open source package library and can be maintained in real time, avoiding the problem of high cost of developing a specific data platform system. Therefore, the present invention provides a real-time carbon dioxide monitoring instrument data analysis and processing system based on the R language shiny package to solve the defect that the R language shiny package has not been implemented in the regional carbon dioxide monitoring visualization system in the prior art. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 The present invention is a schematic diagram of the module structure of a regionalized real-time carbon dioxide monitoring instrument data processing system. DETAILED DESCRIPTION

[0017] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0018] A regional real-time carbon dioxide monitoring instrument data processing system is a data analysis and processing system based on the R language shiny package. Its main functions include integrating the carbon dioxide concentration data of each monitoring point in the monitoring area, automatically performing time series analysis, analyzing the impact of meteorological parameters on data, and performing spatial change characteristic analysis.

[0019] The R language in this embodiment is an open source statistical computing and data visualization tool, known for its rich data analysis package library (such as ggplot2, dplyr, lme4, etc.) and powerful graphics capabilities, providing a solid technical foundation for this system. In particular, the shiny package is an R package in the R language for developing and creating network applications. By using the shiny package, developers can customize various web page interfaces according to their own needs, easily complete the development of web programs, provide users with an intuitive and friendly interface, and process and analyze a large amount of data from regional real-time carbon dioxide monitoring instruments in real time, making the timely display and analysis of carbon dioxide monitoring data efficient and convenient.

[0020] This embodiment aims at the technical field of regional real-time high-resolution carbon dioxide monitoring data processing, and proposes a real-time carbon dioxide monitoring instrument data analysis and processing system based on the R language shiny package. It includes a monitoring instrument, which is connected to a server, and includes a system UI interface, data import, and data analysis and processing visualization. The monitoring instrument is provided with a data import bar and a partition visualization display bar on the side for data import and data analysis and processing visualization;

[0021] In this embodiment, the UI interface mainly includes: a side data import bar and a partition visualization display bar. It mainly includes CO2 monitoring concentration data cleaning and time scale mean calculation, CO2 monitoring concentration data time series change characteristics, specific time-varying mode concentration change characteristics, meteorological data analysis, and multi-monitoring point spatial distribution characteristics; the server includes shiny server; the data import, i.e., user operation, includes: the user uploads and submits the recorded data of each parameter on the monitoring instrument in the system UI interface; the data analysis and visualization include: original data missing value screening and processing, original second-level or minute-level data uniformly converted into different time scale data sets, daily, weekly, and monthly average processing and corresponding data processing algorithm code; visualization includes: mainly including CO2 monitoring concentration data time series change characteristics (hourly time series change characteristics, daily change time series characteristics), specific time-varying mode concentration change characteristics (hourly changes, weekly changes, monthly changes), meteorological data analysis (wind speed and direction characteristics, wind direction pollution characteristics, correlation characteristics analysis), multi-monitoring point spatial distribution characteristics real-time automatic visualization graphics and corresponding data analysis codes.

[0022] The specific operations are as follows: The data analysis and processing visualization uses the R language data reading package library (readxl(), openxlsx(), read.csv(), etc.) to import data files, and calculates the number of missing values ​​in each column through the column missing value statistical function (for example: colSums(is.na(data)) function). The function returns a vector, in which each element represents the number of missing values ​​in the corresponding column. The number of missing values ​​in each column is divided by the total number of rows and multiplied by 100 to obtain the missing value ratio of each column. The missing value deletion function (for example: na.omit() function) is used to delete rows containing any missing values, thereby realizing the missing value screening of the original data.

[0023] Missing ratio = missing values / total number of rows × 100.

[0024] The data analysis and processing visualization converts the time column in the original data into the standard time format in R through the time date conversion function (for example, POSIXct() and other functions), and then uses the grouping aggregation function (such as aggregate() or functions in the dplyr package) to aggregate the data according to the required time scale, such as calculating the average or sum by hour, and processing the daily, weekly and monthly average values ​​for subsequent analysis or visualization. In this way, the original data can be converted from fine-grained seconds or minutes to a higher-level time scale data set to meet different analysis needs, and finally displayed in the system UI interface through the interactive data table function (for example, DT::datatable(), etc.);

[0025] The visualization in the data analysis and processing visualization includes calling the corresponding data processing and visualization toolkit (e.g., tidyverse, ggplot2, etc.) to visualize the data in time series according to the format of the data and the module selected by the user, mainly including the time series change characteristics of CO2 monitoring concentration data (hourly time series change characteristics, daily change time series characteristics), specific time-varying mode concentration change characteristics (hourly change, weekly change, monthly change); calling the correlation visualization function (e.g., corrplot() function) and the wind rose visualization function (e.g., windRose() function) to analyze meteorological data (wind speed and direction characteristics, wind direction pollution characteristics, correlation characteristics analysis); calling the spatial data / object processing package library in R language (e.g., sf and sp) to process spatial data, interpret spatial geographic information, and convert the data into a spatial data frame (all_points_tro). Then, the longitude and latitude are set through the coordinate extraction function (e.g., sp::coordinates()), and the grid is generated using the expanded grid function (expand.grid()). The inverse distance interpolation function (e.g., idw() function) is used to perform inverse distance weighted interpolation to generate the interpolation result (idw_output), which includes the longitude, latitude, and result after interpolation. Finally, the ggplot2 library is used to plot the IDW interpolation results of the study area, thereby realizing real-time automatic visualization and display of the spatial distribution characteristics of multiple monitoring points.

[0026] The present embodiment is for managers, scientific researchers, etc. Shiny project is constructed by R language, and data is imported by loading dashboard package of R language and data reading packages such as readxl, openxlsx, read.csv, sf, etc., so that the user can upload the specific format data (.xlsx, .xls, .csv) of multi-parameters obtained in the carbon dioxide monitoring instrument in the data import box of the system interface and the file format (.geojson) of spatial geographic information. After the user uploads the data, a prompt for successful data upload will be obtained. When the user clicks on the time series variation characteristics, the concentration variation characteristics of the specific time-varying mode, the meteorological data analysis and the spatial distribution characteristics, the system UI interface will switch to the corresponding interface, and the server receives feedback information. According to the format of the data and the module selected by the user, the corresponding package library and function (tidyverse, ggplot2, idw(), etc.) are called to visualize the data in the time series, and the visualization results during the corresponding data are displayed to realize the display of automated data processing and visualization analysis results.

[0027] The basic principles and main features of the present invention are described above. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The above embodiments and descriptions are only for illustrating the principles of the present invention. Without departing from the spirit and scope of the present invention, the present invention may have various changes and improvements, which shall fall within the scope of the present invention to be protected. The scope of protection of the invention shall be defined by the attached claims and their equivalents.

Claims

1. A regionalized real-time carbon dioxide monitoring instrument data processing system, characterized in that: The monitoring instrument includes a monitoring instrument connected to a server, and the monitoring instrument includes a system UI interface, data import, and data analysis and processing visualization. The monitoring instrument is provided with a data import bar and a partition visualization display bar on the side for data import and data analysis and processing visualization; The data analysis and processing visualization mainly includes the cleaning of CO2 monitoring concentration data and the calculation of the mean of each time scale, the time series change characteristics of CO2 monitoring concentration data, the concentration change characteristics of specific time-varying patterns, meteorological data analysis, and the spatial distribution characteristics of multiple monitoring points; the data import, i.e. user operation, includes: the user operates in the system UI interface to upload and submit the recorded data of each parameter on the monitoring instrument; After the user has uploaded the data, they will be prompted that the data has been successfully uploaded. When the user clicks on the time series change characteristics, the specific time-varying pattern concentration change characteristics, the meteorological data analysis, and the spatial distribution characteristics, the system UI interface will switch to the corresponding interface. The server receives the feedback information and calls the corresponding package library and function to visualize the data in time series according to the data format and the module selected by the user, and displays the visualization results of the corresponding data period, so as to realize the automatic data processing and visualization analysis results display.

2. A regionalized real-time carbon dioxide monitoring instrument data processing system according to claim 1, characterized in that: The data analysis and processing visualization uses the R language data reading package library to import data files, and calculates the number of missing values ​​in each column through the column missing value statistics function. The function returns a vector, in which each element represents the number of missing values ​​in the corresponding column. The number of missing values ​​in each column is divided by the total number of rows and multiplied by 100 to obtain the missing value ratio of each column. The missing value deletion function is used to delete rows containing any missing values, thereby realizing the missing value screening of the original data. Missing ratio = missing values / total number of rows × 100.

3. A regionalized real-time carbon dioxide monitoring instrument data processing system according to claim 2, characterized in that: The data analysis and processing visualization converts the time column in the original data into the standard time format in R through the time date conversion function, and then uses the grouping aggregation function to aggregate the data according to the required time scale, such as calculating the average or sum by hour, and processing the daily, weekly and monthly average values ​​for subsequent analysis or visualization. In this way, the original data can be converted from fine-grained second or minute levels to a higher-level time scale data set to meet different analysis requirements, and finally displayed in the system UI interface through the interactive data table function.

4. A regionalized real-time carbon dioxide monitoring instrument data processing system according to claim 3, characterized in that: The visualization in the data analysis and processing visualization includes calling the corresponding data processing and visualization toolkit to perform time series visualization processing on the data according to the format of the data and the module selected by the user, mainly including the time series change characteristics of the CO2 monitoring concentration data and the concentration change characteristics of the specific time-varying mode; The correlation visualization function and wind rose visualization function are used to analyze meteorological data. The spatial data / object processing package library in R language is used to process spatial data, interpret spatial geographic information, and convert the data into a spatial data frame. Then, the longitude and latitude are set through the coordinate extraction function, and the grid is generated using the extended grid function. The inverse distance interpolation function is used to perform inverse distance weighted interpolation to generate interpolation results, including longitude, latitude and results after interpolation. Finally, the ggplot2 library is used to draw the IDW interpolation results of the study area, thereby realizing real-time automatic visualization and display of the spatial distribution characteristics of multiple monitoring points.

Images