Safety production monitoring data management method, device and equipment and storage medium
By processing data using Kafka and Spark, combined with an autoregressive moving average model and Grafana display, the problem of inaccurate analysis results of production safety monitoring data was resolved, the authenticity and validity of the data were ensured, and the accuracy of analysis was improved.
Patent Information
- Application Number
- CN202210431104.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-04-22
- Publication Date
- 2025-10-17
- Estimated Expiration
- 2042-04-22
AI Technical Summary
The analysis results of production safety monitoring data in existing technologies have poor accuracy, mainly due to data quality issues such as data null values, duplications, non-standard types, non-standard time formats, garbled characters and noise, as well as non-human factors such as possible artificial falsification of data.
Kafka distributed message queue technology is used to collect data in real time and store it in the Mango DB database. The Spark big data platform is used to generate the original data set. SQL statements are used to handle data quality issues. The autoregressive moving average model and normal probability distribution model are combined to eliminate invalid data. Grafana is used to visualize trend charts.
It improves the accuracy of production safety monitoring data analysis, ensures the authenticity, integrity and validity of the data, and improves the accuracy of the analysis results.
Smart Images

Figure CN116975138B_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present application relates to the field of production data management, and in particular to a safety production monitoring data management method, device, equipment and storage medium. BACKGROUND
[0002] The State Emergency Management Department has initially built a safety production risk monitoring and early warning system covering major dangerous source storage tank areas and high-risk process devices of dangerous chemicals nationwide, and has accessed massive real-time data of enterprise major dangerous source pressure, temperature, liquid level, flammable and toxic and harmful gas monitoring data. According to different needs, by analyzing these real-time data, various data applications such as safety risk early warning, production status evaluation or management status evaluation can be realized.
[0003] The inventor found that the prior art at least has the following defects in the application mode of monitoring data:
[0004] The analysis results are not accurate enough when used in safety risk early warning, production status evaluation, management status evaluation and other data depth analysis application scenarios.
[0005] The information disclosed in this BACKGROUND section is only intended to increase an understanding of the general context in which the present application can be practiced. It is not admitted that any of the information provided in this section is prior art. SUMMARY
[0006] The purpose of the present application is to improve the accuracy of the analysis results when analyzing safety production monitoring data.
[0007] The present application provides a safety production monitoring data management method, comprising the steps of:
[0008] S11, real-time acquisition of real-time production data in the production process of an enterprise and storage into a preset database;
[0009] S12, extracting multi-source time series data from the preset database to generate an original data set;
[0010] S13, screening out data in the original data set that does not meet the preset rules to generate a preliminary selected data set;
[0011] S14, according to the strong time correlation characteristics of the dangerous chemical production data in the preliminary selected data set, eliminating invalid data through a preset validity rule to generate a valid data set;
[0012] S15, periodically extracting data from the valid data set, and displaying the management results of the time series data through a preset visual display model.
[0013] Preferably, in the present application, the real-time acquisition of real-time production data in the enterprise production process and storage to the preset database comprises:
[0014] The Kafka distributed message queue technology is used to build a data pipeline for collecting real-time production data of national dangerous chemical enterprises, and the real-time production data is stored in the Mango DB database in real time through the data consumption mode.
[0015] Preferably, in the present application, the extraction of multi-source data from the preset database to generate an original data set comprises:
[0016] The Spark big data platform is used to extract massive multi-source time series data in the enterprise production process from the Mango DB database to generate an original data set;
[0017] Preferably, in the present application, the screening of data in the original data set that does not conform to the preset rule to generate a primary selected data set comprises:
[0018] SQL statements are used to configure data standardization judgment rules for data null, continuous repetition, non-standard data type, non-standard time format and other data problems, and the data in the primary selected data set is judged piece by piece. SQL statements are used to delete data that does not conform to the preset rule, generate a primary selected data set that has been standardized and cleaned, and update the related data of the Mango DB database using the data in the primary selected data set through data set integration.
[0019] Preferably, in the present application, according to the strong time correlation characteristics of the dangerous chemical production data in the primary selected data set, invalid data is removed through a preset validity rule to generate a valid data set, comprising:
[0020] For data at time t, the autoregressive moving average model is used to predict the data value pair at the current time t by combining the p real values before time t and the q error values, to obtain the predicted value at the current time t. The formula includes:
[0021]
[0022] Wherein, y t is the predicted value at the current time t, y t is the real value at the current time t, ε t is the error value at the current time t, μ is a constant term, γ i is the autocorrelation coefficient, and λ j is the correlation coefficient associated with the error;
[0023] The difference between the true value and the predicted value at the current time t is calculated, and a set of generated data differences is statistically distributed using a normal probability distribution model. The data validity judgment rule includes:
[0024]
[0025] wherein v is the mathematical expectation of the data set, and sigma is the variance of the data set;
[0026] The data outside the invalid interval in the data set is removed using an SQL statement to obtain an effective data set processed for data validity, and the data in the effective data set is used to update the related data of the Mango DB database through data set integration.
[0027] Preferably, in the present application, the periodic extraction of data from the effective data set is performed through a preset visual display model to display the governance results of time series data in a trend chart, including:
[0028] According to the display requirements of time series data, an open-source time series data visualization tool Grafana is used to establish a data connection with the Mango DB database through a self-provided data source configuration tool, and a visual display model of system data is designed.
[0029] Periodically extract the time series data in the Mango DB database and display the governance results in a trend chart through the visual display model.
[0030] Preferably, in the present application, the periodic extraction of time series data in the Mango DB database and the display of the governance results in a trend chart through the visual display model include:
[0031] Periodically extract the time series data in the Mango DB database using an SQL statement and display the governance results in a trend chart through the visual display model.
[0032] Preferably, in the present application, the periodic extraction of time series data in the Mango DB database includes:
[0033] The time series data in the Mango DB database is extracted once every minute.
[0034] Preferably, in the present application, the safety production monitoring data governance includes:
[0035] Process device production data governance, storage tank area production data governance, or plant gas leakage data governance.
[0036] In another aspect of the present application, a safety production monitoring data management device is also provided, comprising:
[0037] A real-time data acquisition unit is configured to acquire real-time production data in an enterprise production process in real time and store the real-time production data into a preset database.
[0038] An original data set generation unit is configured to extract multi-source time series data from the preset database to generate an original data set.
[0039] A preliminary selected data set generation unit is configured to filter out data in the original data set that does not conform to a preset rule to generate a preliminary selected data set.
[0040] An effective data set generation unit is configured to remove invalid data from the preliminary selected data set according to a strong time correlation characteristic of the hazardous chemical production data in the preliminary selected data set by using a preset validity rule to generate an effective data set.
[0041] A trend chart display unit is configured to periodically extract data from the effective data set and display a trend chart of a management result of the time series data by using a preset visualization display model.
[0042] In another aspect of the present application, a safety production monitoring data management device is also provided, comprising:
[0043] A memory is configured to store a computer program.
[0044] A processor is configured to call and execute the computer program to implement each step of the safety production monitoring data management method according to any one of the above aspects.
[0045] In another aspect of the present application, a storage medium having a computer program stored thereon is also provided, and the computer program is executed by a processor to implement each step of the safety production monitoring data management method according to any one of the above aspects.
[0046] The safety production monitoring data management device includes a computer program stored in a medium, and the computer program includes program instructions, which, when executed by a computer, cause the computer to execute the method according to any one of the above aspects and achieve the same technical effects.
[0047] Compared with the prior art, the present application has the following beneficial effects:
[0048] The inventor has found that the important causes of the poor accuracy of the analysis result obtained when the safety production monitoring data is analyzed in the prior art include the poor quality of the data used for analysis, wherein, not only there are non-human factors such as data null, continuous repetition, non-standard data type, non-standard time format, random code and noise affecting data, but also there are human factors such as fake data, so that the accuracy of the final analysis result is poor.
[0049] Based on the above, in the present application, by standardizing the common problems such as null, repetition and random code, and automatically judging and processing the noise affecting data and fake data according to the strong correlation characteristics of time series data, the data with quality problems (i.e. invalid data) in the real-time production data in the enterprise production process are screened and sorted in multiple targeted ways, and the data with quality problems caused by various human and non-human factors are removed, so as to ensure the authenticity, integrity and effectiveness of the data used for analysis, and thus the accuracy of the analysis result obtained when the safety production monitoring data is analyzed can be effectively improved.
[0050] The above description is only a summary of the technical scheme of the present application, in order to more clearly understand the technical means of the present application and can be implemented according to the content of the specification, at the same time, in order to make the above and other purposes, technical features and advantages of the present application more easily understood, one or more preferred embodiments are listed below, and are described in detail as follows with reference to the accompanying drawings. BRIEF DESCRIPTION OF DRAWINGS
[0051] In order to more clearly illustrate the technical scheme of the present application, the drawings needed for the embodiments will be briefly introduced as follows. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.
[0052] Figure 1 is a step diagram of the safety production monitoring data management method described in the present application;
[0053] Figure 2 is a structural schematic diagram of the safety production monitoring data management device described in the present application;
[0054] Figure 3 is a structural schematic diagram of the safety production monitoring data management device described in the present application. DETAILED DESCRIPTION
[0055] The specific embodiments of the present application will be described in detail below with reference to the accompanying drawings, but it should be understood that the protection scope of the present application is not limited by the specific embodiments.
[0056] Unless specifically stated otherwise, throughout the specification and claims, the term "comprising" or variations such as "comprise" or "comprises" will be understood to imply the inclusion of a stated element or group of elements but not the exclusion of any other element or group of elements.
[0057] In this document, the terms "first", "second", etc. are used to distinguish two different elements or parts, and are not used to define a specific position or relative relationship. In other words, in some embodiments, the terms "first", "second", etc. can also be interchanged with each other.
[0058] Example One
[0059] In order to improve the accuracy of the analysis result when analyzing the safety production monitoring data, as Figure 1 As shown in the embodiment of the present application, a safety production monitoring data management method is provided, comprising the steps of:
[0060] S11, real-time production data in enterprise production process is acquired and stored to a preset database;
[0061] The safety production monitoring data management method in the embodiment of the present application is mainly used for production data management of dangerous chemical enterprises, which can specifically include process device production data management, tank farm production data management or plant gas leakage data management.
[0062] The inventors have found that the reason why the accuracy of the analysis result obtained by analyzing the safety production monitoring data is poor in the prior art is that the quality of the data used for analysis is poor, which not only has non-human factors such as data null, continuous repetition, non-standard data type, non-standard time format, random code and noise affecting data, but also has human factors such as fake data, so that the accuracy of the final analysis result is poor.
[0063] In practical application, the Kafka distributed message queue technology can be used to build a data pipeline for real-time collection of national dangerous chemical enterprise production data according to the high throughput and low delay characteristics of the mass production time series data of dangerous chemical enterprises, and the data can be stored in the local Mango DB database of the system in real time through the data consumption mode, so as to obtain the real-time production data of the enterprise.
[0064] When the embodiment of the present application is used for process device production data management, real-time production data can specifically include time series data such as temperature, pressure, liquid level, rotating speed and the like of a process device in a production process of a dangerous chemical enterprise, which can directly reflect the safe production state in a process flow. The data source is usually an industrial control system such as DCS, PLC and the like of an enterprise device. In the data collection process, data missing, null value, repetition, noise influence and the like quality problems can be caused due to protocol conversion, coding analysis and the like reasons.
[0065] When the embodiment of the present application is used for tank area production data management, real-time production data can specifically include time series data such as temperature, pressure, liquid level, rotating speed and the like which can directly reflect the safe production state of a tank. The data source of these data is usually an industrial control system such as DCS, PLC and the like of a tank area of an enterprise. In the data collection process, data missing, null value, repetition, noise influence and the like quality problems can be caused due to protocol conversion, coding analysis and the like reasons.
[0066] When the embodiment of the present application is used for plant gas leakage data management, real-time production data can specifically include combustible and toxic and harmful gas leakage monitoring data. The combustible and toxic and harmful gas leakage monitoring data is usually from a fire and gas system of an enterprise. Since a gas sensor is usually arranged in an open environment and is affected by a large environmental noise, a large disturbance can be caused to gas leakage sensing data. At the same time, in the data transmission process, data missing, null value, repetition and the like problems can be caused due to protocol conversion, coding analysis and the like reasons.
[0067] S12, extracting multi-source time series data from the preset database to generate an original data set;
[0068] Specifically, this step can be: using a Spark big data platform to extract massive multi-source time series data in an enterprise production process from a system local Mango DB database to form a data set to be standardized and cleaned (i.e., an original data set).
[0069] S13, screening out data in the original data set not conforming to a preset rule to generate a primary selected data set;
[0070] In actual application, for data null value, continuous repetition, data type not standard, time format not standard and the like problems, a data standardization judgment rule can be configured by using a SQL statement, and the extracted data set is judged piece by piece. A SQL statement is used to delete a data row not conforming to a preset rule to obtain a data set after standardization and cleaning (i.e., a primary selected data set), and the data in the primary selected data set is used to update a related data table of the Mango DB database through data set integration.
[0071] S14, according to the strong time correlation characteristics of the dangerous chemical production data in the initial data set, invalid data is removed by a preset validity rule to generate a valid data set;
[0072] In practical applications, according to the strong time correlation characteristics of the dangerous chemical production data, the following sub-steps can be performed on the data set (i.e., the initial data set) after standardization and cleaning:
[0073] For data at time t, the autoregressive moving average model is used to predict the numerical value of the current time t in combination with the p real values before time t and the q error values before time t, to obtain the predicted value of the current time t, and the formula used includes:
[0074]
[0075] Where y t is the predicted value of the current time t, y t is the real value of the current time t, ε t is the error value of the current time t, μ is a constant term, γ i is the autocorrelation coefficient, and λ j is the correlation coefficient associated with the error;
[0076] The difference between the real value and the predicted value of the current time t is calculated, and a normal probability distribution model is used to statistically analyze the probability distribution of the generated set of data differences, and the data validity judgment rule includes:
[0077]
[0078] Where ν is the mathematical expectation of the data set, and σ is the variance of the data set;
[0079] The data outside the invalid interval in the data set is removed using an SQL statement to obtain a valid data set after data validity processing, and the data in the valid data set is used to update the related data of the Mango DB database through data set integration.
[0080] S15, periodically extract data from the valid data set, and display the trend chart of the governance result of the time series data through a preset visual display model.
[0081] Specifically, it can be:
[0082] According to the display requirements of the time series data, an open source time series data visualization tool Grafana is used to establish a data connection with the Mango DB database through a built-in data source configuration tool, and a visual display model of system data is designed;
[0083] Periodically (e.g., every minute) extract the time series data in the Mango DB database using SQL statements, and display the governance results in a trend chart through the visualization model.
[0084] In summary, in the embodiment of the present application, by standardizing the common problems such as null, repetition, and garbled code, and according to the strong correlation characteristics of time series data, the automatic judgment and processing of noise influence data and fake data are realized, the data with quality problems (i.e., invalid data) in the real-time production data in the enterprise production process are subjected to various targeted screening and sorting, and the data with quality problems caused by various human factors and non-human factors are removed, thereby ensuring the authenticity, integrity and effectiveness of the data finally used for analysis, and further improving the accuracy of the analysis results obtained when analyzing the safety production monitoring data.
[0085] Embodiment two
[0086] Corresponding to the method embodiment, in another aspect of the embodiment of the present application, a safety production monitoring data governance device is also provided, Figure 2 The structure diagram of the safety production monitoring data governance device provided by the embodiment of the present application is shown, and the safety production monitoring data governance device is a device corresponding to the Figure 1 The device corresponding to the safety production monitoring data governance method in the corresponding embodiment, that is, the virtual device is realized Figure 1 The safety production monitoring data governance method in the corresponding embodiment, each virtual module of the safety production monitoring data governance device can be executed by an electronic device, such as a network device, a terminal device, or a server. Specifically, the safety production monitoring data governance device in the embodiment of the present application includes:
[0087] The real-time data acquisition unit 01 is used for acquiring real-time production data in the enterprise production process in real time and storing the real-time production data into a preset database;
[0088] The raw data set generation unit 02 is used for extracting multi-source time series data from the preset database to generate a raw data set;
[0089] The preliminary selected data set generation unit 03 is used for screening out data in the raw data set that does not conform to a preset rule to generate a preliminary selected data set;
[0090] The effective data set generation unit 04 is used for removing invalid data according to the strong time correlation characteristics of the dangerous chemical production data in the preliminary selected data set through a preset validity rule to generate an effective data set;
[0091] The trend chart display unit 05 is configured to periodically extract data from the effective data set and display the governance result of the time series data in a trend chart through a preset visualization display model.
[0092] It should be noted that the specific implementation and technical effects of the safety production monitoring data governance device in the embodiments of the present application can refer to Figure 1 the corresponding safety production monitoring data governance method, which will not be described here.
[0093] Embodiment three
[0094] Corresponding to the method embodiment, the present embodiment also provides a safety production monitoring data governance device, such as a terminal, a server, etc. The server can be a stand-alone physical server, a server cluster or a distributed system composed of multiple physical servers, or a cloud server providing cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDNs, and basic cloud computing services such as big data and artificial intelligence platforms. The terminal can be a smart phone, a tablet computer, a notebook computer, a desktop computer, etc., but is not limited thereto.
[0095] An example diagram of the hardware structure block diagram of the safety production monitoring data governance device provided by the present embodiment is shown in Figure 3 may include:
[0096] a processor 1, a communication interface 2, a memory 3, and a communication bus 4;
[0097] The processor 1, the communication interface 2, and the memory 3 can communicate with each other through the communication bus 4;
[0098] Optionally, the communication interface 2 can be an interface of a communication module, such as an interface of a GSM module.
[0099] The processor 1 can be a central processing unit CPU, or a specific integrated circuit ASIC, or one or more integrated circuits configured to implement the embodiments of the present application.
[0100] The memory 3 can include a high-speed RAM memory, and can also include a non-volatile memory, such as at least one disk memory.
[0101] The processor 1 is specifically configured to execute a computer program stored in the memory 3 to perform the following steps:
[0102] S11, real-time acquisition of real-time production data in an enterprise production process and storage to a preset database;
[0103] S12, extraction of multi-source time series data from the preset database to generate an original data set;
[0104] S13, screening of data in the original data set that does not conform to a preset rule to generate a primary selected data set;
[0105] S14, according to the strong time correlation characteristics of the hazardous chemical production data in the primary selected data set, invalid data is removed through a preset validity rule to generate a valid data set;
[0106] S15, periodic extraction of data from the valid data set, and trend chart display of the governance result of the time series data through a preset visualization display model.
[0107] The product can execute the method provided in the embodiment of the application, has the corresponding function modules and beneficial effects of the execution method. Technical details not described in detail in the embodiment can be referred to the safety production monitoring data governance method provided in the embodiment of the application.
[0108] Embodiment four
[0109] In the embodiment of the application, a storage medium is also provided, which can store a program suitable for execution by a processor, and the program is used for:
[0110] S11, real-time acquisition of real-time production data in an enterprise production process and storage to a preset database;
[0111] S12, extraction of multi-source time series data from the preset database to generate an original data set;
[0112] S13, screening of data in the original data set that does not conform to a preset rule to generate a primary selected data set;
[0113] S14, according to the strong time correlation characteristics of the hazardous chemical production data in the primary selected data set, invalid data is removed through a preset validity rule to generate a valid data set;
[0114] S15, periodic extraction of data from the valid data set, and trend chart display of the governance result of the time series data through a preset visualization display model.
[0115] Optionally, the detailed functions and extended functions of the program can be referred to the above description.
[0116] The product can execute the method provided by the embodiment of the application, has the function module and beneficial effect corresponding to the execution method. The technical details not described in detail in the embodiment can refer to the method provided by the embodiment of the application.
[0117] The product can execute the method provided by the embodiment of the application, has the function module and beneficial effect corresponding to the execution method. The technical details not described in detail in the embodiment can refer to the method provided by the embodiment of the application.
[0118] Those skilled in the art can realize that the units and algorithm steps of each example described in combination with the embodiments disclosed herein can be realized by electronic hardware or a combination of computer software and electronic hardware. Whether the functions are realized in hardware or software mode depends on the specific application of the technical solution and the design constraints. The skilled person can use different methods to realize the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.
[0119] In several embodiments provided in the present application, it should be understood that the disclosed system, device and method can be implemented in other ways. In addition, the coupling or direct coupling or communication connection between the displayed or discussed units can be indirect coupling or communication connection through some interface, device or unit, which can be electrical, mechanical or other forms.
[0120] The units described as separate components can or can not be physically separated, and the components displayed as units can or can not be physical units, that is, they can be located in one place or distributed on multiple network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the embodiment.
[0121] In addition, each functional unit in each embodiment of the present application can be integrated into a processing unit, or each unit can exist physically, or two or more units can be integrated into one unit.
[0122] It should be understood that the features in the embodiments of the present application can be combined with each other to achieve the solution of the foregoing technical problems.
[0123] If the functions are implemented in the form of software function units and sold or used as independent products, they can be stored in a computer readable storage medium. Based on this understanding, the technical solutions of the present application essentially or the parts that contribute to the prior art or parts of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes a number of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present application. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a magnetic disk or an optical disk, and various media that can store program codes.
[0124] The above description of the disclosed embodiments enables a person skilled in the art to implement or use the present application. Various modifications to the embodiments will be apparent to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.
Claims
1. A method for managing production safety monitoring data, characterized in that: Including steps: S11. Real-time production data of the enterprise during the production process is obtained and stored in a preset database; the preset database includes a Mango DB database; S12, extracting time series data from multiple sources from the preset database to generate an original data set; S13, filtering out data that does not conform to preset rules in the original data set to generate a preliminary data set; S14. Based on the strong time-correlation characteristics of the hazardous chemical production data in the preliminary selected data set, invalid data is eliminated using preset validity rules to generate a valid data set, including: For the data at time t, the autoregressive moving average model is used to predict the value pair at the current time t by combining the previous p true values and the previous q error values at time t to obtain the predicted value at the current time t. The formula used includes: Among them, y t is the predicted value at the current time t, y t is the true value at the current moment t, ε t is the error value at the current time t, μ is a constant term, γ i is the autocorrelation coefficient, λ j is the correlation coefficient associated with the error; Calculate the difference between the actual value and the predicted value at the current time t, and use the normal probability distribution model to perform probability distribution statistics on the generated set of data differences. The rules for judging the validity of the data include: Among them, ν is the mathematical expectation of the data set, σ is the variance of the data set; Using SQL statements to remove data outside the invalid interval in the data set to obtain a valid data set that has undergone data validity processing, and using the data in the valid data set to update the relevant data in the Mango DB database through data set integration; S15. Periodically extract data from the valid data set, and display the governance results of the time series data in a trend chart through a preset visualization model.
2. The method for managing production safety monitoring data according to claim 1, characterized in that: The real-time acquisition of production data during the enterprise's production process and storage in a preset database includes: Kafka distributed message queue technology is used to build a data pipeline for real-time collection of production data from hazardous chemical companies across the country. At the same time, real-time production data is stored in the Mango DB database through a data consumption model.
3. The method for managing production safety monitoring data according to claim 2, characterized in that: The step of extracting data from multiple sources from the preset database to generate an original data set includes: The Spark big data platform is used to extract massive multi-source time series data from the enterprise production process from the Mango DB database to generate an original data set.
4. The method for managing production safety monitoring data according to claim 3, characterized in that: The step of filtering out data that does not conform to preset rules from the original data set to generate a preliminary data set includes: In response to data problems, SQL statements are used to configure data standardization judgment rules, and the data in the preliminary data set are judged one by one. SQL statements are used to delete data that does not comply with preset rules, generate a preliminary data set that has been standardized and cleaned, and use the data in the preliminary data set through data set integration to update the relevant data in the Mango DB database; the data problems include null values, continuous duplications, irregular data types and irregular time formats.
5. The method for managing production safety monitoring data according to claim 4, characterized in that: The periodic extraction of data from the valid data set and the display of the governance results of the time series data in a trend chart using a preset visual display model include: Based on the display requirements of time series data, we used the open source time series data visualization tool Grafana to establish a data connection with the Mango DB database through its own data source configuration tool, and designed a visualization display model for the system data. The time series data in the Mango DB database is extracted periodically, and the governance results are displayed in a trend chart through the visual display model.
6. The method for managing production safety monitoring data according to claim 5, characterized in that: The periodic extraction of time series data from the Mango DB database and the display of the governance results in a trend chart using the visualization model include: SQL statements are used to periodically extract time series data from the Mango DB database, and the governance results are displayed in a trend chart through the visual display model.
7. The method for managing production safety monitoring data according to claim 6, characterized in that: The periodic extraction of time series data from the Mango DB database includes: The time series data in the Mango DB database is extracted once every minute.
8. The method for managing production safety monitoring data according to claim 1, characterized in that: The safety production monitoring data governance includes: Process unit production data management, tank area production data management, or plant area gas leakage data management.
9. A safety production monitoring data management device, characterized in that: include: Real-time data acquisition unit, used to acquire real-time production data in the enterprise's production process and store it in a preset database; The preset database includes a Mango DB database; An original data set generating unit, configured to extract time series data from multiple sources from the preset database to generate an original data set; A preliminary data set generating unit, configured to filter out data in the original data set that does not conform to preset rules and generate a preliminary data set; The valid data set generation unit is used to eliminate invalid data according to the strong time correlation characteristics of the hazardous chemical production data in the preliminary data set through preset validity rules to generate a valid data set, including: For the data at time t, the autoregressive moving average model is used to predict the value pair at the current time t by combining the previous p true values and the previous q error values at time t to obtain the predicted value at the current time t. The formula used includes: Among them, y′ t is the predicted value at the current time t, y t is the true value at the current moment t, ε t is the error value at the current time t, μ is a constant term, γ i is the autocorrelation coefficient, λ j is the correlation coefficient associated with the error; Calculate the difference between the actual value and the predicted value at the current time t, and use the normal probability distribution model to perform probability distribution statistics on the generated set of data differences. The rules for judging the validity of the data include: Among them, ν is the mathematical expectation of the data set, σ is the variance of the data set; Using SQL statements to remove data outside the invalid interval in the data set to obtain a valid data set that has undergone data validity processing, and using the data in the valid data set to update the relevant data in the Mango DB database through data set integration; The trend chart display unit is used to periodically extract data from the valid data set and display the management results of the time series data in a trend chart through a preset visual display model.
10. A safety production monitoring data management device, characterized in that: include: Memory for storing computer programs; A processor is used to call and execute the computer program to implement the steps of the production safety monitoring data governance method as described in any one of claims 1 to 8.
11. A storage medium, characterized in that: It includes a software program, which is suitable for a processor to execute the steps of the safety production monitoring data management method as described in any one of claims 1-8.
Citation Information
Patent Citations
A time sequence data exception monitoring method and system
CN109697207A
Intelligent alarm analysis method and device, equipment and storage medium
CN113791926A
Cited By
Power production monitoring data management and high-value data asset generation method and system based on storage and calculation integrated time sequence library
CN122285653A
A power production monitoring data governance and high-value data asset generation method and system based on a storage-computation integrated time sequence library
CN122285653B