A comprehensive management system for multi-source data fusion of safety and environmental protection in chemical parks

Abstract

This invention discloses a multi-source data fusion and integrated management system for safety and environmental protection in chemical industrial parks. It includes a multi-source data acquisition module, an integrated analysis and management module, a multi-source data storage and analysis module, a heterogeneous storage analysis module, and an information management output module. This invention relates to the field of multi-source data management technology and solves the technical problem that simple compression storage cannot achieve data security protection and reduce the risk of data theft. This invention analyzes multi-source data, firstly by uniformly converting the data format for easier subsequent overall management, and secondly by storing and analyzing the converted multi-source data. By segmenting the multi-source data and performing different storage and analysis methods based on the characteristics of the segmented multi-source data, it achieves both overall data compression storage and data encryption, thereby improving data security during storage.

Description

Technical Field

[0001] This invention relates to the field of multi-source data management technology, specifically to a multi-source data fusion and integrated management system for safety and environmental protection in chemical industrial parks. Background Technology

[0002] Multi-source data fusion and integrated management refers to the process of aggregating and integrating data from different sources in the construction of digital government, and achieving full lifecycle management of data through comprehensive quality management, in order to support the improvement of government decision-making and services.

[0003] According to patent application number CN202310570310.6, the patent includes an inertial navigation processing unit, an environmental terminal acquisition module, a data acquisition and fusion unit, a data verification unit, an encrypted upload unit, and a database. The inertial navigation processing unit and the environmental terminal acquisition module are both communicatively connected to the data acquisition and fusion unit. The output end of the data acquisition and fusion unit is communicatively connected to the data verification unit. The output end of the data verification unit is communicatively connected to the encrypted upload unit. The output end of the encrypted upload unit is communicatively connected to the database. The database internally includes a classification storage unit and a data management unit.

[0004] The aforementioned patent uses a data verification unit in conjunction with an encrypted upload unit to verify and encrypt the merged data before sending it, ensuring the accuracy and security of the data. The data management unit can avoid the risk of data tampering, and the classification storage unit can classify and store data from the same source, thus facilitating subsequent retrieval and integration, and promoting its overall use.

[0005] However, the aforementioned patents have shortcomings in data storage management. Data storage is a crucial part of data management. Some existing management systems typically use compression to store data. While this method can reduce data volume, it fails to protect the data and may lead to data loss or theft during subsequent use. Summary of the Invention

[0006] To address the shortcomings of existing technologies, this invention provides a multi-source data fusion and integrated management system for safety and environmental protection in chemical industrial parks, which solves the problem that simple compressed storage cannot achieve data security protection and cannot reduce the risk of data theft.

[0007] To achieve the above objectives, the present invention provides the following technical solution: a multi-source data fusion and integrated management system for safety and environmental protection in chemical industrial parks, comprising:

[0008] The multi-source data acquisition module is used to collect the data format and data volume of multi-source data, and transmit the basic information of the collected multi-source data to the overall analysis and management module;

[0009] The overall analysis and management module is used to analyze the basic information of the acquired multi-source data. First, the multi-source data is classified according to the data format, and sorted according to the reading speed and storage capacity of the data format to generate classification format information. Then, a standard format is selected based on the classification format information, and standard format information is generated and transmitted to the multi-source data storage and analysis module.

[0010] The multi-source data storage and analysis module is used to store and analyze multi-source data according to the acquired standard format information, obtain the data characteristics of multi-source data, compare the data characteristics with preset values ​​to classify multi-source data into first feature data and second feature data, and transmit the second feature data to the heterogeneous data storage and analysis module.

[0011] Next, the first feature data is analyzed. Based on the data characteristics of the first feature data, the data capacity is divided equally to obtain the average numerator data. Based on the repeating data capacity of the average numerator data, the data is analyzed and stored to generate storage information. At the same time, the storage information is transmitted to the information management output module.

[0012] The heterogeneous data storage and analysis module is used to analyze the acquired second feature data, determine the parity of the data features of the second feature data, and divide the second feature data equally to obtain the second equal-divided data. It further stores the data according to the data integrity corresponding to the second equal-divided data, and generates storage information. Then, it transmits the storage information to the information management output module.

[0013] The information management output module is used to store multi-source data based on the acquired storage information.

[0014] As a further aspect of the present invention, the specific method by which the overall analysis and management module generates the classification format information is as follows:

[0015] The acquired multi-source data is classified according to its corresponding data format, and the data format is labeled as i, where i = 1, 2, ..., j. At the same time, the reading speed Vi and storage capacity Li corresponding to the data format i are obtained, and the data format i is sorted from fastest to slowest according to the reading speed Vi. Then, the storage capacity Li corresponding to the data format i is matched according to the sorting order, and classification format information is generated.

[0016] As a further aspect of the present invention, the specific method by which the overall analysis and management module performs secondary analysis on the classification format information is as follows:

[0017] The sum of the data format i read speed Vi and storage capacity Li is calculated, and the average of the two values ​​is denoted as Pi. The data format corresponding to the smallest average value Pi is selected as the standard and denoted as the standard format k. The standard format information is then generated.

[0018] As a further aspect of the present invention: the specific method by which the multi-source data storage and analysis module analyzes multi-source data according to standard format information is as follows:

[0019] The data format of the acquired multi-source data is denoted as the initial format. The initial format is converted into a standard format to obtain standard multi-source data. The standard multi-source data is labeled as n, where n = 1, 2, ..., m. The conversion time of the standard multi-source data is further obtained and denoted as Tn. The conversion time is marked as a data feature.

[0020] Next, the standard multi-source data n is sorted from largest to smallest according to the data characteristics. At the same time, the data characteristics are compared with preset values. When the data characteristics are greater than the preset values, the corresponding standard multi-source data is recorded as the first characteristic data. Conversely, when the data characteristics are less than the preset values, the corresponding standard multi-source data is recorded as the second characteristic data.

[0021] As a further aspect of the present invention: the specific method by which the multi-source data storage and analysis module stores and analyzes the first feature data is as follows:

[0022] The parity of the data feature Tn of the first feature data n is determined. When the data feature Tn is odd, the first feature data n is divided into three equal parts according to the data capacity Ln to obtain the average numerator data. When the data feature Tn is even, the first feature data n is divided into four equal parts according to the data capacity Ln to obtain the average numerator data.

[0023] Next, the obtained average molecular data is stored. A set of average molecular data is randomly selected, and the duplicate data is extracted. The remaining average molecular data is recorded as non-duplicate data.

[0024] Next, the three duplicate data sets are integrated to obtain a duplicate data packet. At the same time, the data capacity of the duplicate data packet is obtained, and the reduction in the overall capacity of the first feature data n is calculated. Then, the data capacity of the duplicate data packet is compared with the reduction.

[0025] As a further aspect of the present invention: the specific method by which the multi-source data storage and analysis module compares the data capacity and reduction amount of duplicate data packets is as follows:

[0026] When the data capacity of the duplicate data packet is less than the reduction amount, the duplicate data packet is converted into binary to obtain binary duplicate data, and the binary duplicate data is reversed. At the same time, the reversed binary duplicate data is converted into reverse data. Then, the reverse data is divided into three equal parts, and the reverse data and the equal parts are recombined to obtain recombined data. Finally, the recombined data is stored to generate storage information.

[0027] As a further aspect of the present invention: the specific method by which the multi-source data storage and analysis module compares the data capacity and reduction amount of duplicate data packets is as follows:

[0028] When the data size of duplicate data packets is greater than the reduction amount, the duplicate data packets are removed based on the reduction amount to obtain the removed data packets. The removed data packets are then converted into binary to obtain binary removed data packets. Similarly, the analysis above shows that the data size of duplicate data packets is less than the reduction amount. Finally, reconstructed data is generated and stored to generate storage information.

[0029] As a further aspect of the present invention: the specific method by which the heterogeneous storage analysis module analyzes the second feature data is as follows:

[0030] The data features corresponding to the second feature data are obtained, and the parity of the data features is judged. When the data features are odd, the data capacity of the second feature data is divided equally according to a preset value to obtain the second equal-divided data. When the data features are even, the data capacity of the second feature data is divided equally according to the data features to obtain the second equal-divided data.

[0031] As a further aspect of the present invention: the specific method by which the heterogeneous storage analysis module analyzes the second evenly distributed data is as follows:

[0032] The system acquires the second average data and checks its completeness. If the data corresponding to the second average data is incomplete, it does not process the second average data and directly stores it, generating storage information. Conversely, if the data corresponding to the second average data is complete, it acquires the duplicate data in the second average data, integrates all the duplicate data to obtain duplicate data packets, stores the duplicate data packets to generate storage information, then stores the remaining second average data separately and generates storage information, and finally transmits the generated storage information to the information management output module.

[0033] Beneficial effects

[0034] This invention provides a multi-source data fusion and integrated management system for safety and environmental protection in chemical industrial parks. Compared with existing technologies, it has the following advantages:

[0035] This invention analyzes multi-source data, firstly by uniformly converting the format of the multi-source data to facilitate subsequent overall management, and secondly by storing and analyzing the converted multi-source data. By segmenting the multi-source data and performing different storage and analysis methods based on the characteristics of the segmented multi-source data, it can achieve both overall data compression and data encryption, thereby improving data security during the storage process. Attached Figure Description

[0036] Figure 1 This is a block diagram illustrating the system principle of the present invention. Detailed Implementation

[0037] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0038] Please see Figure 1 This application provides a multi-source data fusion and integrated management system for safety and environmental protection in chemical industrial parks, including: a multi-source data acquisition module, an integrated analysis and management module, a multi-source data storage and analysis module, a heterogeneous storage analysis module, and an information management and output module.

[0039] As an embodiment of the present invention

[0040] The multi-source data acquisition module is used to collect multi-source data and its basic information, and transmit the collected multi-source data basic information to the overall analysis and management module. The specific multi-source data basic information includes: format and data volume.

[0041] Specifically, multi-source data refers to data collected from multiple different sources or channels. These data sources may include, but are not limited to, sensors, social media, official releases, and surveys, and their corresponding data characteristics are diverse.

[0042] The overall analysis and management module is used to analyze and manage the acquired multi-source data, and the specific overall management methods are as follows:

[0043] The acquired multi-source data is classified according to its corresponding data format, which includes structured data, semi-structured data, and unstructured data. Structured data includes Excel and CSV (Comma-Separated Values), while semi-structured data includes JSON (JavaScript Object Notation) and XML (Extensible Markup Language). Unstructured data includes text, images, HTML, and multimedia. The data format is then labeled as i, where i = 1, 2, ..., j. For example, 1 represents text format, 2 represents image format, and so on. All data formats are labeled, and the reading speed Vi and storage capacity Li corresponding to data format i are obtained. Data format i is then sorted from fastest to slowest according to reading speed Vi. Then, the storage capacity Li corresponding to data format i is matched according to the sorting order, and classification format information is generated.

[0044] Next, a secondary analysis is performed on the classification format information. The sum of the reading speed Vi and storage capacity Li of data format i is calculated, and the average value of the two values ​​is calculated and denoted as Pi. The data format corresponding to the smallest average value Pi is selected as the standard and denoted as the standard format k. The standard format information is then generated and transmitted to the multi-source data storage and analysis module.

[0045] The multi-source data storage and analysis module is used to store and analyze multi-source data based on the acquired standard format information. The specific storage and analysis methods are as follows:

[0046] The data format of the acquired multi-source data is denoted as the initial format. The initial format is converted into a standard format to obtain standard multi-source data. The standard multi-source data is labeled as n, where n = 1, 2, ..., m. The conversion time of the standard multi-source data is further obtained and denoted as Tn. The conversion time is marked as a data feature.

[0047] Next, the standard multi-source data n is sorted from largest to smallest according to the data characteristics. At the same time, the data characteristics are compared with preset values. When the data characteristics are greater than the preset values, including cases where they are equal to the preset values, the corresponding standard multi-source data is recorded as the first characteristic data. Conversely, when the data characteristics are less than the preset values, the corresponding standard multi-source data is recorded as the second characteristic data. The specific preset values ​​here are set by the operator.

[0048] Further storage analysis is performed on the data classified as the first feature, and the specific storage analysis method is as follows:

[0049] The parity of the data feature Tn of the first feature data n is determined. When the data feature Tn is odd, the first feature data n is divided into three equal parts according to the data capacity Ln to obtain the average numerator data. When the data feature Tn is even, the first feature data n is divided into four equal parts according to the data capacity Ln to obtain the average numerator data.

[0050] Taking the three equal parts of the data as an example, the equal parts of the data are then stored. Then, a set of equal parts of the data is randomly selected and the duplicate data is extracted. At the same time, the remaining equal parts of the data are recorded as non-duplicate data. The same process is applied to the remaining two sets of equal parts of the data.

[0051] Next, the three sets of duplicate data are integrated to obtain a duplicate data packet. Specifically, the duplicate data packet obtained here contains different forms of duplicate data, and there is only one set of duplicate data of the same form. At the same time, the data capacity of the duplicate data packet is obtained, and the reduction of the overall capacity of the first feature data n is calculated. Then, the data capacity of the duplicate data packet is compared with the reduction.

[0052] When the data capacity of the duplicate data packet is less than the reduction amount, the duplicate data packet is converted into binary to obtain binary duplicate data, and the binary duplicate data is reversed. At the same time, the reversed binary duplicate data is converted into reverse data. Then, the reverse data is divided into three equal parts, and the reverse data and the equal parts are recombined to obtain recombined data. Finally, the recombined data is stored to generate storage information, and the storage information is transmitted to the information management output module.

[0053] Specifically, the obtained inverted data is divided into equal parts, and then the inverted data is recombined with the three equal parts. This method can encrypt the data and compress it relative to the whole data, thereby reducing the overall storage space and making it easier to store.

[0054] When the data capacity of duplicate data packets is greater than the reduction amount, the duplicate data packets are removed based on the reduction amount to obtain the removed data packets. The removed data packets are then converted into binary to obtain binary removed data packets. Similarly, when the data capacity of duplicate data packets is less than the reduction amount, the reconstructed data is generated and stored to generate storage information. At the same time, the storage information is transmitted to the information management output module.

[0055] The information management output module is used to store the first feature data based on the generated storage information.

[0056] As a second embodiment of the present invention, this embodiment is implemented based on the first embodiment, and the difference between the two embodiments is that...

[0057] The multi-source data storage and analysis module transmits the second feature data obtained from classification to the heterogeneous storage analysis module, and then analyzes it through the heterogeneous storage analysis module. The specific analysis method is as follows:

[0058] The data features corresponding to the second feature data are obtained, and the parity of the data features is judged. When the data feature is odd, the data volume of the second feature data is divided equally according to a preset value to obtain the second equal-divided data. When the data feature is even, the data volume of the second feature data is divided equally according to the data feature to obtain the second equal-divided data. For example, if the preset value is 0.6, the second feature data is divided into six equal parts. If the data feature is 0.3, it is divided into three equal parts. If the value is 1, it is divided into ten equal parts. Similarly, if the value is 0.1, it is also divided into ten equal parts.

[0059] Next, the second average data is obtained, and its integrity is judged. If the data corresponding to the second average data is incomplete, the second average data is not processed, and it is directly stored and storage information is generated. Conversely, if the data corresponding to the second average data is complete, the duplicate data in the second average data is obtained, and all the duplicate data is integrated to obtain duplicate data packets. The duplicate data packets are stored and storage information is generated. Then, the remaining second average data is stored separately and storage information is generated. Finally, the generated storage information is transmitted to the information management output module.

[0060] The information management output module is used to store the second feature data based on the acquired storage information.

[0061] As a third embodiment of the present invention, the focus is on combining the implementation processes of the first and second embodiments.

[0062] Furthermore, any content not described in detail in this specification is existing technology known to those skilled in the art.

[0063] The above embodiments are only used to illustrate the technical method of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical method of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical method of the present invention.

Claims

1. A multi-source data fusion and integrated management system for safety and environmental protection in chemical industrial parks, characterized in that, include: The multi-source data acquisition module is used to collect the data format and data volume of multi-source data, and transmit the basic information of the collected multi-source data to the overall analysis and management module; The overall analysis and management module is used to analyze the basic information of the acquired multi-source data. First, the multi-source data is classified according to the data format, and sorted according to the reading speed and storage capacity of the data format to generate classification format information. Then, a standard format is selected based on the classification format information, and standard format information is generated and transmitted to the multi-source data storage and analysis module. The multi-source data storage and analysis module is used to store and analyze multi-source data according to the acquired standard format information, obtain the data characteristics of multi-source data, compare the data characteristics with preset values ​​to classify multi-source data into first feature data and second feature data, and transmit the second feature data to the heterogeneous data storage and analysis module. Next, the first feature data is analyzed. Based on the data characteristics of the first feature data, the data volume is evenly divided to obtain the numerator data. Then, based on the repetitive data volume of the numerator data, analysis and storage are performed to generate storage information. At the same time, the storage information is transmitted to the information management output module. The specific processing method is as follows: The parity of the data feature Tn of the first feature data n is determined. When the data feature Tn is odd, the first feature data n is divided into three equal parts according to the data capacity Ln to obtain the average numerator data. When the data feature Tn is even, the first feature data n is divided into four equal parts according to the data capacity Ln to obtain the average numerator data. Next, the obtained average molecular data is stored. A set of average molecular data is randomly selected, and the duplicate data is extracted. The remaining average molecular data is recorded as non-duplicate data. Next, the three duplicate data sets are integrated to obtain a duplicate data packet. At the same time, the data capacity of the duplicate data packet is obtained, and the reduction in the overall capacity of the first feature data n is calculated. Then, the data capacity of the duplicate data packet is compared with the reduction. When the data capacity of the duplicate data packet is less than the reduction amount, the duplicate data packet is converted into binary to obtain binary duplicate data, and the binary duplicate data is reversed. At the same time, the reversed binary duplicate data is converted into reverse data. Then, the reverse data is divided into three equal parts, and the reverse data and the equal parts are recombined to obtain recombined data. Finally, the recombined data is stored to generate storage information. The heterogeneous data storage and analysis module is used to analyze the acquired second feature data, determine the parity of the data features of the second feature data, and divide the second feature data equally to obtain the second equal-divided data. It further stores the data according to the data integrity corresponding to the second equal-divided data, and generates storage information. Then, it transmits the storage information to the information management output module. The information management output module is used to store multi-source data based on the acquired storage information; The specific method by which the multi-source data storage and analysis module analyzes multi-source data according to standard format information is as follows: the data format of the multi-source data is obtained and recorded as the initial format, and the initial format is converted into the standard format to obtain standard multi-source data. At the same time, the standard multi-source data is labeled as n, where n=1, 2, ..., m. The conversion time of the standard multi-source data is further obtained and recorded as Tn, and the conversion time is marked as a data feature. Then, the standard multi-source data n is sorted from largest to smallest according to the data feature, and the data feature is compared with a preset value. When the data feature is greater than the preset value, the corresponding standard multi-source data is recorded as the first feature data; otherwise, when the data feature is less than the preset value, the corresponding standard multi-source data is recorded as the second feature data.

2. The multi-source data fusion and integrated management system for safety and environmental protection in chemical industrial parks according to claim 1, characterized in that, The specific method by which the overall analysis and management module generates the categorized format information is as follows: The acquired multi-source data is classified according to its corresponding data format, and the data format is labeled as i, where i = 1, 2, ..., j. At the same time, the reading speed Vi and storage capacity Li corresponding to data format i are obtained, and data format i is sorted from fastest to slowest according to reading speed Vi. Then, the storage capacity Li corresponding to data format i is matched according to the sorting order, and classification format information is generated.

3. The multi-source data fusion and integrated management system for safety and environmental protection in chemical industrial parks according to claim 1, characterized in that, The specific method by which the overall analysis and management module performs secondary analysis on the categorized format information is as follows: The sum of the data format i read speed Vi and storage capacity Li is calculated, and the average of the two values ​​is denoted as Pi. The data format corresponding to the smallest average value Pi is selected as the standard and denoted as the standard format k. The standard format information is then generated.

4. The multi-source data fusion and integrated management system for safety and environmental protection in chemical industrial parks according to claim 1, characterized in that, The specific method by which the multi-source data storage and analysis module compares the data capacity and reduction amount of duplicate data packets is as follows: When the data size of duplicate data packets is greater than the reduction amount, the duplicate data packets are removed based on the reduction amount to obtain the removed data packets. The removed data packets are then converted into binary to obtain binary removed data packets. Similarly, the analysis above shows that the data size of duplicate data packets is less than the reduction amount. Finally, reconstructed data is generated and stored to generate storage information.

5. The multi-source data fusion and integrated management system for safety and environmental protection in chemical industrial parks according to claim 1, characterized in that, The specific method by which the heterogeneous data storage and analysis module analyzes the second feature data is as follows: The data features corresponding to the second feature data are obtained, and the parity of the data features is judged. When the data features are odd, the data capacity of the second feature data is divided equally according to a preset value to obtain the second equal-divided data. When the data features are even, the data capacity of the second feature data is divided equally according to the data features to obtain the second equal-divided data.

6. The multi-source data fusion and integrated management system for safety and environmental protection in chemical industrial parks according to claim 5, characterized in that, The specific method by which the heterogeneous data storage and analysis module analyzes the second average data is as follows: The system acquires the second average data and checks its completeness. If the data corresponding to the second average data is incomplete, it does not process the second average data and directly stores it, generating storage information. Conversely, if the data corresponding to the second average data is complete, it acquires the duplicate data in the second average data, integrates all the duplicate data to obtain duplicate data packets, stores the duplicate data packets to generate storage information, then stores the remaining second average data separately and generates storage information, and finally transmits the generated storage information to the information management output module.

Images