Data storage method and device for power internet of things, and storage medium

Abstract

The invention discloses a data storage method and device for an electric power internet of things and a storage medium, and the method comprises the steps: carrying out the preprocessing of system operation data when the real-time system operation data of the electric power internet of things is collected, determining the priority of the system operation data if the preprocessing is completed, and storing the priority of the system operation data in a storage medium; and storing the system operation data in the storage hierarchy matched with the priority, adjusting the priority of the system operation data according to the storage information and the access information of the system operation data in the storage hierarchy, and migrating the system operation data to the storage hierarchy matched with the adjusted priority. Real-time system operation data collected by the electric power internet of things is preprocessed, the priority of the real-time system operation data is determined, the system operation data is stored in a matched storage hierarchy, and the electric power internet of things can preferentially process and store important and frequently-accessed system operation data, so that the system operation data retrieval and analysis speed is increased, and the system operation data retrieval and analysis efficiency is improved. And the operation efficiency and stability of the power Internet of Things are improved.

Description

Technical Field

[0001] The embodiments of the present invention relate to the technical field of electric power Internet of Things, and in particular to a data storage method, device and storage medium of electric power Internet of Things. Background Art

[0002] With the rapid development of power Internet of Things technology, the amount of real-time system operation data of the power Internet of Things has increased significantly. These system operation data not only include information such as equipment status, energy consumption, load changes, etc., but also involve the security, reliability and economy of the power Internet of Things. Therefore, storing and managing massive system operation data is one of the factors affecting the efficiency and stability of the power Internet of Things.

[0003] At present, the power Internet of Things often adopts traditional single-level, static data storage systems, which may lead to low utilization of storage resources and inefficient data access, affecting the real-time response capability of the power Internet of Things as well as the operating efficiency and stability of the power Internet of Things. Summary of the invention

[0004] The present invention provides a data storage method, device and storage medium for an electric power Internet of Things, so as to improve the operation efficiency and stability of the electric power Internet of Things.

[0005] In a first aspect, an embodiment of the present invention provides a data storage method for an electric power Internet of Things, comprising:

[0006] When real-time system operation data is collected from the power Internet of Things, preprocessing is performed on the system operation data;

[0007] If the preprocessing is completed, determining the priority of the system operation data;

[0008] Storing the system operation data in a storage level matching the priority;

[0009] adjusting the priority of the system operation data according to storage information and access information of the system operation data in the storage layer;

[0010] The system operation data is migrated to the storage tier that matches the adjusted priority.

[0011] In a second aspect, an embodiment of the present invention further provides a data storage device for an electric power Internet of Things, comprising:

[0012] A data initialization module, used to perform preprocessing on the system operation data when real-time system operation data is collected from the power Internet of Things;

[0013] A priority determination module, used for determining the priority of the system operation data if the preprocessing is completed;

[0014] A data storage module, used for storing the system operation data in a storage level matching the priority;

[0015] A priority adjustment module, configured to adjust the priority of the system operation data according to storage information and access information of the system operation data in the storage layer;

[0016] A data migration module is used to migrate the system operation data to the storage level that matches the adjusted priority.

[0017] In a third aspect, an embodiment of the present invention further provides a computer device, including:

[0018] one or more processors;

[0019] A storage device for storing one or more programs;

[0020] When the one or more programs are executed by the one or more processors, the one or more processors implement the data storage method for the electric power Internet of Things provided in the first aspect of the present invention.

[0021] In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the data storage method for the electric power Internet of Things provided in the first aspect of the present invention.

[0022] In a fifth aspect, an embodiment of the present invention further provides a computer program product, which includes a computer program. When the computer program is executed by a processor, it implements the data storage method of the power Internet of Things provided in the first aspect of the present invention.

[0023] In an embodiment of the present invention, when the real-time system operation data is collected by the power Internet of Things, the system operation data is preprocessed. If the preprocessing is completed, the priority of the system operation data is determined, the system operation data is stored in a storage layer matching the priority, the priority of the system operation data is adjusted according to the storage information and access information of the system operation data in the storage layer, and the system operation data is migrated to the storage layer matching the adjusted priority. Performing preprocessing on the real-time system operation data collected by the power Internet of Things and determining its priority, and storing the system operation data in a matching storage layer can effectively improve the management efficiency and reliability of the system operation data. In this way, the power Internet of Things can give priority to processing and storing important and frequently accessed system operation data, thereby speeding up the retrieval and analysis of the system operation data. At the same time, the priority of the system operation data is dynamically adjusted according to the storage information and access information to ensure that the system operation data is always in a suitable storage layer, optimize the utilization of storage resources, reduce costs, and enhance the flexibility and responsiveness of the power Internet of Things, thereby improving the operating efficiency and stability of the power Internet of Things. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 A flowchart of a data storage method for an electric power Internet of Things provided in Embodiment 1 of the present invention;

[0025] Figure 2 A structural block diagram of a data storage device for an electric power Internet of Things provided in Embodiment 2 of the present invention;

[0026] Figure 3 A schematic diagram of the structure of a computer device provided in Embodiment 3 of the present invention. DETAILED DESCRIPTION

[0027] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0028] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present invention described herein can cover sequential implementations other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0029] Embodiment 1

[0030] See also Figure 1 , shows a flow chart of a data storage method of the power Internet of Things provided by the first embodiment of the present invention, the method can be executed by a data storage device of the power Internet of Things, the data storage device of the power Internet of Things can be implemented in the form of hardware and / or software, and the data storage device of the power Internet of Things can be configured in a computer device. Figure 1 As shown, the method includes:

[0031] Step 101: When real-time system operation data is collected from the power Internet of Things, preprocessing is performed on the system operation data.

[0032] The power Internet of Things is a network that applies the Internet of Things technology to the power system, using sensors, communication technology and data analysis to achieve real-time monitoring, control and optimization of power equipment and systems. The power Internet of Things continuously collects system operation data in the power grid through various devices and sensors, providing a comprehensive understanding of power production, transmission and consumption.

[0033] System operation data refers to the system operation data related to the operation of the power Internet of Things collected in real time in the power Internet of Things, such as voltage, current, frequency, load status and equipment health status, etc. These system operation data can reflect the operating efficiency and safety of the power Internet of Things.

[0034] In this embodiment, real-time system operation data is collected from the power Internet of Things. Real-time collection helps to promptly discover potential faults and abnormal conditions, allowing operation and maintenance personnel to respond quickly, thereby reducing power outages and economic losses. The collected real-time system operation data is preprocessed to improve the accuracy and availability of the system operation data.

[0035] Exemplarily, the preprocessing may include data cleaning, data formatting, data compression, and data deduplication. The present invention is not limited here.

[0036] Data cleaning refers to the process of identifying and correcting inaccurate, missing or inconsistent system operation data. This process usually includes detecting and processing missing values, erroneous values, duplicate records and outliers to ensure the quality and reliability of system operation data. By performing data cleaning, more accurate and effective system operation data can be analyzed, thereby providing accuracy for subsequent analysis.

[0037] Data formatting is the conversion of system operation data into a consistent format for subsequent processing and analysis. This usually involves converting date, time, currency, and other types of values ​​into a standardized format. In addition, data formatting also includes the conversion of system operation data types, such as converting text data into numeric or categorical data, to ensure that system operation data can be correctly understood and used. Formatted system operation data is easier to compare, analyze, and visualize.

[0038] Data compression is the process of optimizing storage and transmission efficiency by reducing the size of system operation data files. Data compression can use a variety of technologies, such as removing redundant information, encoding conversion, etc., to reduce the space occupied by system operation data while trying to preserve the integrity and availability of system operation data. Through data compression, storage costs can be saved, system operation data transmission speed can be accelerated, and the efficiency of processing large-scale system operation data can be improved.

[0039] Data deduplication is the process of identifying and eliminating duplicate records in system operation data to ensure that each system operation data is unique. Data deduplication can be achieved through various algorithms, such as similarity-based matching or direct comparison of field values. Deduplicated system operation data can not only improve storage efficiency, but also reduce errors and confusion during system operation data analysis, thereby providing more accurate analysis results.

[0040] In one embodiment of the present invention, data compression may perform the following steps:

[0041] Step 1011: Record the first system operation data.

[0042] In this embodiment, the first system operation data is recorded to establish a benchmark. This benchmark system operation data serves as the starting point for subsequent data compression and can help analyze changes in subsequent system operation data. By saving the first system operation data, there is a clear reference point when comparing and calculating deviations, thereby ensuring the accuracy and reliability of system operation data processing.

[0043] Step 1012: for two adjacent system operation data, calculate the former system operation data with the latter system operation data to obtain a data deviation.

[0044] In this embodiment, by quantifying the difference between two adjacent system operation data as data deviation, the change pattern and trend of the system operation data can be identified, and these data deviations will provide key information for subsequent data compression.

[0045] Step 1013: Count the frequency of occurrence of statistical data deviations.

[0046] In this embodiment, the frequency of statistical data deviation is to understand which data deviations occur more frequently and which data deviations occur less frequently. Statistical frequency can help identify the importance and patterns in system operation data, thereby providing basic information for subsequent encoding processes. By focusing on high-frequency data deviations, the data compression algorithm can prioritize large changes, improve encoding efficiency, and ultimately achieve more effective compression effects.

[0047] Step 1014: construct a Huffman tree according to the frequency, and assign variable-length codes to the Huffman tree.

[0048] In this embodiment, the Huffman tree is constructed to achieve effective variable-length coding, so that common deviation data can be represented by shorter variable-length codes, while uncommon deviation data uses longer variable-length codes. This method uses statistical frequency to make variable-length coding have a higher compression rate, thereby reducing the system operation data storage requirements. The Huffman tree optimizes the representation of system operation data, improves compression efficiency and decoding speed.

[0049] Step 1015: Replace the data deviation with the variable-length code in the Huffman tree to obtain a code sequence.

[0050] In this embodiment, the variable-length code generated by the Huffman tree is used to replace the data deviation to form the final code sequence. The system operation data is converted into a more compact variable-length code, which significantly reduces the storage space requirement. The code sequence not only reduces the size of the system operation data, but also facilitates the improvement of efficiency in the transmission and storage of system operation data, making the subsequent system operation data processing, transmission and analysis more efficient.

[0051] Exemplarily, assume that there is a set of voltage data D=[230, 231, 232, 234, 235] in the system operation data, and the voltage data 230 is used as the first voltage data. The voltage data deviation (i.e., Delta coding) is calculated: Δ2=231-230=1, Δ3=232-231=1, Δ4=234-232=2, Δ5=235-234=1, and a data deviation sequence Δ=[1,1,2,1] is obtained; Huffman coding is performed based on the data deviation sequence, and the frequency of occurrence of the voltage data deviation is counted, where Δ=1 appears 3 times and Δ=2 appears 1 time. A Huffman tree is constructed based on the frequency, and variable-length coding is assigned to the Huffman tree. The variable-length coding of Δ=1 is 0, and the variable-length coding of Δ=2 is 10. The compressed coding sequence is: [0, 0, 10, 0].

[0052] In one embodiment of the present invention, when a user or an application needs to access the compressed system operation data, the compressed system operation data is first decompressed.

[0053] The specific decompression operation includes: reading the first system operation data and the coding sequence, decoding the coding sequence into multiple data deviations, starting with the first system operation data, adding the data deviations to the system operation data in sequence, and obtaining the system operation data. This process improves the integrity and accuracy of the system operation data, so that the compressed system operation data can effectively restore the original information, thereby facilitating further analysis and processing by users or applications.

[0054] Exemplarily, taking the compressed voltage data as an example, the first voltage data 230 is read, and the compressed coding sequence [0, 0, 10, 0] is read. Huffman decoding is performed to decode the coding sequence into multiple data deviations to obtain a data deviation sequence Δ=[1, 1, 2, 1]; after Huffman decoding, Delta decoding is performed, starting with the first voltage data, and the voltage data is sequentially added with the data deviation to obtain the voltage data before compression: D2=230+1=231, D2=231+1=232, D2=232+2=234, D2=234+1=235, and the voltage data sequence before compression D=[230, 231, 232, 234, 235] is restored.

[0055] Step 102: If the preprocessing is completed, the priority of the system operation data is determined.

[0056] In this embodiment, in the power Internet of Things, the system operation data collected in real time is preprocessed to improve the accuracy and consistency of the system operation data. After the preprocessing is completed, the priority of the system operation data is determined in order to effectively manage and optimize the storage of system operation data. For system operation data of different priorities, the matching storage locations are different, which is conducive to the allocation of resources and improves the operation efficiency and resource utilization of the power Internet of Things.

[0057] Specifically, the scoring mapping table is loaded in the power Internet of Things. The scoring mapping table is used to record the scoring criteria built based on the actual application scenarios of the power Internet of Things. The system operation data is mapped to the urgency score according to the scoring mapping table. The urgency score reflects the importance of the system operation data to the security and stability of the power Internet of Things; the historical access frequency of the system operation data is converted into an access frequency score, and the access frequency score indicates how frequently the system operation data is accessed; the first weight configured for the urgency score and the second weight configured for the access frequency score are queried, and the sum of the urgency score and the access score is calculated as the priority of the system operation data. The urgency score is the product of the urgency score and the first weight, and the access score is the product of the access frequency score and the second weight. The priority calculation not only helps optimize the storage and access strategies of the system operation data, but also allows key system operation data to be given priority in emergency situations, thereby improving the response capability and operation efficiency of the power Internet of Things. The priority calculation formula is as follows:

[0058] P=w u ·S u +w a ·S a ;

[0059] Where: P represents priority, S u Indicates the urgency score, S a represents the access frequency score, w u represents the first weight, w a represents the second weight, w u +w a =1,w u ·S u represents the urgency score, w a ·S a Indicates the access score.

[0060] Step 103: Store the system operation data in a storage level that matches the priority.

[0061] In this embodiment, the priority of system operation data is calculated in the power Internet of Things, and the system operation data is stored in a storage level that matches the priority, thereby optimizing the management and resource utilization of the system operation data. By classifying and matching the system operation data according to priority, the response capability of the power Internet of Things can be improved, ensuring that important data can be quickly obtained in an emergency.

[0062] The priorities of system operation data include first level, second level and third level. The priority of the first level is higher than that of the second level, and the priority of the second level is higher than that of the third level.

[0063] The storage tiers that match the priority include real-time storage tier, near-line storage tier, and offline storage tier. The real-time storage tier uses an in-memory database, the near-line storage tier is a storage device that combines solid-state drives and mechanical hard drives, and the offline storage tier is a tape library and / or object storage system.

[0064] Specifically, the system operation data belonging to the first level is stored in the real-time storage layer, the system operation data belonging to the second level is stored in the near-line storage layer, and the system operation data belonging to the third level is stored in the offline storage layer. By storing the system operation data according to priority, it can ensure that high-priority system data can be quickly accessed and processed, while reducing storage costs and improving the efficiency of system operation data management. This tiered storage strategy helps to optimize resource usage while ensuring the security and stability of the power Internet of Things.

[0065] The real-time storage layer stores the first-level system operation data that is more important to the security and stability of the power Internet of Things, such as real-time monitoring data, fault alarms, etc. This type of system operation data needs to be accessed quickly to enable timely response and decision-making to ensure the normal operation of the power Internet of Things.

[0066] The near-line storage layer stores the second-level system operation data, which still needs to be accessed regularly but is not as urgent as the first-level system operation data. The second-level system operation data includes historical records, daily operation data, etc., which can be used for regular analysis and reporting to support operational decisions.

[0067] The offline storage layer stores the third level of system operation data, such as archived historical data and backups. These system operation data are usually accessed infrequently, but can be used for long-term trend analysis, compliance checks, or data recovery when they are used.

[0068] Step 104: Adjust the priority of the system operation data according to the storage information and access information of the system operation data in the storage layer.

[0069] In this embodiment, the priority of system operation data is adjusted according to the storage information and access information of the system operation data in the storage hierarchy, which can improve the effective management of system operation data and the rational use of resources. Storage information reflects the importance and access frequency of system operation data, while access information reveals the actual usage and demand changes of system operation data. By combining these two types of storage information and access information, the power Internet of Things can dynamically adjust the priority of system operation data to optimize storage costs, improve access efficiency, and improve the timely response of key system operation data, while reducing the storage occupancy of low-priority data, thereby enhancing the performance and responsiveness of the overall power Internet of Things.

[0070] The storage information includes the score and storage duration calculated based on the storage duration of the system operation data.

[0071] The access information includes the access frequency score of the system operation data, the score calculated based on the last access time of the system operation data, the score calculated based on the update frequency of the system operation data, the access frequency, the access time, and the update frequency.

[0072] Specifically, the storage information and access information of the system operation data in the storage layer are substituted into the following formula to adjust the priority of the system operation data;

[0073]

[0074]

[0075]

[0076]

[0077] P new =w af ·AF score +w lat ·LAT score +w uf ·UF score +w st ·ST score ;

[0078] Among them, P new Indicates the adjusted priority, AF score Indicates the access frequency score of system operation data, LAT score Indicates the score calculated based on the last access time of the system operation data, UF score Indicates the score calculated based on the update frequency of system operation data, ST score Indicates the score calculated based on the storage duration of the system operation data, w afrepresents the visit frequency score weight, w lat represents the scoring weight calculated by the last access time of the system operation data, w uf represents the score weight calculated based on the update frequency of system operation data, w st Indicates the scoring weight calculated based on the storage duration of the system operation data. AF indicates the access frequency of the current system operation data. min Indicates the minimum access frequency of all system operation data within the preset monitoring period, AF max Indicates the maximum access frequency of all system operation data within the preset monitoring period. LAT indicates the last access time of the current system operation data. t indicates the current time. T indicates the preset monitoring period. UF indicates the update frequency of the current system operation data. max Indicates the maximum update frequency of all system operation data within the preset monitoring period, UF min Indicates the minimum update frequency of all system operations within the preset monitoring cycle. ST indicates the storage duration of the current system operation data. ST max Indicates the maximum storage duration of all system operation data within the preset monitoring period, ST min Indicates the minimum storage duration of all system operation data within the preset monitoring period.

[0079] Step 105: Migrate the system operation data to a storage tier that matches the adjusted priority.

[0080] In this embodiment, the priority of the system operation data is recalculated based on the performance of the system operation data in the storage layer to which it belongs, and the system operation data with changed priority is migrated to the storage layer that matches the priority, thereby realizing dynamic storage of the system operation data in the power Internet of Things, thereby optimizing the resource utilization of the power Internet of Things and the efficiency of system operation data management.

[0081] In one embodiment of the present invention, step 105 may include the following steps:

[0082] Step 1051: query the access frequency and / or update frequency of the system operation data at every predetermined time period.

[0083] In this embodiment, the access frequency and / or update frequency of the system operation data are queried at predetermined time intervals to continuously monitor the usage of the system operation data, which helps to identify which system operation data is frequently accessed or updated or which system operation data has not been accessed or updated for a long time, thereby providing a basis for subsequent storage management.

[0084] Step 1052: If the access frequency and / or update frequency changes, determine that the system operation data is an incremental data set.

[0085] In this embodiment, when the access frequency and / or update frequency changes, the system will determine that the relevant system operation data is an incremental data set. Frequency changes usually mean that the value or importance of the system operation data has changed, and higher or lower priority storage may be required. Timely identification of incremental data sets can optimize the management of system operation data and improve the responsiveness of the power Internet of Things. Whether the system operation data access frequency and / or update frequency has changed is indicated by the following set:

[0086] ΔD={d i |AF new (d i )≠AF old (d i )orUF new (d i )≠UF old (d i )};

[0087] Where: ΔD represents the incremental data set, AF new Indicates the system operation data at the end of the predetermined time period i The access frequency, AF old Indicates the system operation data at the start of the predetermined time period i Frequency of visits, UF new Indicates the system operation data at the end of the predetermined time period i Update frequency, UF old Indicates the system operation data at the beginning of the predetermined time period i The update frequency.

[0088] Step 1053: Migrate the incremental data set from the current storage tier to a storage tier that matches the adjusted priority in batches or in parallel.

[0089] In this embodiment, by migrating incremental data sets to the storage level that matches the adjusted priority in batches or in parallel, the efficiency of system operation data migration can be improved and the impact on the performance of the power Internet of Things can be reduced. This process ensures that incremental data sets can be quickly and effectively transferred to the appropriate storage level, ensuring the stability of the power Internet of Things and the availability of system operation data. At the same time, batch or parallel migration helps reduce potential risks and ensures the integrity and security of system operation data during the migration process.

[0090] Exemplarily, the storage tier of the incremental data set and the storage tier matched by the adjusted priority are represented by the following set:

[0091] M={(d i , Lsrc , L dst )|d i ∈ΔD and L dst ≠L src};

[0092] Where: M represents the set, d i represents the incremental data set, L src Indicates the incremental data set d i The current storage level, L dst Indicates the incremental data set d i Adjust the priority to match the storage tier.

[0093] Based on the set M, it is possible to intuitively understand to which storage dataset the incremental dataset needs to be migrated, and migrate the incremental dataset from the current storage tier to the storage tier matching the adjusted priority in batches or in parallel.

[0094] In an embodiment of the present invention, when the real-time system operation data is collected by the power Internet of Things, the system operation data is preprocessed. If the preprocessing is completed, the priority of the system operation data is determined, the system operation data is stored in a storage layer matching the priority, the priority of the system operation data is adjusted according to the storage information and access information of the system operation data in the storage layer, and the system operation data is migrated to the storage layer matching the adjusted priority. Performing preprocessing on the real-time system operation data collected by the power Internet of Things and determining its priority, and storing the system operation data in a matching storage layer can effectively improve the management efficiency and reliability of the system operation data. In this way, the power Internet of Things can give priority to processing and storing important and frequently accessed system operation data, thereby speeding up the retrieval and analysis of the system operation data. At the same time, the priority of the system operation data is dynamically adjusted according to the storage information and access information to ensure that the system operation data is always in a suitable storage layer, optimize the utilization of storage resources, reduce costs, and enhance the flexibility and responsiveness of the power Internet of Things, thereby improving the operating efficiency and stability of the power Internet of Things.

[0095] Embodiment 2

[0096] Figure 2 A schematic diagram of a data storage device for an electric power Internet of Things provided in Embodiment 2 of the present invention is shown in FIG. Figure 2 As shown, the device comprises:

[0097] The data initialization module 201 is used to perform preprocessing on the system operation data when real-time system operation data is collected from the power Internet of Things;

[0098] A priority determination module 202, configured to determine the priority of the system operation data if the preprocessing is completed;

[0099] A data storage module 203, used for storing the system operation data in a storage level matching the priority;

[0100] A priority adjustment module 204, configured to adjust the priority of the system operation data according to storage information and access information of the system operation data in the storage layer;

[0101] The data migration module 205 is used to migrate the system operation data to the storage level matching the adjusted priority.

[0102] In one embodiment of the present invention, the preprocessing includes data cleaning, data formatting, data compression and data deduplication.

[0103] In one embodiment of the present invention, the data initialization module 201 includes:

[0104] A data recording module, used for recording the first system operation data;

[0105] A data deviation acquisition module, for calculating the former system operation data with the latter system operation data for two adjacent system operation data, to obtain a data deviation;

[0106] A frequency statistics module, used to count the frequency of occurrence of the data deviation;

[0107] A Huffman tree construction module, used for constructing a Huffman tree according to the frequency and assigning variable-length codes to the Huffman tree;

[0108] The coding sequence acquisition module is used to replace the data deviation with the variable-length code in the Huffman tree to obtain a coding sequence.

[0109] In one embodiment of the present invention, the data initialization module 201 further includes:

[0110] A data reading module, used for reading the first system operation data and the coding sequence;

[0111] A coding sequence decoding module, used for decoding the coding sequence into a plurality of data deviations;

[0112] The data calculation module is used to start with the first system operation data, add the data deviation to the system operation data in sequence, and obtain the system operation data.

[0113] In one embodiment of the present invention, the priority determination module 202 includes:

[0114] A scoring mapping table loading module is used to load a scoring mapping table; the scoring mapping table is used to record the scoring criteria constructed based on the actual application scenario of the power Internet of Things;

[0115] A data mapping module, used for mapping the system operation data into an urgency score according to the score mapping table;

[0116] An access frequency score acquisition module, used to convert the historical access frequency of the system operation data into an access frequency score;

[0117] A weight query module, used to query a first weight configured for the urgency score and a second weight configured for the access frequency score;

[0118] The priority calculation module is used to calculate the sum of the urgency score and the access score as the priority of the system operation data; the urgency score is the product of the urgency score and the first weight; the access score is the product of the access frequency score and the second weight.

[0119] In one embodiment of the present invention, the priority includes a first level, a second level and a third level, the priority of the first level is higher than the priority of the second level, and the priority of the second level is higher than the priority of the third level; the storage layer includes a real-time storage layer, a near-line storage layer and an offline storage layer; the real-time storage layer adopts an in-memory database; the near-line storage layer is a storage device combining a solid-state hard disk and a mechanical hard disk; the offline storage layer is a tape library and / or an object storage system;

[0120] The data storage module 203 includes:

[0121] A first-level data storage module, used for storing the system operation data belonging to the first level to the real-time storage layer;

[0122] A second-level data storage module, used for storing the system operation data belonging to the second level to the near-line storage layer;

[0123] The third-level data storage module is used to store the system operation data belonging to the third level to the offline storage layer.

[0124] In one embodiment of the present invention, the priority adjustment module 204 includes:

[0125] A priority formula calculation module, used for substituting the storage information and access information of the system operation data in the storage layer into the following formula to adjust the priority of the system operation data;

[0126]

[0127]

[0128]

[0129]

[0130] P new =w af ·AF score +w lat ·LAT score +w uf ·UF score +w st ·ST score ;

[0131] Among them, P new Indicates the adjusted priority, AF score Indicates the access frequency score of the system operation data, LAT score Indicates the score calculated based on the last access time of the system operation data, UF score represents the score calculated based on the update frequency of the system operation data, ST score represents the score calculated based on the storage duration of the system operation data, w af 、w lat 、w uf 、w st Both represent weights. AF represents the access frequency of the current system operation data. min Indicates the minimum access frequency of all system operation data within the preset monitoring period, AF max represents the maximum access frequency of all the system operation data within the preset monitoring period, LAT represents the last access time of the current system operation data, t represents the current time, T represents the preset monitoring period, UF represents the update frequency of the current system operation data, UF max Indicates the maximum update frequency of all system operation data within the preset monitoring period, UF min Indicates the minimum update frequency of all system operations within the preset monitoring period. ST indicates the storage duration of the current system operation data. ST max Indicates the maximum storage duration of all system operation data within the preset monitoring period, ST min Indicates the minimum storage duration of all system operation data within the preset monitoring period.

[0132] In one embodiment of the present invention, the data migration module 205 includes:

[0133] A frequency query module, used for querying the access frequency and / or update frequency of the system operation data at intervals of a predetermined time period;

[0134] an incremental data set determination module, configured to determine that the system operation data is an incremental data set if the access frequency and / or the update frequency changes;

[0135] The incremental data set migration module is used to migrate the incremental data set from the current storage level to the storage level matching the adjusted priority level in batches or in parallel.

[0136] The data storage device of the electric power Internet of Things provided in the embodiment of the present invention can execute the data storage method of the electric power Internet of Things provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects for executing the data storage method of the electric power Internet of Things.

[0137] Embodiment 3

[0138] See also Figure 3 , showing a schematic diagram of the structure of a computer device provided by an embodiment of the present invention. The computer device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workbenches, personal digital assistants, blade servers, mainframe computers, and other suitable computers. The computer device may also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.

[0139] like Figure 3 As shown, the computer device 10 includes at least one processor 11, and a memory connected to the at least one processor 11, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 to the random access memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the computer device 10 can also be stored. The processor 11, ROM 12 and RAM 13 are connected to each other through a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0140] A number of components in the computer device 10 are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the computer device 10 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0141] The processor 11 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The processor 11 performs the various methods and processes described above, such as the data storage method of the power Internet of Things.

[0142] In some embodiments, the data storage method of the electric power Internet of Things may be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed on the computer device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of the data storage method of the electric power Internet of Things described above may be performed. Alternatively, in other embodiments, the processor 11 may be configured to execute the data storage method of the electric power Internet of Things in any other appropriate manner (e.g., by means of firmware).

[0143] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0144] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer program is executed by the processor, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0145] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in combination with an instruction execution system, device or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0146] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer device. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0147] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0148] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The client and server relationship is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.

[0149] Embodiment 4

[0150] An embodiment of the present invention further provides a computer program product, which includes a computer program. When the computer program is executed by a processor, it implements the data storage method of the power Internet of Things provided in any embodiment of the present invention.

[0151] In the process of implementation, the computer program product can be written in one or more programming languages ​​or a combination thereof to perform the computer program code of the present invention, including object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computer (for example, using an Internet service provider to connect through the Internet).

[0152] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.

[0153] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A data storage method for electric power Internet of Things, characterized in that: include: When real-time system operation data is collected from the power Internet of Things, preprocessing is performed on the system operation data; If the preprocessing is completed, determining the priority of the system operation data; Storing the system operation data in a storage level matching the priority; adjusting the priority of the system operation data according to storage information and access information of the system operation data in the storage layer; The system operation data is migrated to the storage tier that matches the adjusted priority.

2. The method according to claim 1, characterized in that: The preprocessing includes data cleaning, data formatting, data compression and data deduplication.

3. The method according to claim 2, characterized in that The preprocessing of the system operation data includes: Recording the first system operation data; For two adjacent system operation data, the latter system operation data is calculated with the former system operation data to obtain a data deviation; Count the frequency of occurrence of the data deviation; constructing a Huffman tree according to the frequency, and assigning variable-length codes to the Huffman tree; The variable-length code replaces the data deviation in the Huffman tree to obtain a code sequence.

4. The method according to claim 3, characterized in that The method further comprises: Reading the first system operation data and the coding sequence; Decoding the coded sequence into a plurality of the data deviations; Starting with the first system operation data, the system operation data is sequentially added with the data deviation to obtain the system operation data.

5. The method according to claim 1, characterized in that Determining the priority of the system operation data includes: Loading a scoring mapping table; the scoring mapping table is used to record the scoring criteria constructed based on the actual application scenario of the power Internet of Things; Mapping the system operation data into an urgency score according to the score mapping table; Converting the historical access frequency of the system operation data into an access frequency score; Querying a first weight configured for the urgency score and a second weight configured for the access frequency score; The sum of the urgency score and the access score is calculated as the priority of the system operation data; the urgency score is the product of the urgency score and the first weight; the access score is the product of the access frequency score and the second weight.

6. The method according to claim 1, characterized in that The priority includes the first level, the second level and the third level, the priority of the first level is higher than the priority of the second level, and the priority of the second level is higher than the priority of the third level; the storage level includes the real-time storage layer, the near-line storage layer and the offline storage layer; the real-time storage layer adopts the memory database; the near-line storage layer is a storage device combining a solid-state hard disk and a mechanical hard disk; The offline storage layer magnetic tape library and / or object storage system; The storing the system operation data into a storage level matching the priority level includes: Storing the system operation data belonging to the first level in the real-time storage layer; storing the system operation data belonging to the second level in the near-line storage layer; The system operation data belonging to the third level is stored in the offline storage layer.

7. The method according to claim 5, characterized in that The adjusting the priority of the system operation data according to the storage information and access information of the system operation data in the storage layer includes: Substituting the storage information and access information of the system operation data in the storage layer into the following formula to adjust the priority of the system operation data; P new =in af ·AF score +in lat ·YEARS score +in uf ·UF score +in st ·ST score ; Among them, P new Indicates the adjusted priority, AF score Indicates the access frequency score of the system operation data, LAT score Indicates the score calculated based on the last access time of the system operation data, UF score represents the score calculated based on the update frequency of the system operation data, ST score represents the score calculated based on the storage duration of the system operation data, w af 、w lat 、w uf 、w st Both represent weights. AF represents the access frequency of the current system operation data. min Indicates the minimum access frequency of all system operation data within the preset monitoring period, AF max represents the maximum access frequency of all the system operation data within the preset monitoring period, LAT represents the last access time of the current system operation data, t represents the current time, T represents the preset monitoring period, UF represents the update frequency of the current system operation data, UF max Indicates the maximum update frequency of all system operation data within the preset monitoring period, UF min Indicates the minimum update frequency of all system operations within the preset monitoring period. ST indicates the storage duration of the current system operation data. ST max Indicates the maximum storage duration of all system operation data within the preset monitoring period, ST min Indicates the minimum storage duration of all system operation data within the preset monitoring period.

8. The method according to any one of claims 1 to 7, characterized in that Migrating the system operation data to the storage level matching the adjusted priority level includes: Inquiring the access frequency and / or update frequency of the system operation data at predetermined time intervals; If the access frequency and / or the update frequency changes, determining that the system operation data is an incremental data set; The incremental data set is migrated from the current storage tier to the storage tier matching the adjusted priority in batches or in parallel.

9. A computer device, characterized in that: include: one or more processors; A storage device for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement a data storage method for an electric power Internet of Things as described in any one of claims 1-8.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, a data storage method for an electric power Internet of Things is implemented as described in any one of claims 1-8.

Images