Implementation method, system and electronic device for short-term data writing library
An implementation method and data writing technology, which is applied in the field of power system information, can solve problems such as blocking and slow writing, achieve the effects of improving performance, avoiding synchronous blocking problems, and improving data processing and data writing performance
- Summary
- Abstract
- Description
- Claims
- Application Information
AI Technical Summary
Problems solved by technology
Method used
Image
Examples
Embodiment 1
[0045] Embodiment 1: The present invention proposes a method for implementing short-term data writing library, comprising the following steps:
[0046] Step 1: The process of set assembly according to the business demand plan calculation model is as follows:
[0047] S11, a virtual slot is established based on a foreign key field value of a business unit corresponding to a KEY, and the FNV (FowlerNolVo) algorithm is used to hash the field value. The FNV-1 algorithm can hash a large amount of data while maintaining a small conflict rate. The calculation process is,
[0048] initialization: hash=2166136261;
[0049] Enter the loop and determine the number of loops according to the length of the input data bytes. The hash result is a 32-bit unsigned integer, so it is necessary to discard the high bits and keep the low 32 bits: hash=(2166136261*16777619) mod 2 32 = 0x050c5d1f;
[0050] To perform XOR operation, first convert 0x56 to 32-bit value 0x00000056, and then perform XOR...
Embodiment 2
[0079] Embodiment 2: The present invention also proposes an implementation system for short-term data writing library, including:
[0080] The set assembly module is used to execute: according to the business demand plan, the billing model is assembled and assembled, and the billing result set is generated;
[0081] A parsing module for executing: parsing the billing result set and putting it into a memory queue;
[0082] A queue pulling module for executing: spin-pulling queue data from the memory queue;
[0083] A token access module for executing: creating an access token to connect to the database according to the queue data;
[0084] The archiving module is used to perform: batch archiving of multi-data sources of calculation collection;
[0085] The task judgment module is used to execute: judgment task return token;
[0086] The exception collection module is used to execute: establish a collection container for abnormal tasks of volume fee.
Embodiment 3
[0087] Embodiment 3: The present invention also proposes an electronic device, including a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that the processor implements the method when executing the program. step.
PUM
Login to View More Abstract
Description
Claims
Application Information
Login to View More 


