A kaiwu database load information collection method

Abstract

The application discloses a KAIWU database load information collection method and relates to the technical field of data collection; load information is read from a KAIWU database by using a load information reading tool, and the load information is written into a BlockQueue; the load information is cached through the BlockQueue; the load information is read from the BlockQueue by using a data writing tool and is written into external storage; the collection and the persistence of the load information generated in the KAIWU database are solved; and the load information generated by the KAIWU database system is collected and stored persistently.

Description

Technical Field

[0001] This invention discloses a method relating to the field of data acquisition technology, specifically a method for acquiring load information from a KAIWU database. Background Technology

[0002] KAIWU database follows the Share-Nothing design philosophy, employing a fully peer-to-peer architecture that ensures every node participates in data storage and computation. It supports online horizontal scaling of nodes, achieving linear performance improvements while maintaining business continuity, and can support the storage and management of petabyte-scale data in the big data era. Furthermore, the database possesses cloud-native, multi-center, high-availability, and strong transaction consistency characteristics, supporting mixed row and column storage to meet the needs of HTAP scenarios. Currently, its business scope covers multiple industries including manufacturing, government, energy, and finance. Based on a distributed database-driven business data service platform, it focuses on rapidly developing areas such as the industrial internet and digital energy, leveraging a cloud-edge-device integrated architecture to vigorously promote the digital transformation of industrial enterprises and the construction of digital energy.

[0003] KAIWU database supports the collection of load information, but if the load information inside the KAIWU database is not read and persisted externally in a timely manner, data loss may occur. Summary of the Invention

[0004] This invention addresses the problems of existing technologies by providing a method for collecting load information from a KAIWU database. This method solves the problem of collecting and persisting load information generated within the KAIWU database, and collects and persists the load information generated by the KAIWU database system to external storage.

[0005] The specific solution proposed in this invention is as follows:

[0006] This invention provides a method for collecting load information from a KAIWU database. The method utilizes a load information reading tool to read load information from the KAIWU database and writes the load information into a BlockQueue. The load information reading tool includes the following method:

[0007] The `init()` method is used to initialize the necessary information required to connect to the KAIWU database.

[0008] The `getConnection()` method is used to create a connection to the KAIWU database.

[0009] The `loadCacheFromFile()` method is used to load the cluster's read status each time the load information reading tool starts.

[0010] The `getStartTimeFromCache()` method is used to retrieve the start time of each read from the KAIWU database.

[0011] The `getNextWorkloadInfo()` method is used to periodically read load information from the KAIWU database for a certain period of time and write the load information to a BlockQueue.

[0012] The saveCacheToFile() method is used to persist the cluster's load information reading status to a file after each load information is read from the KAIWU database.

[0013] Load information is cached using a BlockQueue, and load information is read from the BlockQueue and written to external storage using a data writing tool, wherein the data writing tool includes the following methods:

[0014] The `init()` method is used to initialize the necessary information required for linking to external storage.

[0015] The getConnection() method is used to create a connection to external storage.

[0016] The CheckKWDBWorkloadInfoValid() method is used to check whether the load information is valid.

[0017] The `parseAisDataFromWorkloadInfo()` method is used to parse the information that needs to be written to external storage from the load information.

[0018] The InsertAisData() method is used to write the information parsed by the parseAisDataFromWorkloadInfo() method to external storage.

[0019] Furthermore, the load information reading tool in the KAIWU database load information collection method also includes the getStartTimeFromKaiWuDataBase() method, which is used to obtain the generation time of the earliest load information when reading load information from the KAIWU database for the first time.

[0020] Furthermore, in the aforementioned KAIWU database load information collection method, the length of the BlockQueue is specified, and the put() method is used to put the load information read by the load information reading tool into the BlockQueue. When the load information inside the BlockQueue is full, the load information reading tool is blocked from continuing to write load information.

[0021] Furthermore, in the aforementioned KAIWU database load information collection method, the take() method is used to read load information from within the BlockQueue, and when there is no load information within the BlockQueue, the data writing tool is blocked from reading load information.

[0022] The present invention also provides a KAIWU database load information acquisition device, including a load information acquisition module, a queue cache module, and a data writing module.

[0023] The load information acquisition module uses a load information reading tool to read load information from the KAIWU database and writes the load information into a BlockQueue. The load information reading tool includes the following methods:

[0024] The `init()` method is used to initialize the necessary information required to connect to the KAIWU database.

[0025] The `getConnection()` method is used to create a connection to the KAIWU database.

[0026] The `loadCacheFromFile()` method is used to load the cluster's read status each time the load information reading tool starts.

[0027] The `getStartTimeFromCache()` method is used to retrieve the start time of each read from the KAIWU database.

[0028] The `getNextWorkloadInfo()` method is used to periodically read load information from the KAIWU database for a certain period of time and write the load information to a BlockQueue.

[0029] The saveCacheToFile() method is used to persist the cluster's load information reading status to a file after each load information is read from the KAIWU database.

[0030] The queue caching module caches load information through BlockQueue, and the data writing module reads load information from BlockQueue and writes it to external storage using a data writing tool, wherein the data writing tool includes the following methods:

[0031] The `init()` method is used to initialize the necessary information required for linking to external storage.

[0032] The getConnection() method is used to create a connection to external storage.

[0033] The CheckKWDBWorkloadInfoValid() method is used to check whether the load information is valid.

[0034] The `parseAisDataFromWorkloadInfo()` method is used to parse the information that needs to be written to external storage from the load information.

[0035] The InsertAisData() method is used to write the information parsed by the parseAisDataFromWorkloadInfo() method to external storage.

[0036] Furthermore, the load information acquisition module in the KAIWU database load information acquisition device also uses the getStartTimeFromKaiWuDataBase() method in the load information reading tool to obtain the generation time of the earliest load information when reading load information from the KAIWU database for the first time.

[0037] Furthermore, in the KAIWU database load information acquisition device, the queue cache module specifies the length of the BlockQueue, uses the put() method to put the load information read by the load information reading tool into the BlockQueue, and blocks the load information reading tool from continuing to write load information when the load information inside the BlockQueue is full.

[0038] Furthermore, in the KAIWU database load information acquisition device, the queue cache module uses the take() method to read load information from the BlockQueue, and when there is no load information in the BlockQueue, it blocks the data writing tool from reading the load information.

[0039] The advantages of this invention are:

[0040] This invention provides a method for collecting load information from a KAIWU database. A specific method using a load information reading tool reads the load information from the KAIWU database and exports it externally. A specific method using a data writing tool then parses, checks, transforms, and writes the data to external storage, achieving the transformation and permanent storage of the KAIWU database load information. Simultaneously, the collected load information can be used for other external analysis, machine learning, and other services. This solves the problem of data loss that occurs if the load information from the KAIWU database is not promptly read and persisted externally. Attached Figure Description

[0041] Figure 1 This is a schematic diagram of the application framework of the method of the present invention.

[0042] Figure 2 This is a schematic diagram of the KWDBReader implementation structure in this invention.

[0043] Figure 3 This is a schematic diagram of the implementation structure of BlockQueue in this invention.

[0044] Figure 4 The schematic diagram of the AISWriter implementation structure in this invention. Detailed Implementation

[0045] The present invention will be further described below with reference to the accompanying drawings and specific embodiments, so that those skilled in the art can better understand and implement the present invention. However, the embodiments described are not intended to limit the present invention.

[0046] This invention provides a method for collecting load information from a KAIWU database. The method utilizes a load information reading tool to read load information from the KAIWU database and writes the load information into a BlockQueue. The load information reading tool includes the following method:

[0047] The `init()` method is used to initialize the necessary information required to connect to the KAIWU database.

[0048] The `getConnection()` method is used to create a connection to the KAIWU database.

[0049] The `loadCacheFromFile()` method is used to load the cluster's read status each time the load information reading tool starts.

[0050] The `getStartTimeFromCache()` method is used to retrieve the start time of each read from the KAIWU database.

[0051] The `getNextWorkloadInfo()` method is used to periodically read load information from the KAIWU database for a certain period of time and write the load information to a BlockQueue.

[0052] The saveCacheToFile() method is used to persist the cluster's load information reading status to a file after each load information is read from the KAIWU database.

[0053] Load information is cached using a BlockQueue, and load information is read from the BlockQueue and written to external storage using a data writing tool, wherein the data writing tool includes the following methods:

[0054] The `init()` method is used to initialize the necessary information required for linking to external storage.

[0055] The getConnection() method is used to create a connection to external storage.

[0056] The CheckKWDBWorkloadInfoValid() method is used to check whether the load information is valid.

[0057] The `parseAisDataFromWorkloadInfo()` method is used to parse the information that needs to be written to external storage from the load information.

[0058] The InsertAisData() method is used to write the information parsed by the parseAisDataFromWorkloadInfo() method to external storage.

[0059] The method of this invention can continuously retrieve load information from the KAIWU database and persist the load information to external storage. In specific applications, some embodiments of the method of this invention can be referred to... Figures 1-4 The specific process is as follows:

[0060] Step 1: Use a load information reading tool called KWDBReader to read load information from the KAIWU database and write the load information to a BlockQueue. The init() method of the load information reading tool initializes the necessary information for connecting to the KAIWU database.

[0061] The getConnection() method is used to create a connection to the KAIWU database.

[0062] The `loadCacheFromFile()` method loads and caches the cluster's read status each time the load information reading tool starts.

[0063] The `getStartTimeFromCache()` method retrieves the start time of each read from the KAIWU database from the cache.

[0064] The `getNextWorkloadInfo()` method periodically reads load information from the KAIWU database for a given period and writes the load information to a BlockQueue.

[0065] The saveCacheToFile() method is used to persist the cluster's load information reading status to a file after each load information is read from the KAIWU database.

[0066] Furthermore, the getStartTimeFromKaiWuDataBase() method can be used to obtain the generation time of the earliest load information when the load information is read from the KAIWU database for the first time.

[0067] Step 2: Cache load information using BlockQueue, and read the load information from BlockQueue using a data writing tool called AISWriter and write it to external storage. Specifically, the Init() method of the data writing tool initializes the necessary information for connecting to external storage, the getConnection() method creates the connection to external storage, and the CheckKWDBWorkloadInfoValid() method checks the validity of the load information.

[0068] The `parseAisDataFromWorkloadInfo()` method is used to parse the information that needs to be written to external storage from the load information.

[0069] The InsertAisData() method is used to write the information parsed by the parseAisDataFromWorkloadInfo() method to external storage.

[0070] Furthermore, BlockQueue is a finite-length blocking queue. The length of BlockQueue is specified, and the put() method is used to put the load information read by the load information reading tool into BlockQueue. When the load information inside BlockQueue is full, the load information reading tool is blocked from continuing to write load information.

[0071] Furthermore, the `take()` method is used to read load information from within the BlockQueue, and when there is no load information within the BlockQueue, the data writing tool is blocked from reading load information.

[0072] The present invention also provides a KAIWU database load information acquisition device, including a load information acquisition module, a queue cache module, and a data writing module.

[0073] The load information acquisition module uses a load information reading tool to read load information from the KAIWU database and writes the load information into a BlockQueue. The load information reading tool includes the following methods:

[0074] The `init()` method is used to initialize the necessary information required to connect to the KAIWU database.

[0075] The `getConnection()` method is used to create a connection to the KAIWU database.

[0076] The `loadCacheFromFile()` method is used to load the cluster's read status each time the load information reading tool starts.

[0077] The `getStartTimeFromCache()` method is used to retrieve the start time of each read from the KAIWU database.

[0078] The `getNextWorkloadInfo()` method is used to periodically read load information from the KAIWU database for a certain period of time and write the load information to a BlockQueue.

[0079] The saveCacheToFile() method is used to persist the cluster's load information reading status to a file after each load information is read from the KAIWU database.

[0080] The queue caching module caches load information through BlockQueue, and the data writing module reads load information from BlockQueue and writes it to external storage using a data writing tool, wherein the data writing tool includes the following methods:

[0081] The `init()` method is used to initialize the necessary information required for linking to external storage.

[0082] The getConnection() method is used to create a connection to external storage.

[0083] The CheckKWDBWorkloadInfoValid() method is used to check whether the load information is valid.

[0084] The `parseAisDataFromWorkloadInfo()` method is used to parse the information that needs to be written to external storage from the load information.

[0085] The InsertAisData() method is used to write the information parsed by the parseAisDataFromWorkloadInfo() method to external storage.

[0086] The information interaction and execution process between the modules in the above-mentioned device are based on the same concept as the method embodiment of the present invention, and the specific details can be found in the description in the method embodiment of the present invention, and will not be repeated here.

[0087] Similarly, the device of this invention reads the load information from the KAIWU database to the external storage using a specific method of the load information reading tool, and then parses, checks, transforms, and writes it to external storage using a specific method of the data writing tool. This achieves the transformation and permanent storage of the KAIWU database load information, and the collected load information can also be used for other external analysis, machine learning, and other services. This solves the problem of data loss that occurs if the load information from the KAIWU database is not read and persisted externally in a timely manner, preventing the collected load information from becoming invalid, and also reducing the burden on the KAIWU database.

[0088] The above-described embodiments are merely preferred embodiments provided to fully illustrate the present invention, and the scope of protection of the present invention is not limited thereto. Equivalent substitutions or modifications made by those skilled in the art based on the present invention are all within the scope of protection of the present invention. The scope of protection of the present invention is defined by the claims.

Claims

1. A method for collecting load information from a KAIWU database, characterized in that: Load information is read from the KAIWU database using a load information reading tool and written to a BlockQueue. This load information reading tool includes the following methods: The `init()` method is used to initialize the necessary information required to connect to the KAIWU database. The `getConnection()` method is used to create a connection to the KAIWU database. The `loadCacheFromFile()` method is used to load the cluster's read status each time the load information reading tool starts. The `getStartTimeFromCache()` method is used to retrieve the start time of each read from the KAIWU database. The `getNextWorkloadInfo()` method is used to periodically read load information from the KAIWU database for a certain period of time and write the load information to a BlockQueue. The saveCacheToFile() method is used to persist the cluster's load information reading status to a file after each load information is read from the KAIWU database. Load information is cached using a BlockQueue, and load information is read from the BlockQueue and written to external storage using a data writing tool, wherein the data writing tool includes the following methods: The `init()` method is used to initialize the necessary information required for linking to external storage. The getConnection() method is used to create a connection to external storage. The CheckKWDBWorkloadInfoValid() method is used to check whether the load information is valid. The `parseAisDataFromWorkloadInfo()` method is used to parse the information that needs to be written to external storage from the load information. The InsertAisData() method is used to write the information parsed by the parseAisDataFromWorkloadInfo() method to external storage.

2. The method for collecting KAIWU database load information according to claim 1, characterized in that: The load information reading tool also includes the getStartTimeFromKaiWuDataBase() method, which is used to obtain the generation time of the earliest load information when reading load information from the KAIWU database for the first time.

3. The method for collecting KAIWU database load information according to claim 1, characterized in that: Specify the length of the BlockQueue, use the put() method to put the load information read by the load information reading tool into the BlockQueue, and block the load information reading tool from continuing to write load information when the BlockQueue is full.

4. A method for collecting KAIWU database load information according to any one of claims 1-3, characterized in that: The `take()` method is used to read load information from within the BlockQueue. If there is no load information within the BlockQueue, the data writing tool is blocked from reading load information.

5. A KAIWU database load information acquisition device, characterized in that... It includes a load information acquisition module, a queue caching module, and a data writing module. The load information acquisition module uses a load information reading tool to read load information from the KAIWU database and writes the load information into a BlockQueue. The load information reading tool includes the following methods: The `init()` method is used to initialize the necessary information required to connect to the KAIWU database. The `getConnection()` method is used to create a connection to the KAIWU database. The `loadCacheFromFile()` method is used to load the cluster's read status each time the load information reading tool starts. The `getStartTimeFromCache()` method is used to retrieve the start time of each read from the KAIWU database. The `getNextWorkloadInfo()` method is used to periodically read load information from the KAIWU database for a certain period of time and write the load information to a BlockQueue. The saveCacheToFile() method is used to persist the cluster's load information reading status to a file after each load information is read from the KAIWU database. The queue caching module caches load information through BlockQueue, and the data writing module reads load information from BlockQueue and writes it to external storage using a data writing tool, wherein the data writing tool includes the following methods: The `init()` method is used to initialize the necessary information required for linking to external storage. The getConnection() method is used to create a connection to external storage. The CheckKWDBWorkloadInfoValid() method is used to check whether the load information is valid. The `parseAisDataFromWorkloadInfo()` method is used to parse the information that needs to be written to external storage from the load information. The InsertAisData() method is used to write the information parsed by the parseAisDataFromWorkloadInfo() method to external storage.

6. The KAIWU database load information acquisition device according to claim 5, characterized in that: The load information acquisition module also uses the getStartTimeFromKaiWuDataBase() method in the load information reading tool to obtain the generation time of the earliest load information when reading load information from the KAIWU database for the first time.

7. A KAIWU database load information acquisition device according to claim 5, characterized in that it has a queue. The caching module specifies the length of the BlockQueue, uses the put() method to put the load information read by the load information reading tool into the BlockQueue, and blocks the load information reading tool from continuing to write load information when the BlockQueue is full.

8. A KAIWU database load information acquisition device according to any one of claims 5-7, characterized in that the queue... The caching module uses the take() method to read load information from within the BlockQueue, and when there is no load information within the BlockQueue, it blocks the data writing tool from reading load information.

Images