A method and apparatus for supporting memory data table monitoring

Abstract

The application provides a method and device for supporting memory data table monitoring, and the method comprises the following steps: defining 256 tables of a memory database Fcdb, and storing data by using a kv database; when data changes, the Fcdb detects whether a client is monitored on the table; if the table is not monitored by the client, no check is performed; if the table is monitored by the client, it is judged whether the value of a key prefix is null; if the value of the key prefix is null, it indicates that the client monitors the whole table, and the key and value are sent to the corresponding client; if the value of the key prefix is not null, it is judged whether the key of the table is monitored by the way of prefix matching; if the key of the table matches the monitoring, the key and value are informed to the corresponding monitoring client; if no match is found, the update is ignored. In this way, the memory database Fcdb stores data by using the kv database, and has high concurrent processing capacity; the memory database Fcdb checks the monitoring of the corresponding table and performs notification for the change of each key.

Description

Technical Field

[0001] The embodiments of the present invention generally relate to the field of in-memory database technology, and more particularly to a method and apparatus for supporting in-memory data table monitoring. Background Technology

[0002] With the continuous development of information technology, various types of databases have emerged. To manipulate data tables and data objects within these databases, database providers offer database-based operation actions, such as Data Manipulation Language (DML) operations, which can include insert, delete, and update operations. When these operations occur in a database, the database records the resulting operation log information. However, under the pressure of sudden high concurrency and high traffic, traditional relational databases become overwhelmed, slowing down system services and significantly impacting the system's concurrency capabilities. Furthermore, Redis database listeners only return keys, not values.

[0003] For example, the patent "A Data Consistency Method for Distributed Architecture Based on Zookeeper (Application No.: CN201910036845.9)" states that the invention includes the following steps: buffer data initialization: after each node in the distributed system starts, it loads the data in Zookeeper into the memory of each node; deployment of subscription listeners: a subscription listener method is defined, and each node subscribes to the Zookeeper directory and data, and deploys listeners; data synchronization: after detecting changes in data in Zookeeper, the memory data of each node is updated through the deployed listener method to maintain data synchronization between nodes. This invention uses the Zookeeper component as a cache data synchronization manager in the distributed architecture system, and utilizes Zookeeper's publish and subscribe mechanism to synchronize the cache data of distributed multi-nodes in real time. At the same time, the memory-based Zookeeper database greatly improves the data access speed, resulting in a significant improvement in performance and data consistency in the distributed architecture system. However, it does not solve the problems of system service slowdown and failure to return values ​​under instantaneous high concurrency and high traffic access pressure. Summary of the Invention

[0004] To address the above issues, the in-memory database Fcdb of this invention uses a key-value database to store data, which has high concurrency processing capabilities. For each key's addition, modification, deletion, and timeout, the in-memory database Fcdb checks the corresponding table's listener. If a match is found, a notification is sent, and the key+value is returned. In addition, prefix matching can be performed on the listened keys.

[0005] According to embodiments of the present invention, a method and apparatus for supporting memory data table monitoring are provided.

[0006] In a first aspect of the invention, a method for supporting in-memory data table monitoring is provided. The method includes:

[0007] S01: Define 256 tables for the in-memory database Fcdb, using a key-value database to store data;

[0008] S02: When the data changes, Fcdb checks if any client is listening on this table;

[0009] S03: If this table is not being monitored by the client, then no check will be performed;

[0010] S04: If this table is being monitored by a client, check if the value of the key prefix is ​​null. If the value of the key prefix is ​​null, it means that the client is monitoring the entire table, and send the key and value to the corresponding client.

[0011] S05: If the value of the key prefix is ​​not null, then the key of this table is determined by prefix matching;

[0012] S06: If the key in this table matches the listener, then notify the corresponding listener client of the key and value; otherwise, ignore the update.

[0013] Furthermore, the memory described in S01 contains 256 arrays, each array having a number with a value of 0 to 255.

[0014] Furthermore, the number value corresponds to the id value in the table.

[0015] Furthermore, the client described in S02 stores a client structure in the database, and the client structure has the function of listening to multiple tables.

[0016] Furthermore, the database contains a listening list, which includes the table ID and the listenens pointer that the table points to. If listenens is not null, it indicates that a client is listening to this table.

[0017] Furthermore, the prefix described in S02 is user-defined.

[0018] Furthermore, the Fcdb also has a monitoring function, the specific steps of which are as follows:

[0019] S011: Provide the table number to be monitored;

[0020] S012: Select whether to provide a key prefix;

[0021] S013: If a key prefix is ​​provided, Fcdb will determine the data change based on the key prefix;

[0022] S014: If no key prefix is ​​provided, Fcdb will listen to all keys.

[0023] In a second aspect of the invention, an apparatus for supporting memory data table monitoring is provided. The apparatus includes:

[0024] Definition module: Used to define the 256 tables of the in-memory database Fcdb, which uses a key-value database to store data;

[0025] Client monitoring and judgment module: When data changes, Fcdb checks whether a client is monitoring this table. If the table is not being monitored by a client, no check is performed.

[0026] The monitoring and judgment module is used to determine whether the key prefix value is null if the table is being monitored by a client. If the key prefix value is null, it means that the client is monitoring the entire table, and the key and value are sent to the corresponding client. If the key prefix value is not null, the module uses prefix matching to determine whether the key of the table is being monitored.

[0027] Listener matching module: If the key in this table matches the listener, the key and value are sent to the corresponding listener client; otherwise, the update is ignored.

[0028] Furthermore, the memory defined in the module contains 256 arrays, each with a number ranging from 0 to 255.

[0029] Furthermore, the number value corresponds to the id value in the table.

[0030] Furthermore, the client in the client listening and judgment module stores a client structure in the database, and the client structure has the function of listening to multiple tables.

[0031] Furthermore, the database contains a listening list, which includes the table ID and the listenens pointer that the table points to. If listenens is not null, it indicates that a client is listening to this table.

[0032] Furthermore, the prefix mentioned in the monitoring and judgment module is user-defined.

[0033] Furthermore, the Fcdb also includes:

[0034] Select module: Used to provide the table numbers to be monitored;

[0035] Judgment module: Used to select whether to provide a key prefix. If a key prefix is ​​provided, Fcdb will determine the data change based on the key prefix; if no key prefix is ​​provided, Fcdb will listen to all keys.

[0036] The above-mentioned English abbreviations are explained as follows:

[0037] Fcdb: In-memory database

[0038] Keyword

[0039] Value: Keyword

[0040] ID: Identity document, ID card number

[0041] Null: A value reserved in computation to indicate that a pointer does not refer to a valid object.

[0042] KV: Key-value database, a type of database that stores data in key-value pairs.

[0043] Redis: Remote Dictionary Server, is an open-source, ANSI C-written, network-enabled, in-memory or persistent, log-structured, key-value database that provides APIs for multiple languages.

[0044] The in-memory database Fcdb of this invention uses a key-value database to store data, which has high concurrency processing capabilities. For each key's addition, modification, deletion, and timeout, the in-memory database Fcdb checks the corresponding table's listener. If found, it notifies the user and returns the key+value. In addition, it can also perform prefix matching on the listened keys.

[0045] It should be understood that the description in the Summary of the Invention is not intended to limit the key or essential features of the embodiments of the present invention, nor is it intended to restrict the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0046] The above and other features, advantages, and aspects of the various embodiments of the present invention will become more apparent from the accompanying drawings and the following detailed description. Wherein:

[0047] Figure 1 A flowchart illustrating a method for supporting in-memory data table monitoring according to an embodiment of the present invention is shown;

[0048] Figure 2A block diagram of an apparatus supporting memory data table monitoring according to an embodiment of the present invention is shown. Detailed Implementation

[0049] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0050] According to embodiments of the present invention, a method and apparatus for supporting memory data table monitoring are proposed.

[0051] The principles and spirit of the present invention will be explained in detail below with reference to several representative embodiments.

[0052] Figure 1 This is a schematic flowchart of a method for supporting in-memory data table monitoring according to an embodiment of the present invention. The method includes:

[0053] S01: Define 256 tables for the in-memory database Fcdb, using a key-value database to store data;

[0054] S02: When the data changes, Fcdb checks if any client is listening on this table;

[0055] S03: If this table is not being monitored by the client, then no check will be performed;

[0056] S04: If this table is being monitored by a client, check if the value of the key prefix is ​​null. If the value of the key prefix is ​​null, it means that the client is monitoring the entire table, and send the key and value to the corresponding client.

[0057] S05: If the value of the key prefix is ​​not null, then the key of this table is determined by prefix matching;

[0058] S06: If the key in this table matches the listener, then notify the corresponding listener client of the key and value; otherwise, ignore the update.

[0059] It should be noted that although the operation of the method of the present invention has been described in a specific order in the above embodiments and figures, this does not require or imply that the operations must be performed in that specific order, or that all the operations shown must be performed to achieve the desired result. Additionally or alternatively, certain steps may be omitted, multiple steps may be combined into one step, and / or one step may be broken down into multiple steps.

[0060] To provide a clearer explanation of the above-described method for supporting in-memory data table monitoring, two specific embodiments are described below. However, it is worth noting that these embodiments are only for better illustrating the present invention and do not constitute an improper limitation of the present invention.

[0061] The following two specific examples will further illustrate the methods for supporting in-memory data table monitoring:

[0062] Example 1:

[0063] Define 256 tables in the in-memory database Fcdb, numbered from 0 to 255, corresponding to the table ID, and use a key-value database to store the data;

[0064] Each table corresponds to two pointers: one pointing to the hash table and the other pointing to the listener list (listens). This listener list points to the client's listen items. Each client stores a client structure in the database. A client can listen to multiple tables, hence the listener list. Each list item has the table ID being listened to and the listener list that the table points to.

[0065] The key of table0's abc was updated: the original value data 001 was updated to 100. Fcdb checks if a client is listening on this table, that is, it checks the value of listens pointed to by the table. If the value of listens is null, then this table is being listened to by a client. Then it checks the key prefix listenKey. If the value of listenKey is null, it means that the client is listening to the data of the entire table. The data is sent to the corresponding client and the key+value is returned.

[0066] The key of table10's 'ab' was updated: the original value '111' was changed to '010'. Fcdb checks if a client is listening on this table by checking the value of 'listens' pointed to by the table. If the value of 'listens' is not null, then the table is being listened to by a client. Next, it checks the key prefix 'listenKey'. If the value of 'listenKey' is not null, it compares 'listenKey' and 'key' using prefix matching. If 'listenKey' is 'a' and 'key' is 'ab', the prefix matches, the data is sent to the corresponding client, and key + value is returned.

[0067] The key for table 255, "abc", was updated: the original value 100 was changed to 111. Fcdb checks if a client is listening on this table by checking the value of `listens`, which the table points to. If `listens` is null, then the table is being listened to by a client. Next, it checks the prefix `listenKey`. If `listenKey` is not null, it compares `listenKey` and `key` using prefix matching. Since `listenKey` is "abd" and the prefix of `key` is "ab", the update is ignored.

[0068] Example 2:

[0069] The client starts, connects to Fcdb, and specifies that it should listen to table 0 with the key prefix listenKey set to null. It then selects to listen to table 255 with the prefix listenKey set to abd.

[0070] The key of 'ab' in table10 was updated: the original value '111' was updated to '010', the value of listenKey was found to be null, the key and value were sent to the corresponding client, and key+value was returned.

[0071] The key of table255 was updated: the original value data 100 was updated to 111. If the value of listenKey is not null, listenKey and key are compared by prefix matching. If listenKey is abd and key is abd, the prefix is ​​matched, the data is sent to the corresponding client, and key+value is returned.

[0072] Based on the same inventive concept, this invention also proposes a device for supporting in-memory data table monitoring. The implementation of this device can be found in the implementation of the method described above, and repeated details will not be elaborated further. Figure 2 As shown, the device 100 includes:

[0073] Define module 101: Used to define 256 tables for the in-memory database Fcdb, which uses a key-value database to store data;

[0074] Client listening and judgment module 102: When data changes, Fcdb checks whether a client is listening on this table. If the table is not being listened to by a client, no check is performed.

[0075] Listening and Judgment Module 103: If this table is being listened to by a client, it determines whether the value of the key prefix is ​​null. If the value of the key prefix is ​​null, it means that the client is listening to the entire table, and the key and value are sent to the corresponding client. If the value of the key prefix is ​​not null, it determines whether the key of this table is being listened to by prefix matching.

[0076] Listener matching module 104: If the key of this table matches the listener, the key and value are notified to the corresponding listener client; if no match is found, the update is ignored.

[0077] This invention proposes a device that supports in-memory data table monitoring. The in-memory database Fcdb uses a key-value database to store data, which has high concurrency processing capabilities. For each key's addition, modification, deletion, and timeout, the in-memory database Fcdb checks the corresponding table for monitoring. If a match is found, it notifies the user and returns the key+value pair. In addition, it can also perform prefix matching on the monitored keys.

[0078] While the spirit and principles of the invention have been described with reference to several specific embodiments, it should be understood that the invention is not limited to the disclosed specific embodiments, and the division of aspects does not imply that features in these aspects cannot be combined for benefit; such division is merely for ease of description. The invention is intended to cover various modifications and equivalent arrangements included within the spirit and scope of the appended claims.

[0079] Regarding the limitation of the scope of protection of this invention, those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art without creative effort based on the technical solution of this invention are still within the scope of protection of this invention.

Claims

1. A method for supporting in-memory data table monitoring, characterized in that, The method includes: S01: Define 256 tables for the in-memory database Fcdb, using a key-value database to store data; S02: When data changes, Fcdb checks if any client is listening on the table containing the data; S03: If the table to which the data belongs is not being monitored by the client, no check will be performed; S04: If the table to which the data belongs is being monitored by the client, then determine whether the value of the key prefix is ​​null. If the value of the key prefix is ​​null, it means that the client is monitoring the entire table, and send the key and value to the corresponding client. S05: If the value of the key prefix is ​​not null, then the key of the table to which the data belongs is determined by prefix matching; S06: If the key of the table to which the data belongs matches this listener, then notify the corresponding listener client of the key and value; if no match is found, then ignore this update.

2. The method for supporting in-memory data table monitoring according to claim 1, characterized in that, The memory described in S01 contains 256 arrays, each with a number ranging from 0 to 255.

3. The method for supporting in-memory data table monitoring according to claim 2, characterized in that, The number value mentioned corresponds to the id value in the table.

4. The method for supporting in-memory data table monitoring according to claim 1, characterized in that, The client described in S02 stores a client structure in the database, and the client structure has the function of listening to multiple tables.

5. A method for supporting in-memory data table monitoring according to claim 4, characterized in that, The database contains a listening list, which includes the table ID and the listenens pointer that the table points to. If listenens is not null, it means that a client is listening to this table.

6. The method for supporting in-memory data table monitoring according to claim 1, characterized in that, The prefix mentioned in S04 is user-defined.

7. The method for supporting in-memory data table monitoring according to claim 1, characterized in that, The aforementioned Fcdb also has a listening function, the specific steps of which are as follows: S011: Provide the table number to be monitored; S012: Select whether to provide a key prefix; S013: If a key prefix is ​​provided, Fcdb will determine the data change based on the value of the key prefix; S014: If no key prefix is ​​provided, Fcdb will listen to all keys.

8. A device supporting in-memory data table monitoring, characterized in that, The device includes: Definition module: Used to define the 256 tables of the in-memory database Fcdb, which uses a key-value database to store data; Client monitoring and judgment module: When data changes, Fcdb checks whether a client is monitoring the table to which the data belongs. If the table to which the data belongs is not being monitored by a client, no check is performed. The monitoring and judgment module is used to determine whether the value of the key prefix is ​​null if the table to which the data belongs is being monitored by the client. If the value of the key prefix is ​​null, it means that the client is monitoring the entire table, and the key and value are sent to the corresponding client. If the value of the key prefix is ​​not null, the module determines whether the key of this table is being monitored by prefix matching. Listener matching module: If the key of the table to which the data belongs matches this listener, the key and value are notified to the corresponding listener client; if no match is found, the update is ignored.

9. The apparatus for supporting memory data table monitoring according to claim 8, characterized in that, The memory defined in the module contains 256 arrays, each with a number ranging from 0 to 255.

10. The apparatus for supporting memory data table monitoring according to claim 9, characterized in that, The number value mentioned corresponds to the id value in the table.

11. The apparatus for supporting memory data table monitoring according to claim 8, characterized in that, The client in the client listening and judgment module stores a client structure in the database, and the client structure has the function of listening to multiple tables.

12. The apparatus for supporting memory data table monitoring according to claim 11, characterized in that, The database contains a listening list, which includes the table ID and the listenens pointer that the table points to. If listenens is not null, it means that a client is listening to this table.

13. The apparatus for supporting memory data table monitoring according to claim 8, characterized in that, The prefix mentioned in the monitoring and judgment module is user-defined.

14. The apparatus for supporting memory data table monitoring according to claim 8, characterized in that, The Fcdb also includes: Select module: Used to provide the table numbers to be monitored; Judgment module: Used to select whether to provide a key prefix. If a key prefix is ​​provided, Fcdb determines the data change based on the value of the key prefix; if no key prefix is ​​provided, Fcdb listens to all keys.

Images