Memory table storage method and system, electronic device and storage medium

Abstract

The application relates to a memory table storage method and system, an electronic device and a storage medium. The method comprises the following steps: if the timestamp of to-be-written data is greater than or equal to the maximum timestamp of written data in an ordered list, the to-be-written data is written into the ordered list, and the maximum timestamp is updated; if the timestamp of the to-be-written data is less than the maximum timestamp, the to-be-written data is determined as disordered data, the disordered data is written into an incremental list, and index information corresponding to the disordered data is inserted into an incremental index tree, the key in the index information is the timestamp of the disordered data, and the value in the index information comprises a first position of the disordered data in the incremental list and a second position that needs to be accessed before the first position, the second position being the position of target data in the ordered list. The method can improve the query performance of the memory table while guaranteeing the writing performance of the memory table.

Description

Technical Field

[0001] This application relates to the field of memory technology, and in particular to a memory table storage method, system, electronic device, and storage medium. Background Technology

[0002] With the rapid development of internet technology, devices and systems generate massive amounts of time-series data every moment, such as temperature, pressure, and vibration data collected in real time by sensors. Time-series data typically has the following characteristics: 1. High write throughput: For example, in the steel industry, which has a high level of automation and informatization, a large amount of time-series data is generated in a short period of time due to the high frequency of data collection during the production process; 2. Time-ordered query results: When querying time-series data in a time-series database, the retrieved data is ordered by time, and the time-series database usually needs to have the ability to filter duplicate timestamp data.

[0003] Currently, most time-series databases typically use LSM (Log Structured Merge) trees as their underlying storage engine. In-memory tables are a crucial component of the LSM tree storage engine, used to store data written by clients but not yet written to disk. When a memory table meets certain conditions, it becomes a read-only memory table, and the system writes the data from the read-only memory table to disk. Simultaneously, the LSM data storage engine creates a new memory table. Existing memory table structures include ordered skip lists and unordered memory tables. However, these memory table structures have poor write and query performance, thus impacting the overall performance of the time-series database. Summary of the Invention

[0004] This application provides a memory table storage method, system, electronic device, and storage medium to solve the technical problem of poor write and query performance of memory table structures in the prior art.

[0005] This application provides a memory table storage method, wherein the memory table includes: an ordered list, an incremental list, and an incremental index tree, and the method includes: If the timestamp of the data to be written is greater than or equal to the maximum timestamp of the data already written in the ordered list, then the data to be written is written to the ordered list and the maximum timestamp is updated. If the timestamp of the data to be written is less than the maximum timestamp, then the data to be written is determined to be out-of-order data. The out-of-order data is written to the incremental list, and index information corresponding to the out-of-order data is inserted into the incremental index tree. The key in the index information is the timestamp of the out-of-order data, and the value in the index information includes the first position of the out-of-order data in the incremental list and the second position that needs to be accessed before accessing the first position. The second position is the position of the target data in the ordered list, and the timestamp of the target data is the adjacent timestamp of the current out-of-order data and is less than the timestamp of the current out-of-order data.

[0006] In some embodiments of this application, the method further includes: Receive data query instructions; Based on the data query instruction, the ordered list, the incremental list, and the incremental index tree are accessed to read the data in an ordered manner, thereby obtaining an ordered distribution list of the data written in the memory table. Based on the ordered distribution list, data queries are performed to obtain and return the data query results.

[0007] In some embodiments of this application, both the first position and the second position are stored in the form of position IDs, where the position ID corresponding to the first position is a first ID and the position ID corresponding to the second position is a second ID; By accessing the ordered list, the incremental list, and the incremental index tree, data is read in an ordered manner to obtain an ordered distribution list of data written to the memory table, including: The minimum timestamp in the incremental index tree is determined as the target timestamp; Obtain the second ID corresponding to the target timestamp from the incremental index tree, obtain the target ID of the read position in the ordered list, if the target ID is empty, or the target ID is less than the second ID corresponding to the target timestamp, then read the written data in the ordered list in order, store the read data in a preset ordered distribution list, and update the read position until the target ID of the read position is greater than or equal to the second ID corresponding to the target timestamp; If the target ID of the read position is greater than or equal to the second ID corresponding to the target timestamp, then according to the first ID corresponding to the target timestamp, read the out-of-order data corresponding to the target timestamp from the incremental list and write the out-of-order data corresponding to the target timestamp into the ordered distribution list; The timestamps in the incremental index tree are traversed to complete the filling of the ordered distribution list. If all the data in the incremental list has been written into the ordered distribution list, and there is data in the ordered list that has not been written into the ordered distribution list, then the remaining data in the ordered list is written into the ordered distribution list.

[0008] In some embodiments of this application, if the target ID of the read position is greater than or equal to the second ID corresponding to the target timestamp, then according to the first ID corresponding to the target timestamp, the out-of-order data corresponding to the target timestamp is read from the incremental list, and the out-of-order data corresponding to the target timestamp is written into the ordered distribution list, including: If the target ID of the read position is greater than the second ID corresponding to the target timestamp, and the timestamp of the read position is equal to the target timestamp, then according to the first ID corresponding to the target timestamp, the out-of-order data corresponding to the target timestamp is read from the incremental list, and the latest data written in the ordered distribution list is updated to the out-of-order data corresponding to the target timestamp.

[0009] In some embodiments of this application, by accessing the ordered list, the incremental list, and the incremental index tree, data is read in an ordered manner to obtain an ordered distribution list of data written to the memory table, which further includes: If the incremental index tree is empty, data is read directly from the ordered list in sequence, and the read data is written into the ordered distribution list.

[0010] In some embodiments of this application, the method further includes: Obtain the memory usage data of the memory table, wherein the memory usage data is the sum of the memory usage of the ordered list, the incremental list, and the incremental index tree; If the memory usage data exceeds a preset memory usage threshold, and / or the lifespan of the data written in the memory table exceeds a preset lifespan threshold, then the data in the ordered distribution list will be written to the target disk. Upon completion of data writing to the target disk, the ordered list and the incremental list are released to reset the array resources occupied by the ordered list and the incremental list. The reset array resources are returned to the preset memory management module. Furthermore, the incremental index tree is released to reset the node resources occupied by the incremental index tree. The reset node resources are returned to the memory management module.

[0011] In some embodiments of this application, before writing the data to be written into the ordered list, the method further includes: The ordered list, the incremental list, and the incremental index tree are initialized. The initialization includes: requesting a fixed-length array resource from a preset memory management module to complete the initialization of the ordered list and the incremental list; and requesting multiple node resources from the memory management module to complete the initialization of the incremental index tree. The method further includes: if the array resources allocated to the ordered list are already full, and the array resources allocated to the current ordered list are less than a preset first resource threshold, then request additional array resources from the memory management module to complete the expansion of the ordered list; if the array resources allocated to the incremental list are already full, and the array resources allocated to the current incremental list are less than a preset second resource threshold, then request additional array resources from the memory management module to complete the expansion of the incremental list; if the node resources allocated to the incremental index tree are already full, and the node resources allocated to the current incremental index tree are less than a preset node resource threshold, then request additional node resources from the memory management module to complete the expansion of the incremental index tree.

[0012] This application also provides a memory table storage system, wherein the memory table includes: an ordered list, an incremental list, and an incremental index tree, and the system includes: An ordered write module is used to write the data to be written into the ordered list and update the maximum timestamp if the timestamp of the data to be written is greater than or equal to the maximum timestamp of the data already written in the ordered list. An incremental write module is configured to determine that the data to be written is out of order if the timestamp of the data to be written is less than the maximum timestamp, write the out-of-order data into the incremental list, and insert index information corresponding to the out-of-order data into the incremental index tree. The key in the index information is the timestamp of the out-of-order data, and the value in the index information includes the first position of the out-of-order data in the incremental list and the second position that needs to be accessed before accessing the first position. The second position is the position of the target data in the ordered list, and the timestamp of the target data is the adjacent timestamp of the current out-of-order data and is less than the timestamp of the current out-of-order data.

[0013] This application also provides an electronic device, including a processor, a memory, and a communication bus; the communication bus is used to connect the processor and the memory; the processor is used to execute a computer program stored in the memory to implement the memory table storage method as described in any of the preceding claims.

[0014] This application also provides a computer-readable storage medium having a computer program stored thereon, the computer program being used to cause a computer to perform the memory table storage method as described in any of the preceding claims.

[0015] The beneficial effects of this application are as follows: This application provides a special in-memory table structure (the in-memory table includes an ordered list, an incremental list, and an incremental index tree), which optimizes the in-memory table structure of the storage engine for time-series databases. Based on this in-memory table structure, it provides an in-memory table storage method, system, electronic device, and storage medium. In this method, if the timestamp of the data to be written is greater than or equal to the maximum timestamp of the data already written in the ordered list, the data to be written is written to the ordered list, and the maximum timestamp is updated. If the timestamp of the data to be written is less than the maximum timestamp, the data to be written is determined to be out-of-order data, and the out-of-order data is written to the incremental list. Index information corresponding to the out-of-order data is inserted into the incremental index tree. The key in the index information is the timestamp of the out-of-order data, and the value in the index information includes the first position of the out-of-order data in the incremental list and the second position that needs to be accessed before accessing the first position. The second position is the position of the target data in the ordered list, and the timestamp of the target data is the adjacent timestamp of the current out-of-order data and is less than the current out-of-order data timestamp. Through this method, the query performance of the in-memory table can be improved while ensuring the write performance of the in-memory table. Understandably, compared to non-ordered data structures like in-memory tables (such as simple lists, which do not guarantee order over time, requiring copying and sorting of the copied list for querying, resulting in low query efficiency), the aforementioned in-memory table storage method only needs to read data sequentially based on the first and second positions in the index information, eliminating the need for list copying and sorting, thus significantly improving query performance. Compared to ordered data structures like skip lists (a space-for-time data structure that uses multiple ordered linked lists for binary search, but requires more garbage collection due to the creation / destruction of numerous linked list nodes, impacting write performance), the aforementioned in-memory table storage method allows most data to be written to an ordered list, with only a small portion of out-of-order data written to an incremental list, generating corresponding index information (which requires node resources). This reduces node resource consumption, thereby improving the write performance of the in-memory table. Attached Figure Description

[0016] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application. It is obvious that the drawings described below are merely some embodiments of this application, and those skilled in the art can obtain other drawings based on these drawings without any inventive effort.

[0017] In the attached diagram: Figure 1 A flowchart illustrating a memory table storage method provided in an embodiment of this application; Figure 2 This is a schematic diagram of the process of reading the memory table in a memory table storage method provided in one embodiment of this application; Figure 3 This is a schematic diagram of the structure of a memory table storage system provided in one embodiment of this application; Figure 4 This is another schematic diagram of the memory table storage system provided in one embodiment of this application; Figure 5 This is a schematic diagram of the structure of an electronic device provided in one embodiment of this application. Detailed Implementation

[0018] The following specific examples illustrate the implementation of this application. Those skilled in the art can easily understand other advantages and effects of this application from the content disclosed in this specification. This application can also be implemented or applied through other different specific embodiments. Various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of this application. In the absence of conflict, the following embodiments and features in the embodiments can be combined with each other.

[0019] It should be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of this application. The drawings only show the components related to this application and are not drawn according to the actual number, shape and size of the components in the actual implementation. In the actual implementation, the form, quantity and proportion of each component can be arbitrarily changed, and the layout of the components may also be more complex.

[0020] In the following description, numerous details are explored to provide a more thorough explanation of embodiments of the present application. However, it will be apparent to those skilled in the art that embodiments of the present application may be practiced without these specific details. In other embodiments, well-known structures and devices are shown in block diagram form rather than in detail to avoid obscuring embodiments of the present application.

[0021] Please refer to Figure 1 , Figure 1 This is a flowchart illustrating a memory table storage method provided in an embodiment of this application. The memory table includes: an ordered list, an incremental list, and an incremental index tree, as shown below. Figure 1 As shown, the method includes: S110: If the timestamp of the data to be written is greater than or equal to the maximum timestamp of the data already written in the ordered list, then the data to be written is written to the ordered list and the maximum timestamp is updated.

[0022] In some examples of this embodiment, the timestamp is represented in numerical form. The earlier the time, the smaller the corresponding timestamp value, and vice versa.

[0023] In some examples of this embodiment, if the maximum timestamp of the data already written in the ordered list cannot be obtained, such as when the ordered list is empty, the data to be written is directly written to the ordered list, and the maximum timestamp of the ordered list is updated.

[0024] In some examples of this embodiment, the data to be written can be temperature data, pressure data, vibration data, and other data collected by the sensor that need to be stored in a time-series database. There is no restriction on the specific type of data to be written here.

[0025] Understandably, by adopting the above steps, most of the data to be written can be written into an ordered list, which helps to improve writing efficiency.

[0026] In some examples of this embodiment, the incremental index tree can be an incremental index B+ tree.

[0027] S120: If the timestamp of the data to be written is less than the maximum timestamp, then the data to be written is determined to be out-of-order data. The out-of-order data is written into the incremental list, and index information corresponding to the out-of-order data is inserted into the incremental index tree. The key in the index information is the timestamp of the out-of-order data. The value in the index information includes the first position of the out-of-order data in the incremental list and the second position that needs to be accessed before accessing the first position. The second position is the position of the target data in the ordered list. The timestamp of the target data is the adjacent timestamp of the current out-of-order data and is less than the current out-of-order data timestamp.

[0028] In some examples of this embodiment, the determination of target data includes: obtaining the timestamp of data already written in the ordered list; if the timestamp of data already written in the ordered list is less than the timestamp of the current out-of-order data, then the written data is determined as prior data; by filtering multiple prior data, the prior data whose timestamp is closest to the timestamp of the current out-of-order data is obtained, and the timestamp of the prior data is the adjacent timestamp of the current out-of-order data; and the prior data is determined as target data.

[0029] Understandably, due to network latency, data entry delays, and other reasons, the timestamps of a small amount of data to be written may be less than the maximum timestamp of the data already written in the ordered list. Therefore, the above embodiment identifies such data as out-of-order data, stores the out-of-order data in the incremental list, and inserts the corresponding index information in the incremental index tree, which can effectively manage out-of-order data and avoid rebuilding tables.

[0030] It is understandable that by adopting the above method, it is easier to read the data in the ordered list and the incremental list in an orderly manner based on the index information in the incremental index tree, which helps to improve reading efficiency and data query efficiency.

[0031] In some embodiments, the method further includes: 1. Receive data query instructions.

[0032] In some examples of this embodiment, the data query command can be a command sent by an external terminal device, such as a mobile phone or a computer. The data query command can specify the time range for the query, etc.

[0033] 2. Based on the data query instruction, by accessing the ordered list, the incremental list, and the incremental index tree, the data is read in an ordered manner to obtain an ordered distribution list of the data written in the memory table.

[0034] Understandably, by providing the structure of the memory table described above, it is easier to read data in an orderly manner.

[0035] 3. Based on the ordered distribution list, perform data query, obtain data query results, and provide feedback.

[0036] For example, based on the ordered distribution list, data can be queried to obtain data stored in a memory table within a certain time period.

[0037] In some embodiments, both the first position and the second position are stored in the form of position IDs (identifiers), where the position ID corresponding to the first position is a first ID (deltaID) and the position ID corresponding to the second position is a second ID (stableID).

[0038] In some embodiments, by accessing the ordered list, the incremental list, and the incremental index tree, data is read in an ordered manner to obtain an ordered distribution list of data written to the memory table, including: 1. Determine the minimum timestamp in the incremental index tree as the target timestamp.

[0039] In some examples of this embodiment, the smallest unread timestamp in the incremental index tree can be determined as the target timestamp, and subsequent data reading can be performed based on the target timestamp.

[0040] 2. Obtain the second ID corresponding to the target timestamp from the incremental index tree, obtain the target ID of the read position in the ordered list, if the target ID is empty, or the target ID is less than the second ID corresponding to the target timestamp, then read the written data in the ordered list in order, store the read data in a preset ordered distribution list, and update the read position until the target ID of the read position is greater than or equal to the second ID corresponding to the target timestamp.

[0041] 3. If the target ID of the read position is greater than or equal to the second ID corresponding to the target timestamp, then according to the first ID corresponding to the target timestamp, read the out-of-order data corresponding to the target timestamp from the incremental list and write the out-of-order data corresponding to the target timestamp into the ordered distribution list.

[0042] 4. Traverse the timestamps in the incremental index tree to complete the filling of the ordered distribution list. If all the data in the incremental list has been written into the ordered distribution list, and there is data in the ordered list that has not been written into the ordered distribution list, then write the remaining data in the ordered list into the ordered distribution list.

[0043] Specifically, after writing the out-of-order data corresponding to the target timestamp into the ordered distribution list, the minimum timestamp is determined from the timestamps that have not been read in the incremental index tree, and subsequent data reading is performed based on the minimum timestamp.

[0044] Understandably, the above reading method is easy to implement and has high reading efficiency.

[0045] Figure 2 This is a schematic diagram illustrating the process of reading the memory table in a memory table storage method provided in one embodiment of this application. Please refer to... Figure 2 , Figure 2 The text provides examples of ordered lists, incremental lists, and incremental index trees. Both ordered lists and incremental lists include timestamps and corresponding values. Figure 2 The rightmost table is the ordered distribution list that was read. Figure 2 The ordered lists, incremental lists, and populated data in the incremental index tree are merely examples and have no actual meaning; they will not be elaborated upon here. Figure 2 As shown, during a read operation, the smallest timestamp is first determined from the timestamps that have not been read in the incremental index tree (the keys in the index information). Figure 2 The minimum timestamp is 2. Next, obtain the second ID (stableID) corresponding to this minimum timestamp, such as... Figure 2As shown, the stableID corresponding to the minimum timestamp is 0, indicating that before reading the out-of-order data corresponding to the current minimum timestamp, it is necessary to read the data at position 0 of the ordered list, i.e., the data "Timestamp=1, Value=60" in the ordered list. This data is then written to the ordered distribution list. Afterwards, the first ID (deltaID) corresponding to the minimum timestamp is found from the incremental index tree. Figure 2 The minimum timestamp corresponds to a deltaID of 1. Next, the data "Timestamp=2, Value=10" with position ID 1 is read from the incremental list and written to the ordered distribution list. Then, the minimum timestamp is re-determined from the timestamps not read in the incremental index tree, and the above process is repeated, and so on. Figure 2 In the incremental index tree, "root" represents the root node, and all nodes below it are leaf nodes. Each leaf node includes index information for out-of-order data.

[0046] In some embodiments, if the target ID of the read position is greater than or equal to the second ID corresponding to the target timestamp, then according to the first ID corresponding to the target timestamp, the out-of-order data corresponding to the target timestamp is read from the incremental list, and the out-of-order data corresponding to the target timestamp is written into the ordered distribution list, including: If the target ID of the read position is greater than the second ID corresponding to the target timestamp, and the timestamp of the read position is equal to the target timestamp, then according to the first ID corresponding to the target timestamp, the out-of-order data corresponding to the target timestamp is read from the incremental list, and the latest data written in the ordered distribution list is updated to the out-of-order data corresponding to the target timestamp.

[0047] Understandably, in the case where "the timestamp of the read position is equal to the target timestamp", this embodiment uses the out-of-order data in the incremental list as the standard to better adapt to the application scenario of data supplementation.

[0048] In some embodiments, by accessing the ordered list, the incremental list, and the incremental index tree, data is read in an ordered manner to obtain an ordered distribution list of data written to the memory table, which further includes: If the incremental index tree is empty, data is read directly from the ordered list in sequence, and the read data is written into the ordered distribution list.

[0049] Understandably, this method can help improve reading and query efficiency.

[0050] In some embodiments, the method further includes: 1. Obtain the memory usage data of the memory table, wherein the memory usage data is the sum of the memory usage of the ordered list, the incremental list, and the incremental index tree.

[0051] 2. If the memory usage data exceeds the preset memory usage threshold, and / or the lifespan of the data written in the memory table exceeds the preset lifespan threshold, then the data in the ordered distribution list will be written to the target disk.

[0052] Understandably, the above method can effectively complete the disk flushing process.

[0053] Third, after completing the data writing to the target disk, release the ordered list and the incremental list to reset the array resources occupied by the ordered list and the incremental list, return the reset array resources to the preset memory management module, and release the incremental index tree to reset the node resources occupied by the incremental index tree, return the reset node resources to the memory management module.

[0054] Understandably, resetting resources allows the memory management module to reuse the reset resources, minimizing garbage collection and avoiding frequent requests for memory from the operating system or virtual machine, thus improving read and write performance, i.e., write and query performance.

[0055] In some embodiments, before writing the data to be written to the ordered list, the method further includes: The ordered list, the incremental list, and the incremental index tree are initialized. The initialization includes: requesting a fixed-length array resource from a preset memory management module to complete the initialization of the ordered list and the incremental list; and requesting multiple node resources from the memory management module to complete the initialization of the incremental index tree.

[0056] In some embodiments, the method further includes: if the array resources allocated to the ordered list are already full, and the array resources allocated to the current ordered list are less than a preset first resource threshold, then requesting additional array resources from the memory management module to complete the expansion of the ordered list; if the array resources allocated to the incremental list are already full, and the array resources allocated to the current incremental list are less than a preset second resource threshold, then requesting additional array resources from the memory management module to complete the expansion of the incremental list; if the node resources allocated to the incremental index tree are already full, and the node resources allocated to the current incremental index tree are less than a preset node resource threshold, then requesting additional node resources from the memory management module to complete the expansion of the incremental index tree.

[0057] In some examples of this embodiment, the fixed length can be set according to the actual situation. When requesting more resources, that is, when requesting expansion, resources of twice the fixed length can be requested.

[0058] It should be mentioned that in the above embodiments, when writing data to the list, the data is inserted at the end of the list to complete the data writing.

[0059] Please refer to Figure 3 This embodiment also provides a memory table storage system, wherein the memory table includes: an ordered list, an incremental list, and an incremental index tree, and the system includes: The ordered writing module 310 is used to write the data to be written into the ordered list and update the maximum timestamp if the timestamp of the data to be written is greater than or equal to the maximum timestamp of the data already written in the ordered list. The incremental write module 320 is used to determine that the data to be written is out of order if the timestamp of the data to be written is less than the maximum timestamp, write the out-of-order data into the incremental list, and insert index information corresponding to the out-of-order data into the incremental index tree. The key in the index information is the timestamp of the out-of-order data, and the value in the index information includes the first position of the out-of-order data in the incremental list and the second position that needs to be accessed before accessing the first position. The second position is the position of the target data in the ordered list, and the timestamp of the target data is the adjacent timestamp of the current out-of-order data and is less than the timestamp of the current out-of-order data.

[0060] It should be noted that the memory table storage method and memory table storage system provided in the above embodiments belong to the same concept. The specific methods of operation of each module have been described in detail in the method embodiments and will not be repeated here. In practical applications, the memory table storage system provided in the above embodiments can be assigned to different functional modules as needed, that is, the internal structure of the system can be divided into different functional modules to complete all or part of the functions described above. This is not a limitation here.

[0061] Figure 4 This is another schematic diagram of the memory table storage system provided in one embodiment of this application. Please refer to it. Figure 4 , Figure 4In this embodiment, MemTable represents a memory table. The memory table storage system further includes a reading module 410, used to determine the minimum timestamp in the incremental index tree as the target timestamp; obtain the second ID corresponding to the target timestamp from the incremental index tree; obtain the target ID of the read position in the ordered list; if the target ID is empty, or the target ID is less than the second ID corresponding to the target timestamp, then read the written data in the ordered list in sequence, store the read data in a preset ordered distribution list, and update the read position until the target ID of the read position is greater than or equal to the target timestamp. The second ID corresponding to the timestamp; if the target ID of the read position is greater than or equal to the second ID corresponding to the target timestamp, then according to the first ID corresponding to the target timestamp, read the out-of-order data corresponding to the target timestamp from the incremental list, and write the out-of-order data corresponding to the target timestamp into the ordered distribution list; traverse the timestamps in the incremental index tree to complete the filling of the ordered distribution list; if all the data in the incremental list has been written into the ordered distribution list, and there is data in the ordered list that has not been written into the ordered distribution list, then write the remaining data in the ordered list into the ordered distribution list; The memory management module 420 provides fixed-length array resources during the initialization of the ordered list and incremental list, and multiple node resources during the initialization of the incremental index tree. When the ordered list and incremental list require expansion, it provides new array resources for expansion; when the incremental index tree requires expansion, it provides new node resources for expansion. In essence, the memory management module 420 manages and reuses array and node resources.

[0062] It's understandable; after the data is flushed, it will be stored as a file on the target disk.

[0063] In some embodiments, an electronic device is also provided, which may be a server, and its internal structure diagram is shown below. Figure 5 As shown, the electronic device includes a processor, memory, network interface, and database connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile and / or volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The network interface is used to communicate with external clients via a network connection. When the computer program is executed by the processor, it implements the functions or steps of the server-side method described above.

[0064] In some embodiments, an electronic device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it performs the following steps: if the timestamp of the data to be written is greater than or equal to the maximum timestamp of the data already written in the ordered list, the data to be written is written to the ordered list, and the maximum timestamp is updated; if the timestamp of the data to be written is less than the maximum timestamp, the data to be written is determined to be out-of-order data, the out-of-order data is written to the incremental list, and index information corresponding to the out-of-order data is inserted into the incremental index tree. The key in the index information is the timestamp of the out-of-order data, and the value in the index information includes a first position of the out-of-order data in the incremental list and a second position that needs to be accessed before accessing the first position. The second position is the position of the target data in the ordered list, and the timestamp of the target data is the adjacent timestamp of the current out-of-order data and is less than the current timestamp of the out-of-order data.

[0065] In some embodiments, a computer-readable storage medium is provided having a computer program stored thereon. When the computer program is executed by a processor, it performs the following steps: if the timestamp of the data to be written is greater than or equal to the maximum timestamp of the data already written in the ordered list, then the data to be written is written to the ordered list, and the maximum timestamp is updated; if the timestamp of the data to be written is less than the maximum timestamp, then the data to be written is determined to be out-of-order data, the out-of-order data is written to the incremental list, and index information corresponding to the out-of-order data is inserted into the incremental index tree. The key in the index information is the timestamp of the out-of-order data, and the value in the index information includes a first position of the out-of-order data in the incremental list and a second position that needs to be accessed before accessing the first position. The second position is the position of the target data in the ordered list, and the timestamp of the target data is the adjacent timestamp of the current out-of-order data and is less than the timestamp of the current out-of-order data.

[0066] It should be noted that the functions or steps that can be implemented by the computer-readable storage medium or electronic device described above can be referred to the relevant descriptions on the server side and client side in the foregoing method embodiments. To avoid repetition, they will not be described one by one here.

[0067] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of methods and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0068] The above embodiments are merely illustrative of the principles and effects of this application and are not intended to limit this application. Any person skilled in the art can modify or alter the above embodiments without departing from the spirit and scope of this application. Therefore, all equivalent modifications or alterations made by those skilled in the art without departing from the spirit and technical concept disclosed in this application should still be covered by the claims of this application.

Claims

1. A method for storing a memory table, characterized in that, The memory table includes: an ordered list, an incremental list, and an incremental index tree; the method includes: If the timestamp of the data to be written is greater than or equal to the maximum timestamp of the data already written in the ordered list, then the data to be written is written to the ordered list and the maximum timestamp is updated. If the timestamp of the data to be written is less than the maximum timestamp, then the data to be written is determined to be out-of-order data. The out-of-order data is written to the incremental list, and index information corresponding to the out-of-order data is inserted into the incremental index tree. The key in the index information is the timestamp of the out-of-order data, and the value in the index information includes the first position of the out-of-order data in the incremental list and the second position that needs to be accessed before accessing the first position. The second position is the position of the target data in the ordered list, and the timestamp of the target data is the adjacent timestamp of the current out-of-order data and is less than the timestamp of the current out-of-order data.

2. The memory table storage method according to claim 1, characterized in that, The method further includes: Receive data query instructions; Based on the data query instruction, the ordered list, the incremental list, and the incremental index tree are accessed to read the data in an ordered manner, thereby obtaining an ordered distribution list of the data written in the memory table. Based on the ordered distribution list, data queries are performed to obtain and return the data query results.

3. The memory table storage method according to claim 2, characterized in that, Both the first position and the second position are stored in the form of position IDs, where the position ID corresponding to the first position is the first ID and the position ID corresponding to the second position is the second ID. By accessing the ordered list, the incremental list, and the incremental index tree, data is read in an ordered manner to obtain an ordered distribution list of data written to the memory table, including: The minimum timestamp in the incremental index tree is determined as the target timestamp; Obtain the second ID corresponding to the target timestamp from the incremental index tree, and obtain the target ID of the read position in the ordered list. The target ID refers to the position ID of the most recently read data in the ordered list. If the target ID is empty, or the target ID is less than the second ID corresponding to the target timestamp, then read the written data in the ordered list in order, store the read data in a preset ordered distribution list, and update the read position until the target ID of the read position is greater than or equal to the second ID corresponding to the target timestamp. If the target ID of the read position is greater than or equal to the second ID corresponding to the target timestamp, then according to the first ID corresponding to the target timestamp, read the out-of-order data corresponding to the target timestamp from the incremental list and write the out-of-order data corresponding to the target timestamp into the ordered distribution list; The timestamps in the incremental index tree are traversed to complete the filling of the ordered distribution list. If all the data in the incremental list has been written into the ordered distribution list, and there is data in the ordered list that has not been written into the ordered distribution list, then the remaining data in the ordered list is written into the ordered distribution list.

4. The memory table storage method according to claim 3, characterized in that, If the target ID of the read position is greater than or equal to the second ID corresponding to the target timestamp, then according to the first ID corresponding to the target timestamp, read the out-of-order data corresponding to the target timestamp from the incremental list, and write the out-of-order data corresponding to the target timestamp into the ordered distribution list, including: If the target ID of the read position is greater than the second ID corresponding to the target timestamp, and the timestamp of the read position is equal to the target timestamp, then according to the first ID corresponding to the target timestamp, the out-of-order data corresponding to the target timestamp is read from the incremental list, and the latest data written in the ordered distribution list is updated to the out-of-order data corresponding to the target timestamp.

5. The memory table storage method according to claim 3, characterized in that, By accessing the ordered list, the incremental list, and the incremental index tree, data is read in an ordered manner to obtain an ordered distribution list of data written to the memory table, which also includes: If the incremental index tree is empty, data is read directly from the ordered list in sequence, and the read data is written into the ordered distribution list.

6. The memory table storage method according to claim 2, characterized in that, The method further includes: Obtain the memory usage data of the memory table, wherein the memory usage data is the sum of the memory usage of the ordered list, the incremental list, and the incremental index tree; If the memory usage data exceeds a preset memory usage threshold, and / or the lifespan of the data written in the memory table exceeds a preset lifespan threshold, then the data in the ordered distribution list will be written to the target disk. Upon completion of data writing to the target disk, the ordered list and the incremental list are released to reset the array resources occupied by the ordered list and the incremental list. The reset array resources are returned to the preset memory management module. Furthermore, the incremental index tree is released to reset the node resources occupied by the incremental index tree. The reset node resources are returned to the memory management module.

7. The memory table storage method according to claim 1, characterized in that, Before writing the data to be written into the ordered list, the process also includes: The ordered list, the incremental list, and the incremental index tree are initialized. The initialization includes: requesting a fixed-length array resource from a preset memory management module to complete the initialization of the ordered list and the incremental list; and requesting multiple node resources from the memory management module to complete the initialization of the incremental index tree. The method further includes: if the array resources allocated to the ordered list are already full, and the number of array resources allocated to the current ordered list is less than the number of resources corresponding to a preset first resource threshold, then requesting additional array resources from the memory management module to complete the expansion of the ordered list; if the array resources allocated to the incremental list are already full, and the number of array resources allocated to the current incremental list is less than the number of resources corresponding to a preset second resource threshold, then requesting additional array resources from the memory management module to complete the expansion of the incremental list; if the node resources allocated to the incremental index tree are already full, and the number of node resources allocated to the current incremental index tree is less than the number of resources corresponding to a preset node resource threshold, then requesting additional node resources from the memory management module to complete the expansion of the incremental index tree.

8. A memory table storage system, characterized in that, The memory table includes: an ordered list, an incremental list, and an incremental index tree; the system includes: An ordered write module is used to write the data to be written into the ordered list and update the maximum timestamp if the timestamp of the data to be written is greater than or equal to the maximum timestamp of the data already written in the ordered list. An incremental write module is configured to determine that the data to be written is out of order if the timestamp of the data to be written is less than the maximum timestamp, write the out-of-order data into the incremental list, and insert index information corresponding to the out-of-order data into the incremental index tree. The key in the index information is the timestamp of the out-of-order data, and the value in the index information includes the first position of the out-of-order data in the incremental list and the second position that needs to be accessed before accessing the first position. The second position is the position of the target data in the ordered list, and the timestamp of the target data is the adjacent timestamp of the current out-of-order data and is less than the timestamp of the current out-of-order data.

9. An electronic device, characterized in that, It includes a processor, a memory, and a communication bus; the communication bus is used to connect the processor and the memory; the processor is used to execute a computer program stored in the memory to implement the memory table storage method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, It stores a computer program that causes the computer to perform the memory table storage method as described in any one of claims 1 to 7.

Images