Hot data management method, system, equipment, medium and product

Abstract

The embodiment of the invention discloses a hot data management method, system and device, a medium and a product, and the method comprises the steps: writing hot data in a cache pool into a local file in response to determining that the resource utilization rate of a target server within a first preset time is lower than a preset value; wherein the hot data comprises first hot data with an updating mark, and second hot data which is not persisted within a second preset time; in response to determining that the database receives the data access demand, determining whether to-be-accessed data is in the cache pool based on a preset hash index table; in response to determining that the to-be-accessed data is not in the cache pool, determining whether the to-be-accessed data is hot data; and in response to determining that the to-be-accessed data is hot data, traversing the to-be-accessed data in the local file, and loading the to-be-accessed data to the cache pool. According to the method, the hot data is written when the server is idle, and loading is carried out according to the hash table as required when the database is started, so that the starting time delay is shortened.

Description

Technical Field

[0001] The application generally relates to the fields of communication technology and network transmission, and in particular to a method, system, device, medium and product for managing thermal data. Background Technology

[0002] Cloud-native databases generally adopt a compute-storage separation architecture. In this architecture, compute nodes are responsible for executing database queries and data processing operations using SQL (Structured Query Language), while cloud storage is responsible for storing the data. To reduce access latency, compute nodes typically prioritize accessing frequently accessed data in the local cache pool, then access local data files, and finally access cloud data storage. They also periodically write frequently accessed data from the cache pool to local files in the form of snapshots.

[0003] In related technologies, compute nodes must sequentially complete the following steps upon each restart: fully loading the local snapshot, replaying the redo log to the latest snapshot point, and providing services. If the local snapshot is old or too large, the replay process is significantly time-consuming, resulting in slow node recovery. Furthermore, if the local snapshot is corrupted, it is necessary to access remote cloud storage, further increasing latency and impacting system availability and user experience. Summary of the Invention

[0004] In view of the above, this application provides a method, system, device, medium, and product for managing thermal data, addressing some of the deficiencies mentioned in the background art.

[0005] In a first aspect, embodiments of this application provide a method for managing thermal data, comprising: In response to determining that the resource utilization rate of the target server is lower than a preset value within a first preset time period, hot data in the cache pool is written to a local file; wherein, the hot data includes first hot data with an update tag, and second hot data that has not been persisted within a second preset time period. In response to determining that the database has received a data access request, the system determines whether the data to be accessed is in the cache pool based on a preset hash index table. In response to determining that the data to be accessed is not in the cache pool, determine whether the data to be accessed is hot data; In response to determining that the data to be accessed is hot data, the data to be accessed is traversed in the local file and loaded into the cache pool.

[0006] Optionally, the cache pool includes several hot data items; The step of writing hot data from the cache pool to a local file includes: Based on the write information and update information of several types of thermal data, the first thermal data is determined; Among the several thermal data, a second thermal data that is not persisted within a second preset time period is determined; The first thermal data and the second thermal data are divided to obtain the divided thermal data. The partitioned hot data is written to a local file in sequence.

[0007] Optionally, determining the first hot data based on the write information and update information of the several hot data includes: For any one of the hot data, obtain the first timestamp of the hot data being written to the local file, and the second timestamp of the hot data being updated; The first timestamp and the second timestamp are compared; In response to determining that the first timestamp is earlier than the second timestamp, the updated hot data is updated and marked to obtain the first hot data.

[0008] Optionally, in response to determining that the database has received a data access request, the method further includes: In response to the database startup operation, the description information of the data pages in the cache pool is obtained and a hash index table is constructed based on the description information.

[0009] Optionally, in response to determining that a data access request has been received, determining whether the data to be accessed is hot data further includes: In response to determining that the data to be accessed is not hot data, the historical access frequency of the data to be accessed is determined; The data to be accessed is sorted in descending order of historical access frequency to obtain sorted data to be accessed. According to the order of the sorted data to be accessed, the sorted data to be accessed is loaded into the cache pool in sequence.

[0010] Optionally, the method further includes: In response to determining that the data to be accessed is in a preset state in the local file; wherein the preset state includes a corrupted state; Based on the configuration information of the current computing node, determine the nearest neighboring computing node to the current computing node; The relevant information of the data to be accessed is obtained based on the adjacent computing nodes; The relevant information of the data to be accessed is loaded into the cache pool.

[0011] In a second aspect, embodiments of this application provide a management system for thermal data, comprising: The hot data writing module is configured to: in response to determining that the resource utilization rate of the target server is lower than a preset value within a first preset time period, write hot data in the cache pool to a local file; wherein, the hot data includes first hot data with an update tag, and second hot data that has not been persisted within a second preset time period. The data access module is configured to: in response to determining that the database has received a data access request, determine whether the data to be accessed is in the cache pool based on a preset hash index table; The hot data determination module is configured to: in response to determining whether the data to be accessed is hot data in response to determining that the data to be accessed is not in the cache pool; The hot data loading module is configured to: in response to determining that the data to be accessed is hot data, traverse the data to be accessed in the local file and load the data to be accessed into the cache pool.

[0012] In a third aspect, embodiments of this application provide a transmission device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the thermal data management method described in the first aspect above.

[0013] In a fourth aspect, embodiments of this application provide a storage medium storing computer instructions for causing the computer to execute the thermal data management method described in the first aspect.

[0014] In a fifth aspect, embodiments of this application provide a computer program product, including computer program instructions, which, when executed on a computer, cause the computer to perform the hot data management method described in the first aspect. Attached Figure Description

[0015] Figure 1 A flowchart illustrating a thermal data management method provided in an embodiment of this application is shown. Figure 2 This illustration shows a flowchart of a method for writing hot data to a local file according to an embodiment of this application; Figure 3 This illustration shows a flowchart of a method for generating first thermal data according to an embodiment of this application. Figure 4 This paper shows a schematic diagram of the structure of a thermal data management system provided in an embodiment of this application; Figure 5 A schematic diagram of a more specific electronic device hardware structure provided in this embodiment is shown. Detailed Implementation

[0016] The present application will now be described in further detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the application and not intended to limit it. Furthermore, it should be noted that, for ease of description, the accompanying drawings show only the parts relevant to the present application, not the entire structure.

[0017] As described in the background section, cloud-native databases generally adopt a compute-storage separation architecture. Compute nodes are responsible for executing database queries and data processing operations using SQL (Structured Query Language), while cloud storage is responsible for storing the data. To reduce access latency, compute nodes typically prioritize accessing frequently accessed data in the local cache pool, then access local data files, and finally access cloud data storage. They also periodically write snapshots of frequently accessed data from the cache pool to local files.

[0018] However, as the business expands and operating time increases, the relevant technologies have gradually revealed the following pain points: 1. Accumulation of outdated data Local snapshots accumulate over a long period of time. When a compute node restarts, it needs to fully load and replay the redo logs to the latest point. The amount of replay is large, and the node recovery time is uncontrollable.

[0019] 2. Linear expansion of data volume The number of times snapshot files are written to local files increases linearly, and the time required for a full load also increases linearly, severely impacting the system's ability to quickly go online.

[0020] 3. Local file corruption triggers cloud storage relocation. If a local snapshot is corrupted or lost, the compute node can only roll back to the remote cloud storage to read it. This results in high bandwidth and latency across data centers, causing an instantaneous degradation in service response.

[0021] Therefore, there is an urgent need for a hot data management method that can continuously reduce the proportion of old data and control the size of local snapshots during operation, and load data on demand and hot during the startup phase, while using neighboring nodes to quickly replenish data, thereby fundamentally shortening restart time, reducing the probability of cloud storage access, and improving the availability and cost-effectiveness of cloud-native databases.

[0022] Figure 1 The diagram illustrates a flow chart of a thermal data management method according to an embodiment of the present invention, the method comprising the following steps: S102: In response to determining that the resource utilization rate of the target server is lower than the preset value within a first preset time period, write the hot data in the cache pool to the local file.

[0023] In some optional embodiments, the hot data management method of this disclosure can be implemented through a hot data management system. Specifically, in order to solve the problems of old data accumulation and high time consumption of full data write-to-disk when computing nodes restart each time, this disclosure adopts an idle-time on-demand disk write-to-disk strategy. That is, hot data in the cache pool is only written to the local file when the resource utilization of CPU (Central Processing Unit) and IO (Input / Output) is simultaneously lower than a preset value, and each write only writes the first hot data with an update mark, or the second hot data that has not been persisted within a second preset time, thereby reducing the amount of data written while ensuring the freshness of the local file and avoiding the problem of being slowed down by disk write-to-disk during peak business periods.

[0024] In some optional embodiments, the first preset time in this disclosure can be set to 5 minutes. That is, every 5 minutes, the hot data management system collects the CPU and IO usage; once both are lower than the preset value, the system can sort the hot data from high to low according to the access frequency of the hot data, and write the hot data that "has not been persisted within the second preset time" or "has been marked with an update" to the local file in batches according to the sorting order, thereby minimizing the amount of data written to disk while ensuring data freshness.

[0025] It is understood that the aforementioned first preset time can also be other values. This disclosure only uses 5 minutes as an example for illustration, but it does not mean that the first preset time in this disclosure can only be 5 minutes. Correspondingly, the preset value in this disclosure can also be set by the user, and can be a value customized by the user according to the peak and valley characteristics of business. This disclosure does not limit this.

[0026] Figure 2 This illustration shows a flowchart of a method for generating first thermal data according to an embodiment of this application. Figure 3 A schematic diagram of the structure of a thermal data management system provided in an embodiment of this application is shown.

[0027] like Figure 2 He Ru Figure 3 As shown, in some optional embodiments, step S102 further includes: S1022: Based on the write information and update information of several hot data, determine the first hot data.

[0028] In some optional embodiments, step S1022 further includes: S10222: For any one of the hot data, obtain the first timestamp of the hot data being written to the local file, and the second timestamp of the hot data being updated.

[0029] S10224: Compare the first timestamp and the second timestamp.

[0030] S10226: In response to determining that the first timestamp is before the second timestamp, the updated hot data is updated and marked to obtain the first hot data.

[0031] S1024: Among the plurality of thermal data, determine the second thermal data that has not been persisted within a second preset time period.

[0032] S1026: Divide the first thermal data and the second thermal data into partitioned thermal data.

[0033] S1028: Write the divided hot data into local files in sequence.

[0034] Next, this disclosure will provide a detailed explanation of how to accurately identify and efficiently write the first and second hot data. Specifically, the cache pool in this disclosure can include several hot data sets. The management system can first obtain the first timestamp of any hot data set being written to the local file, and the second timestamp of the hot data set being updated. Then, the first and second timestamps are compared. If the first timestamp is earlier than the second timestamp, the updated hot data is marked as updated, thus obtaining the first hot data set. Therefore, by comparing the historical write time and the latest update time of the hot data, hot data sets that are "cache-updated but not disk-synchronized" can be instantly identified, avoiding duplicate writes and reducing I / O overhead.

[0035] In some optional embodiments, after obtaining the first hot data, the hot data management system can further identify a second hot data that has not been persisted within a second preset time period from among several hot data. The second preset time period can be set by the user based on business tolerance, and this disclosure does not limit it.

[0036] Subsequently, to alleviate the CPU and I / O resource burden on computing nodes and prevent impact on their external service provision, the hot data management system, after obtaining the first and second hot data, can further divide the first and second hot data into data batches, thus obtaining N batches of hot data. Finally, the management system can sequentially write the divided hot data into local files. Therefore, through the dual strategy of time window and batch division in this disclosure, the one-time full write in related technologies can be changed to batch sequential write, significantly improving disk throughput.

[0037] In some optional embodiments, the system of this disclosure may first determine the second hot data, and then determine the first hot data based on the second hot data. That is, the cache pool is first scanned according to the "non-persistence duration" to filter out data pages that have not been written to the local file for more than a second preset time, and these are marked as second hot data; then, among these second hot data, pages that have been updated are filtered out by comparing the "update time" and the "write time", and these are marked as first hot data. This sequential processing can avoid repeated scanning of hot data that has just been written but has not yet been updated, reducing CPU overhead; at the same time, the second hot data, as a fallback set of the "time window", can ensure that no data is missed, improving persistence integrity and system throughput efficiency.

[0038] In some optional embodiments, after obtaining the first and second hot data, the hot data management system can intelligently batch-divide the first and second hot data based on data size, update sequence, etc. For example, it can first sort them in ascending order by update time, and then divide them according to a preset capacity threshold, thereby generating N ordered batches of hot data. Metadata (such as batch number, number of records, CRC checksum, etc.) can be appended to the header of each batch, thus forming a hot data unit with its own index. Finally, the management system can sequentially write the data to a local file according to the batch sequence number. This achieves a change from a one-time full write to a batch sequential write.

[0039] In some optional embodiments, when the database cache pool uses more memory than a preset limit, the system can mark "non-hot data" pages as pending writes based on LRU-2Q (cache eviction algorithm) or a user-defined eviction policy, and immediately trigger a background asynchronous thread to write them in batches to the local data file, thus freeing up space for the database cache pool. Simultaneously, the system can reserve a 10% memory buffer to ensure that frontend write requests can continue inserting new data without waiting for eviction to complete, avoiding "write blocking".

[0040] S104: In response to determining that the database has received a data access request, determine whether the data to be accessed is in the cache pool based on a preset hash index table.

[0041] In some optional embodiments, after confirming that the database has received a data access request, the system can determine whether the data to be accessed exists in the cache pool based on a preset hash index table. This hash index table can be constructed all at once during the database startup phase, i.e., by scanning the description information (such as page number, key value, version number, etc.) of each data page in the cache pool, and the "key and cache slot" mapping is written into shared memory. Subsequent accesses only require a single hash calculation to complete the hit determination in a short time, avoiding the CPU overhead caused by a full table scan.

[0042] In some optional embodiments, the hash table and data page lifecycle can be linked in real time: when a page is swapped out or updated, the system can atomically clear or refill the corresponding slot, ensuring that the index and cache state are always consistent, thereby significantly reducing response latency and improving concurrent query throughput.

[0043] In some optional embodiments, upon receiving a data access request, the system can immediately perform hash calculation on the access key according to a preset hash index table and quickly determine whether the data to be accessed exists in the cache pool. If the hash hits and the corresponding slot is marked as "valid," the cache pointer is returned directly, avoiding a full table scan. If the hash misses or the slot is marked as "invalid," a page fault is triggered, and data is loaded from the disk as needed and the hash table is updated. This mechanism reduces the lookup complexity of "key → cache location" to constant level, significantly reducing CPU usage and response latency. Simultaneously, the hash table can be updated in conjunction with the aforementioned LRU-2Q linked list, ensuring real-time index synchronization after hot data replacement and avoiding duplicate loading.

[0044] S106: In response to determining that the data to be accessed is not in the cache pool, determine whether the data to be accessed is hot data.

[0045] S108: In response to determining that the data to be accessed is hot data, the data to be accessed is traversed in the local file and loaded into the cache pool.

[0046] In some optional embodiments, if the system determines that the data to be accessed is not in the cache pool, it can further determine whether it is hot data. Specifically, the system can first extract the page number and the most recent access timestamp of the data page, and then compare it with the "access frequency-time" vector maintained by the cache pool. If the number of accesses to the data to be accessed exceeds a threshold within a unit of time, it can be marked as hot data. Subsequently, after determining that the data to be accessed is hot data, the system can search for whether the data to be accessed exists in the local file and load the data to be accessed into the cache pool. S118: In response to determining that the data to be accessed is not hot data, determine the historical access frequency of the data to be accessed.

[0047] S120: Sort the data to be accessed in descending order of historical access frequency to obtain sorted data to be accessed.

[0048] S122: Load the sorted data to be accessed into the cache pool in sequence according to the order of the sorted data to be accessed.

[0049] In some optional embodiments, if the system determines that the data to be accessed is not hot data, it can immediately read the historical access frequency of the data to be accessed and quickly sort all the data to be accessed in descending order of frequency to obtain a sorted list of data to be accessed. Subsequently, the system can load these data into the cache pool according to the order of the list.

[0050] In some optional embodiments, the method in this disclosure may further include: S124: In response to determining that the state of the data to be accessed in the local file is a preset state; wherein the preset state includes a corrupted state.

[0051] S126: Based on the configuration information of the current computing node, determine the nearest neighboring computing node to the current computing node.

[0052] S128: Obtain relevant information about the data to be accessed based on the adjacent computing nodes.

[0053] S130: Load the relevant information of the data to be accessed into the cache pool.

[0054] In some optional embodiments, when the system detects that the data to be accessed is in a preset state in the local file (e.g., corrupted, verification failed, or read timed out), it can immediately trigger the "inter-node data rescue" process. Specifically, firstly, the system can determine the nearest neighboring computing node to the current computing node based on the current computing node's configuration information (including rack location, switch network topology, and real-time network latency) using a shortest path algorithm.

[0055] Subsequently, the system can send a "data rescue request" to the adjacent compute node. The request can include a unique identifier of the data to be accessed (such as a page number or key-value hash) and a consistency version number. Upon receiving the request, the adjacent compute node can read the corresponding data block from its local cache or disk and return the relevant information of the data block (including data content, version number, and checksum) to the current compute node via RDMA or a high-speed internal network.

[0056] In some optional embodiments, upon receiving data, the current computing node can perform an integrity check. If the check passes, the data can be written to the local cache pool, and global metadata can be updated to indicate that the data block has been successfully rescued. If the check fails, a rescue request is sent to the next nearest node until valid data is successfully obtained. This mechanism automatically and quickly repairs damaged data without the user's awareness, avoiding business interruption. At the same time, through inter-node collaboration, it improves the overall system's availability and fault tolerance.

[0057] Based on the same inventive concept, corresponding to any of the above embodiments, this application also provides a thermal data management system.

[0058] Figure 4 A schematic diagram of the structure of a thermal data management system provided in an embodiment of this application is shown.

[0059] refer to Figure 4 As shown, the management system for the thermal data includes: S202: Hot data writing module, configured to: in response to determining that the resource utilization rate of the target server is lower than a preset value within a first preset time period, write hot data in the cache pool to a local file; wherein, the hot data includes first hot data with an update tag, and second hot data that has not been persisted within a second preset time period.

[0060] In some optional embodiments, step S202 further includes: S2022: Based on the write information and update information of several hot data, determine the first hot data; In some optional embodiments, step S2022 further includes: S20222: For any one of the hot data in the plurality of hot data, obtain the first timestamp of the hot data being written to the local file, and the second timestamp of the hot data being updated; S20224: Compare the first timestamp and the second timestamp; S20226: In response to determining that the first timestamp is before the second timestamp, the updated hot data is updated and marked to obtain the first hot data.

[0061] S2024: Among the plurality of thermal data, determine the second thermal data that has not been persisted within a second preset time. S2026: Divide the first thermal data and the second thermal data to obtain the divided thermal data; S2028: Write the divided hot data into local files in sequence.

[0062] S2030: In response to the database startup operation, obtain the description information of the data pages in the cache pool and construct a hash index table based on the description information.

[0063] S204: Data access module, configured to: in response to determining that the database has received a data access request, determine whether the data to be accessed is in the cache pool based on a preset hash index table.

[0064] S206: Hot data determination module, configured to: determine whether the data to be accessed is hot data in response to determining that the data to be accessed is not in the cache pool.

[0065] In some optional embodiments, step S206 further includes: S2062: In response to determining that the data to be accessed is not hot data, determine the historical access frequency of the data to be accessed; S2064: Sort the data to be accessed in descending order of historical access frequency to obtain sorted data to be accessed; S2066: Load the sorted data to be accessed into the cache pool in sequence according to the order of the sorted data to be accessed.

[0066] S208: Hot data loading module, configured to: in response to determining that the data to be accessed is hot data, traverse the data to be accessed in the local file and load the data to be accessed into the cache pool.

[0067] S210: In response to determining that the state of the data to be accessed in the local file is a preset state; wherein, the preset state includes a corrupted state; S212: Based on the configuration information of the current computing node, determine the nearest neighboring computing node to the current computing node; S214: Obtain relevant information about the data to be accessed based on the adjacent computing nodes; S216: Load the relevant information of the data to be accessed into the cache pool.

[0068] For ease of description, the above system is described by dividing it into various modules based on their functions. Of course, in implementing this application, the functions of each module can be implemented in one or more software and / or hardware.

[0069] The system described above can be used to implement the corresponding thermal data management method in any of the foregoing embodiments, and has the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0070] Based on the same inventive concept, corresponding to the methods of any of the above embodiments, this application also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the thermal data management method described in any of the above embodiments.

[0071] Figure 5This embodiment illustrates a more specific hardware structure of an electronic device, which may include a processor 810, a memory 820, an input / output interface 830, a communication interface 840, and a bus 850. The processor 810, memory 820, input / output interface 830, and communication interface 840 are interconnected internally via the bus 850. The processor 810 can be implemented using a general-purpose CPU (Central Processing Unit), microprocessor, application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the thermal data management method provided in any of the foregoing embodiments of this specification.

[0072] The memory 820 can be implemented in the form of ROM (Read Only Memory), RAM (Random Access Memory), static storage device, dynamic storage device, etc. The memory 820 can store the operating system and other applications. When the technical solutions provided in the embodiments of this specification are implemented by software or firmware, the relevant program code is stored in the memory 820 and is called and executed by the processor 810.

[0073] The input / output interface 830 is used to connect input / output modules to enable information input and output. Input / output modules can be configured as components within the device (not shown in the figure) or externally connected to the device to provide corresponding functions. Input devices may include keyboards, mice, touchscreens, microphones, various sensors, etc., while output devices may include displays, speakers, vibrators, indicator lights, etc.

[0074] The communication interface 840 is used to connect the communication module (not shown in the figure) to enable communication between this device and other devices. The communication module can communicate via wired means (such as USB, Ethernet cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.). Bus 1050 includes a pathway for transmitting information between various components of the device, such as processor 810, memory 820, input / output interface 830, and communication interface 840.

[0075] It should be noted that although the above-described device only shows the processor 810, memory 820, input / output interface 830, communication interface 840, and bus 850, in specific implementations, the device may also include other components necessary for normal operation. Furthermore, those skilled in the art will understand that the above-described device may only include the components necessary for implementing the embodiments of this specification, and not necessarily all the components shown in the figures. The electronic devices described above are used to implement the corresponding thermal data management methods in any of the foregoing embodiments, and have the beneficial effects of the corresponding method embodiments, which will not be repeated here. Based on the same inventive concept, corresponding to the methods of any of the above embodiments, this disclosure also provides a non-transitory computer-readable storage medium storing computer instructions for causing the computer to execute the hot data management method as described in any of the above embodiments.

[0076] The computer-readable medium of this embodiment includes permanent and non-permanent, removable and non-removable media, and information storage can be implemented by any method or technology. Information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transfer medium that can be used to store information accessible by a computing device. The aforementioned non-transitory computer-readable storage media can be any available medium or data storage device that a computer can access, including but not limited to magnetic storage (e.g., floppy disks, hard disks, magnetic tapes, magneto-optical disks (MOs), etc.), optical storage (e.g., CDs, DVDs, BDs, HVDs, etc.), and semiconductor storage (e.g., ROMs, EPROMs, EEPROMs, non-volatile memory (NAND flash), solid-state drives (SSDs)). The computer instructions stored in the storage medium of the above embodiments are used to cause the computer to execute the thermal data management method as described in any of the embodiments in the exemplary method section above, and have the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0077] Those skilled in the art should understand that the discussion of any of the above embodiments is merely exemplary and is not intended to imply that the scope of this application (including the claims) is limited to these examples; within the framework of this application, the technical features of the above embodiments or different embodiments can also be combined, the steps can be implemented in any order, and there are many other variations of different aspects of the embodiments of this application as described above, which are not provided in the details for the sake of brevity.

[0078] Additionally, to simplify the description and discussion, and to avoid obscuring the embodiments of this application, the well-known power / ground connections to integrated circuit (IC) chips and other components may or may not be shown in the provided drawings. Furthermore, the apparatus may be shown in block diagram form to avoid obscuring the embodiments of this application, and this also takes into account the fact that the details of the implementation of these block diagram apparatuses are highly dependent on the platform on which the embodiments of this application will be implemented (i.e., these details should be fully understood by those skilled in the art). While specific details (e.g., circuits) have been set forth to describe exemplary embodiments of this application, it will be apparent to those skilled in the art that the embodiments of this application can be implemented without these specific details or with variations thereof. Therefore, these descriptions should be considered illustrative rather than restrictive.

[0079] Although this application has been described in conjunction with specific embodiments thereof, many substitutions, modifications, and variations of these embodiments will be apparent to those skilled in the art from the foregoing description. For example, other memory architectures (e.g., dynamic RAM (DRAM)) may be used with the embodiments discussed.

[0080] The embodiments of this application are intended to cover all such substitutions, modifications, and variations that fall within the broad scope of the appended claims. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the embodiments of this application should be included within the protection scope of this application.

Claims

1. A method for managing thermal data, characterized in that, include: In response to determining that the resource utilization rate of the target server is lower than a preset value within a first preset time period, hot data in the cache pool is written to a local file; wherein, the hot data includes first hot data with an update tag, and second hot data that has not been persisted within a second preset time period. In response to determining that the database has received a data access request, the system determines whether the data to be accessed is in the cache pool based on a preset hash index table. In response to determining that the data to be accessed is not in the cache pool, determine whether the data to be accessed is hot data; In response to determining that the data to be accessed is hot data, the data to be accessed is traversed in the local file and loaded into the cache pool.

2. The management method according to claim 1, characterized in that, The cache pool includes several hot data items; The step of writing hot data from the cache pool to a local file includes: Based on the write information and update information of several types of thermal data, the first thermal data is determined; Among the several thermal data, a second thermal data that is not persisted within a second preset time period is determined; The first thermal data and the second thermal data are divided to obtain the divided thermal data. The partitioned hot data is written to a local file in sequence.

3. The management method according to claim 2, characterized in that, The determination of the first hot data based on the write information and update information of several hot data includes: For any one of the hot data, obtain the first timestamp of the hot data being written to the local file, and the second timestamp of the hot data being updated; The first timestamp and the second timestamp are compared; In response to determining that the first timestamp is earlier than the second timestamp, the updated hot data is updated and marked to obtain the first hot data.

4. The management method according to claim 1, characterized in that, Before determining that the database has received a data access request, the method further includes: In response to the database startup operation, the description information of the data pages in the cache pool is obtained and a hash index table is constructed based on the description information.

5. The management method according to claim 1, characterized in that, The step of determining whether the data to be accessed is hot data in response to determining that the data to be accessed is not in the cache pool further includes: In response to determining that the data to be accessed is not hot data, the historical access frequency of the data to be accessed is determined; The data to be accessed is sorted in descending order of historical access frequency to obtain sorted data to be accessed. According to the order of the sorted data to be accessed, the sorted data to be accessed is loaded into the cache pool in sequence.

6. The management method according to claim 1, characterized in that, The method further includes: In response to determining that the data to be accessed is in a preset state in the local file; wherein the preset state includes a corrupted state; Based on the configuration information of the current computing node, determine the nearest neighboring computing node to the current computing node; The relevant information of the data to be accessed is obtained based on the adjacent computing nodes; The relevant information of the data to be accessed is loaded into the cache pool.

7. A management system for thermal data, characterized in that, include: The hot data writing module is configured to: in response to determining that the resource utilization rate of the target server is lower than a preset value within a first preset time period, write hot data in the cache pool to a local file; wherein, the hot data includes first hot data with an update tag, and second hot data that has not been persisted within a second preset time period. The data access module is configured to: in response to determining that the database has received a data access request, determine whether the data to be accessed is in the cache pool based on a preset hash index table; The hot data determination module is configured to: in response to determining whether the data to be accessed is hot data in response to determining that the data to be accessed is not in the cache pool; The hot data loading module is configured to: in response to determining that the data to be accessed is hot data, traverse the data to be accessed in the local file and load the data to be accessed into the cache pool.

8. An electronic device, characterized in that, It includes a memory and a processor, the memory storing a computer program, and the processor executing the program to implement the thermal data management method according to claims 1 to 6.

9. A computer-readable storage medium, characterized in that, The medium stores computer instructions for causing the computer to execute the thermal data management method according to claims 1 to 6.

10. A computer program product comprising computer program instructions that, when executed on a computer, cause the computer to perform a method for managing hot data as described in any one of claims 1 to 6.

Images