Data processing method and electronic device

Abstract

The application provides a data processing method and an electronic device; the method comprises: initializing a memory pool of a first file; wherein the memory pool corresponds to a three-level storage structure, the three-level storage structure comprises a first-level cache ring stored by using a ring queue, a second-level cache unit stored by using a continuous storage space, and a third-level cache slice stored by using a continuous storage space; the first-level cache ring comprises a plurality of second-level cache units; the second-level cache unit comprises a plurality of third-level cache slices; writing data into the memory pool, reading data in the memory pool, or performing remote synchronization on the data in the memory pool. Through the application, the memory utilization and the data access efficiency can be improved.

Description

Technical Field

[0001] This application relates to computer technology, and more particularly to a data processing method and an electronic device. Background Technology

[0002] In distributed systems, data processing and storage performance are key factors affecting overall efficiency. With the increasing demand for large-scale data access, efficient cache management and improved read / write speeds have become research priorities. Most current distributed file systems adopt a user-space file system (fuse) architecture, taking over file read / write interfaces in user space to implement custom data read / write methods and storage locations, and combining this with local caching mechanisms to improve performance. Among these, JuiceFS, as one of the mainstream file systems, optimizes the data read / write process through a hierarchical caching structure (such as chunks, slices, and blocks), and maintains cache utilization and read / write speed through traversal and merging operations.

[0003] In related technologies, such file systems typically use non-contiguous storage structures, such as dividing files into multiple chunks, each chunk containing several slices, and each slice further divided into multiple blocks. Because these units may have non-contiguous or overlapping logical addresses, their state information needs to be frequently maintained, and fragment cleanup and merging operations need to be performed, increasing system overhead and impacting performance. Summary of the Invention

[0004] This application provides a data processing method and an electronic device, which can improve memory utilization and data access efficiency.

[0005] The technical solution of this application embodiment is implemented as follows:

[0006] This application provides a data processing method, the method comprising:

[0007] Initialize the memory pool for the first file; wherein, the memory pool corresponds to a three-level storage structure, the three-level storage structure includes a first-level cache ring using a circular queue, a second-level cache unit using contiguous storage space, and a third-level cache slice using contiguous storage space; the first-level cache ring includes multiple second-level cache units; the second-level cache unit includes multiple third-level cache slices;

[0008] Write data to the memory pool, read data from the memory pool, or remotely synchronize data in the memory pool.

[0009] This application provides a data processing apparatus, which includes:

[0010] A processing unit is used to initialize the memory pool of the first file; wherein, the memory pool corresponds to a three-level storage structure, the three-level storage structure includes a first-level cache ring using a circular queue, a second-level cache unit using contiguous storage space, and a third-level cache slice using contiguous storage space; the first-level cache ring includes multiple second-level cache units; the second-level cache unit includes multiple third-level cache slices;

[0011] The processing unit is also used to write data to the memory pool, read data from the memory pool, or remotely synchronize data in the memory pool.

[0012] This application provides an electronic device, the electronic device comprising:

[0013] Memory is used to store executable instructions or computer programs.

[0014] The processor, when executing computer-executable instructions or computer programs stored in the memory, implements the data processing method provided in the embodiments of this application.

[0015] This application provides a computer-readable storage medium storing a computer program or computer-executable instructions for implementing the data processing method provided in this application when executed by a processor.

[0016] This application provides a computer program product, including a computer program or computer executable instructions. When the computer program or computer executable instructions are executed by a processor, they implement the data processing method provided in this application.

[0017] The embodiments of this application have the following beneficial effects: This application proposes a memory pool acceleration method based on a three-level storage structure (including a first-level cache ring in a circular queue, a second-level cache unit with contiguous storage space, and a third-level cache slice). The first-level cache ring of the circular queue can cyclically use a fixed-size cache area, thereby avoiding the fragmentation problem that may occur in the linked list cache structure used in related technologies, and improving access efficiency. The second-level cache unit is a contiguous storage space, and each cache unit points to one or more third-level cache slices, making cache management more efficient. The third-level cache slice is an independent smallest storage unit used to store the actual data content. The three-level cache structure provided by this application can quickly locate the read and write positions and achieve efficient cache management and data synchronization, thereby improving memory utilization and data access efficiency. Attached Figure Description

[0018] Figure 1 This is a first flowchart illustrating the data processing method provided in an embodiment of this application;

[0019] Figure 2 This is a schematic diagram of a three-level storage structure provided in an embodiment of this application;

[0020] Figure 3 This is a flowchart illustrating a remote synchronization method provided in an embodiment of this application;

[0021] Figure 4 This is a flowchart of a writing process provided in an embodiment of this application;

[0022] Figure 5 This is a flowchart of a reading process provided in an embodiment of this application;

[0023] Figure 6 This is a schematic diagram of the structure of the data processing apparatus provided in the embodiments of this application;

[0024] Figure 7 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application.

[0025] It should be noted that the terms "first" and "second" mentioned above are only used to distinguish between different options and do not represent the degree of superiority or inferiority of the options or their priority in the implementation process. Detailed Implementation

[0026] To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings. The described embodiments should not be regarded as limitations on this application. All other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0027] The terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish different objects, not to describe a specific order. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or apparatuses.

[0028] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0029] Before explaining this application, the juicefs in the related art is described here:

[0030] JuiceFS is a high-performance distributed file system designed for cloud-native environments. Its memory caching employs tiered storage, divided into chunks, slices, and blocks. All reads and writes are located at chunks based on offsets, or new chunks are generated accordingly. A chunk may contain multiple slices that overlap and are not contiguous. A slice represents a single continuous write operation. Slices are divided into multiple blocks for concurrent remote distribution. When writing data, JuiceFS iterates through all chunks of the file, accumulating the number of slices within each chunk to determine if the limit has been exceeded. Then, it iterates through all slices within a chunk, searching for writable slices based on offsets and write lengths; if none are found, a new slice is created. Since the number and length of slices within a chunk are variable, and their logical addresses are not contiguous and may overlap, their state information must be maintained, and fragment cleanup and slice merging must be performed regularly to ensure cache utilization and read / write speed.

[0031] Embodiments of this application provide a data processing method applied to an electronic device, with reference to... Figure 1 As shown, the method includes the following steps:

[0032] Step 101: Initialize the memory pool for the first file.

[0033] The memory pool corresponds to a three-level storage structure, which includes a first-level cache ring using a circular queue, a second-level cache unit using contiguous storage space, and a third-level cache slice using contiguous storage space. The first-level cache ring includes multiple second-level cache units, and the second-level cache unit includes multiple third-level cache slices.

[0034] In this embodiment, a circular queue (or circular buffer) is a first-in, first-out (FIFO) data structure. Its ends are connected to form a closed loop, and it is used cyclically within a fixed-size buffer. Circular queues are suitable for high-concurrency, low-latency scenarios. For example, in streaming media transmission, the receiving end can use a circular queue to buffer received data packets, ensuring that data is played in order and preventing stuttering due to network fluctuations. Similarly, in real-time audio processing, a circular queue can be used to temporarily store input audio data so that subsequent processing modules can read it promptly.

[0035] In this embodiment, the memory pool is used to cache file read and write data to improve access performance and reduce frequent access to remote storage. The first-level cache ring is a circular queue structure that can cyclically use a fixed-size cache area, thus avoiding fragmentation problems that may occur in linked list-based cache structures in related technologies and improving access efficiency. The second-level cache unit is a contiguous storage space, with each cache unit pointing to one or more third-level cache slices, making cache management more efficient. The third-level cache slice is an independent, smallest storage unit used to store the actual data content. Therefore, the above three-level cache structure can quickly locate read and write positions and achieve efficient cache management and data synchronization.

[0036] In this embodiment, contiguous storage space refers to a set of adjacent address spaces in memory. This set of adjacent address spaces can be allocated and released at once, resulting in high access efficiency. Compared to non-contiguous storage, contiguous storage reduces addressing overhead and improves cache hit rate. For example, in database systems, contiguous storage space is often used to store table data to speed up queries. Similarly, in image processing, image pixels are typically stored in a contiguous space as a two-dimensional array for fast read / write and processing.

[0037] In some embodiments, the ring element of the first-level cache ring is the starting address of each second-level cache unit and the first storage identifier of each second-level cache unit.

[0038] Here, the first storage identifier is used to indicate whether the corresponding second-level cache unit is readable, writable, or needs to be reclaimed.

[0039] Here, there is a logical mapping relationship between the first-level cache ring and the second-level cache unit. When data at a certain offset needs to be accessed, the system first calculates the index position of that offset in the first-level cache ring, and then finds the corresponding second-level cache unit based on that index. This hierarchical structure improves data location efficiency and simplifies memory management logic.

[0040] In some embodiments, the second-level cache unit stores the starting address of each third-level cache slice, the timeout period of each third-level cache slice, the second storage identifier of each third-level cache slice, the storage offset range of the target data in the first file, and a first parameter; the first parameter is used to verify whether the data is written repeatedly.

[0041] Here, the second-level cache unit is an intermediate layer structure connecting the first-level cache ring and the third-level cache slice.

[0042] Here, the timeout period is used to mark the validity period of the third-level cache. If the third-level cache is not accessed or updated, it will be reclaimed after the timeout period.

[0043] Here, the second storage identifier is used to indicate the status of the third-level cache. For example, when the second storage identifier is white, it means that it is readable and writable; when the second storage identifier is gray, it means that it needs to be synchronized to remote storage.

[0044] Here, there is a direct data reference relationship between the second-level cache unit and the third-level cache slice. The second-level cache unit points to a specific third-level cache slice through its starting address, and the second-level cache unit, together with other metadata, manages the lifecycle and state of the third-level cache slice.

[0045] In this embodiment, the storage identifier includes color identifiers, graphic identifiers, text identifiers, number identifiers, location identifiers, etc. The storage identifier can be a first identifier, such as a gray identifier; the storage identifier can be a second identifier, such as a white identifier.

[0046] This application employs a color-coding mechanism to identify the status of cache slices and cache units, with white and gray being the most common colors. White indicates that the cache slice is empty or has been successfully synchronized, and the cache slice can be overwritten or deleted; gray indicates that the cache slice is being written to or is awaiting synchronization, and the cache slice cannot be directly overwritten, but must wait for synchronization to complete before proceeding. Color coding simplifies cache status management and helps determine when it is safe to perform overwrite or deletion operations.

[0047] In some embodiments, the third-level cache stores the data of the first file.

[0048] Here, the L3 cache slice is the lowest-level cache unit in the memory pool, used to store the actual file data. Each L3 cache slice is a contiguous block of memory with a fixed size that cannot be changed after it is created. Multiple L3 cache slices are independent of each other, with no overlap or dependency, so the system can flexibly allocate and reclaim these L3 cache slices.

[0049] Figure 2 This is a schematic diagram of a three-level storage structure provided in an embodiment of this application; as shown... Figure 2 As shown, the first-level cache ring is a circular queue of length L. The ring element is the starting address of the second-level storage and the color mark of that interval. The colors are gray and white. White indicates that the data is waiting for overdue collection, while gray indicates that the data slice needs to be synchronized remotely. After successful synchronization, it turns white.

[0050] The second level of storage is a contiguous storage space, which can be represented by an array. It stores the starting address of the third level of storage, the expiration time, the color mark, and the storage offset range of the data in the file, denoted as start and end, as well as an auto-incrementing id (i.e., the first parameter), which is used to mark whether a write has occurred during the remote synchronization process. If so, synchronization is repeated.

[0051] The third level of storage consists of actual data cache slices, which are contiguous storage spaces with an initial configurable size that cannot be changed after generation. The cache slices are independent of each other and can cache data written by the kernel as well as remote data read from it.

[0052] like Figure 2 As shown, the first-level cache ring with a read / write ring length of L includes three intervals (three second-level cache units); two intervals are marked in white, and one interval is marked in gray. Figure 2 The text uses thick lines to represent the color markings (gray). Within the gray area, there are M cache slices (0 to M-1), each containing N bytes. Of these, M-1 cache slices are marked white, and one cache slice is marked gray. Figure 2 The data is represented by a thick line and marked in gray, which indicates that the data is to be synchronized to the remote location.

[0053] Step 102: Write data to the memory pool, read data from the memory pool, or remotely synchronize the data in the memory pool.

[0054] In this embodiment of the application, after the internal storage is initialized, a management thread is started to manage the data in the memory pool.

[0055] In this embodiment, remote synchronization refers to writing data from the memory pool to remote storage to ensure data consistency and persistence. In practice, remote synchronization can be automatically triggered when the cache slice turns gray, or it can be manually triggered when the user explicitly calls the synchronization command.

[0056] This application provides a data processing method. The method includes: initializing a memory pool for a first file; wherein the memory pool corresponds to a three-level storage structure, the three-level storage structure including a first-level cache ring stored in a circular queue, a second-level cache unit stored in contiguous storage space, and a third-level cache slice stored in contiguous storage space; the first-level cache ring includes multiple second-level cache units; the second-level cache unit includes multiple third-level cache slices; writing data to the memory pool, reading data from the memory pool, or remotely synchronizing data in the memory pool. In other words, this application proposes a memory pool acceleration method based on a three-level storage structure (including a first-level cache ring in a circular queue, second-level cache units in contiguous storage space, and third-level cache slices). The first-level cache ring in the circular queue can cyclically use a fixed-size cache area, thereby avoiding the fragmentation problem that may occur in the linked list cache structure used in related technologies, and improving access efficiency. The second-level cache unit is a contiguous storage space, and each cache unit points to one or more third-level cache slices, making cache management more efficient. The third-level cache slice is an independent smallest storage unit used to store the actual data content. The three-level cache structure provided in this application can quickly locate the read and write positions and achieve efficient cache management and data synchronization, thereby improving memory utilization and data access efficiency.

[0057] In some embodiments, the remote synchronization of data in the memory pool in step 102 includes the following steps:

[0058] Step A1: Starting from the position where data is written in the memory pool, scan the first-level cache ring in the first direction and determine the target second-level cache unit.

[0059] Among them, the target second-level cache unit is the second-level cache unit with the first storage identifier as the first identifier among multiple second-level cache units; the first direction is the read / write direction or the data growth direction.

[0060] In this embodiment, the first direction refers to the traversal order used when scanning the first-level cache ring. The first direction can be the direction of read / write operations (e.g., in access order) or the direction of data growth (i.e., the direction in which data is appended sequentially).

[0061] Step A2: Traverse the target second-level cache unit and determine the target third-level cache slice.

[0062] Among them, the target third-level cache slice is the third-level cache slice whose second storage identifier is the first identifier among multiple third-level cache slices.

[0063] Step A3: Synchronize the cache slices included in the target third-level cache slice to the remote location.

[0064] In this embodiment, after initializing the ring data, a new thread is started to scan each element on the ring in the direction of data growth from the starting write position, skipping white storage items. If a gray mark is found, its secondary cache is scanned in index growth order, also skipping white items. The valid data block in the third-level cache corresponding to the gray index is synchronized to the remote location, along with parameters: offset, len, data, id, etc. The offset is the start value stored in the secondary storage, len is the length of the valid data, end-start+1, and the id is incremented with each write and sent to the remote location. After receiving a successful remote response message, the id returned by the remote location is compared with the cache ID. If they match, the gray mark in the secondary location is set to white; otherwise, it remains gray and waits for the next round of scanning. During this period, multiple caches can be concatenated and sent concurrently. After the remote location returns a successful response, the relevant metadata needs to be persisted. After the secondary scan is completed, if all secondary caches turn white, the corresponding first-level ring element is set to white, and the scan continues along the ring, while the white expired caches are released.

[0065] Figure 3 This is a flowchart of a remote synchronization method provided in an embodiment of this application, such as... Figure 3 As shown,

[0066] Step 301: Scan the first-level circular table of the memory pool in the read / write direction. If a timeout white entry is encountered, delete it and proceed to step 302.

[0067] Step 302: Determine if the first-level circular list is entirely white; if so, the thread sleeps for a while and executes step 301; otherwise, execute step 303.

[0068] Step 303: Traverse all secondary caches corresponding to the first-level gray items and execute step 304.

[0069] Step 304: Determine if the second-level cache is completely empty; if yes, proceed to step 305; otherwise, proceed to step 306.

[0070] Step 305: Clear the corresponding first-level cache item and execute step 301.

[0071] Step 306: Synchronize the Level 3 cache slices corresponding to the Level 2 gray items to the remote location, and then execute Step 307.

[0072] Step 307: Determine if remote storage was successful. If yes, proceed to step 308; otherwise, proceed to step 309.

[0073] Step 308: Persist the metadata, change the original cache slice color, and then execute step 303.

[0074] Step 309: Determine if the number of retries has been reached. If yes, proceed to step 310; otherwise, proceed to step 306.

[0075] Step 310: Clear all gray storage information within the ring; white information that has not expired is still readable.

[0076] In some embodiments, writing data to the memory pool in step 102 can be achieved through steps B1 to B2, or through steps B1 and B3:

[0077] Step B1: Determine the target third-level cache slice based on the starting address of each third-level cache slice and the starting address of each second-level cache unit.

[0078] In this embodiment, the third-level cache slice refers to the smallest granularity of storage unit used to actually store data content. The size of the third-level cache slice is fixed and immutable. The second-level cache unit is a metadata structure that manages multiple third-level cache slices, recording information such as the offset range, color mark, and expiration time of each third-level cache slice. By comparing the offset of the current write position with the offset range covered by each second-level cache unit, the specific target third-level cache slice can be quickly located. This hierarchical mapping method can effectively reduce lookup overhead and improve write efficiency. In addition, since all third-level cache slices are of fixed size, there is no need to deal with fragmentation merging issues, thereby reducing system complexity.

[0079] Step B2: If the second storage identifier corresponding to the target third-level cache is the second identifier, modify the second storage identifier corresponding to the target third-level cache to the first identifier, and write the first data into the target third-level cache.

[0080] In this embodiment, the second storage identifier is a flag used to indicate the status of the third-level cache, typically used to distinguish whether the third-level cache is in a writable state. For example, the second identifier may indicate that a certain third-level cache is currently empty or unoccupied, while the first identifier indicates that a certain third-level cache has been used and is being written to. When the system detects that a certain third-level cache is in an idle state, the system updates the second storage identifier of the third-level cache to the first identifier and writes the first data to the third-level cache.

[0081] Step B3: If the second storage identifier corresponding to the target third-level cache is the first identifier, and the starting address corresponding to the target third-level cache matches the preset address, write the first data into the target third-level cache.

[0082] In this embodiment, when the third-level cache is already occupied, writing is only allowed if the starting address of the third-level cache matches a preset address. This preset address is typically a specific address value calculated based on the file offset, ensuring that written data does not overwrite existing data or damage the third-level cache structure.

[0083] In some embodiments, writing the first data to the target third-level cache includes: obtaining the first length of the first data; if the first length is greater than a first threshold, writing bytes of the second length to the target third-level cache, and returning the second length if the target third-level cache is successfully updated; wherein the second length is the difference between the first length and the first threshold; if the first length is equal to the first threshold, writing bytes of the first threshold to the target third-level cache, and returning the first threshold if the target third-level cache is successfully updated; incrementing the value corresponding to the first parameter and saving it to the target third-level cache.

[0084] In this embodiment, the first data refers to the data content to be written to the memory pool, typically originating from write operation requests in kernel space or user space. The first length represents the number of bytes occupied by the first data, used to determine whether it can be written to the target L3 cache in one go. The first length of the first data provides guidance for subsequent segmented writes and increment operations. For example, in a streaming write scenario, each write call may carry data of different sizes, and the system needs to determine whether segmented processing is required based on the first length of the first data.

[0085] In this embodiment, the first parameter typically refers to a version number or auto-incrementing ID used to identify the status of the target level 3 cache. The function of the first parameter is to record the number of times the target level 3 cache has been modified, facilitating remote storage synchronization and conflict detection. By incrementing the value corresponding to the first parameter, the system can mark that the target level 3 cache has been updated, and this update behavior will trigger subsequent synchronization and cleanup operations.

[0086] In this embodiment, the increment mechanism not only provides a historical record of changes to the target L3 cache, but also supports version control during concurrent writes. The increment mechanism helps maintain the consistency and integrity of the target L3 cache data. Especially in multi-threaded or multi-node environments, the increment mechanism can effectively prevent data overwriting or loss.

[0087] In this embodiment, the write process of the memory pool first calculates the write position and the length of the data to be written. Then, it determines whether to write directly or wait based on the color of the cache slice at the write position. Finally, after writing the data, it needs to update the metadata in the cache slice. The specific process is as follows:

[0088] Step 1: Locate the address of the third-level cache slice to be written. The write function takes data and offset as input. We assume the length of the third-level cache slice is M bytes, and the number of second-level cache units is N. Then, one unit on the first-level cache ring can represent a length of M * N bytes. Let the ring length be L, and the total size of the memory pool occupies PL (PL = L * M * N) bytes. The index of the first-level ring position corresponding to the offset is i (i = offset % PL % (M * N)), and the second-level index is j (j = (offset % PL - i * (M * N)) / M), to quickly locate the storage location at each level.

[0089] Step 2: Determine the write action based on the second-level metadata status. Assuming the cache slice is currently white or empty, any operation is allowed under concurrency-safe conditions. First, set it to gray (i.e., gray out the cache slice), and overwrite the data at the corresponding position of the cache slice with new data. Record the start and end values ​​of the new data, save the auto-incremented id, and return the result. If the cache slice is currently gray, it is necessary to determine whether the write offset is within the offset range represented by the current cache slice. If it is within this range, update the cache slice data and metadata while ensuring concurrency safety, and return the result. If it is not within this range, time out and wait for the cache slice to turn white before refreshing the data again.

[0090] Ideally, each write operation should return a result immediately after writing to the cache, and the kernel should immediately detect whether the write operation successfully wrote to the remote storage pool. This presents a contradiction. To alleviate the performance and consistency issues caused by this contradiction, and to address the problem that a write operation might fail first but succeed later, this method is designed to only inform the kernel of the result of the last write operation that synchronized to the remote storage. Intermediate write operations only return the result of writing to the cache. If the remote storage fails or times out in the intermediate steps, the kernel will also be notified of the write failure. It is not necessary to pay attention to which specific failure occurred, but only to whether the operation ultimately failed. If it failed, the relevant cache slices in the ring are cleared.

[0091] Because it is streaming data, it is impossible to detect whether there is still data to be written after a certain call to write. However, the length of the data written in the last call to write can always be 1. As long as the data length is 1, it can be regarded as the end of this write. Therefore, this method is not suitable for situations where the number of bytes written each time is small, which will lead to the degradation of cache performance.

[0092] Assuming `write` writes N data items, where N > 1, the memory pool only receives the first N-1 data items. It then notifies `write` to return, informing the kernel that N-1 bytes have been successfully written, allowing the kernel to initiate the next round of `write` calls. This ensures that the last data sent by `write` is always 1 byte. If N = 1, it needs to synchronously wait for all relevant data on the ring to be successfully written to the remote memory pool. If a data slice fails to be written remotely, the memory pool returns a failure, the kernel detects the `write` failure, and rewrites the data based on the saved metadata location.

[0093] In addition, the length of each cached data needs to be cut according to the size of the third-level cache slice to ensure that each cache does not cross cache slices.

[0094] Figure 4 This is a flowchart of a writing process provided in an embodiment of this application, such as... Figure 4 As shown,

[0095] Step 401: Locate the third-level cache slice based on the offset, and then proceed to step 402.

[0096] Step 402: Determine if the found third-level cache slice is grayed out; if yes, proceed to step 403; otherwise, proceed to step 404.

[0097] Step 403: Determine if the offset matches. If the offset matches (i.e., yes), proceed to step 406; if the offset does not match (i.e., no), proceed to step 405.

[0098] Step 404: Set the gray buffer and proceed to step 406.

[0099] Step 405: Determine if a timeout has occurred. If no timeout has occurred, sleep for the corresponding time (sleep ms); if a timeout has occurred, write returns an error.

[0100] Step 406: Determine if the write length N is greater than 1. If N is greater than 1, proceed to step 407; if N is less than or equal to 1, proceed to step 408.

[0101] Step 407: Write N-1 bytes into the buffer and determine whether the update was successful. If the update is successful, write returns the successful write length: N-1; if the update is unsuccessful, write returns an error.

[0102] Step 408: Write the byte into the buffer and check if the update was successful. If the update was unsuccessful, write returns an error. If the update was successful, check if the relevant buffer is completely empty. If the relevant buffer is completely empty, write returns a success write length: 1. If the relevant buffer is not completely empty, check if the write timed out. If the write timed out, write returns an error. If the write did not time out, check if the relevant buffer is completely empty.

[0103] In some embodiments, the timing of writing the last byte is determined based on the changes in bytes in the memory pool during each write operation; and security protection is implemented at the write timing.

[0104] This application monitors the data writing process in the memory pool and records the number of bytes and offset position corresponding to each write operation, thereby accurately determining which write operation is the last write operation.

[0105] When a write operation is detected, a series of security measures are initiated to ensure data integrity and consistency. These security measures include, but are not limited to: synchronizing the data in the current cache to the remote storage node, updating the metadata status (e.g., modifying color markers), and clearing the local cache after successful synchronization. Furthermore, if an error or timeout occurs during data synchronization, the system will notify the kernel layer to implement appropriate recovery strategies, such as resending the data or rolling back part of the written content.

[0106] Security protection refers to a series of operations taken to ensure data consistency and reliability upon detecting the completion of a write operation. Security protection includes persisting data from the local cache to remote storage, verifying successful data writing, and cleaning up cached resources that are no longer needed. By performing these security protection operations at the time of write, the system can effectively prevent data loss or corruption without impacting performance, thus improving its stability and data security.

[0107] In some embodiments, reading data from the memory pool in step 102 can be achieved through steps C1 to C2, or through steps C1 and C3:

[0108] Step C1: Determine the target third-level cache slice based on the storage offset range of the target data in the first file.

[0109] In this embodiment, the target data refers to the data block to be read, typically specified by a read request from the file system. The storage offset range represents the start and end positions of the target data in the first file, used to precisely locate the position of the target data within the first file.

[0110] Step C2: If the target third-level cache is located in the second-level cache unit, read the target data from the target third-level cache.

[0111] Step C3: If the target third-level cache slice is not located in the second-level cache unit, retrieve the target data from the remote storage space, and determine whether to write the target data to the memory pool based on the second storage identifier corresponding to the target third-level cache slice.

[0112] This application introduces remote storage space as a backup data source, ensuring that the required data can be provided in a timely manner even if the cache misses; at the same time, by combining the second storage identifier to determine whether to cache, the system can effectively control memory usage while ensuring performance and avoiding resource waste.

[0113] In this application, the read process of the memory pool first needs to calculate the read position and the length of the read data. Then, based on whether the read hits the cache, it decides whether to read the data directly or fetch the data remotely. The specific logic is as follows:

[0114] Reading does not change the color of the cache slice. The location of the cache slice is calculated based on the read offset. There are two cases: read hit and read miss. If the read offset falls into the offset range represented by the cache slice, it is a hit; otherwise, it is a read miss.

[0115] If a read hit occurs, there is no need to pay attention to the cache slice color. After ensuring concurrency safety, the data can be read and returned. At the same time, it is determined whether read-ahead is enabled. If enabled, it is determined whether several cache slices around this cache slice can be hit. If no hit occurs, a remote read request is initiated.

[0116] If a read miss occurs, a remote read data process is triggered. Upon success, the data is returned to the kernel, and the kernel determines whether the data can be added to the pool based on the cache slice color: if it is gray, wait for it to turn white, and then update the cache slice. During the update, the ID also needs to be incremented.

[0117] If read-ahead is required in certain scenarios, more data can be fetched from the remote end. If the index of the cache slice to be read is i, then additional cache slices with indices i+1, i+2, i+3, and i-1 that were not hit by reads (if any) need to be fetched. The number of cache slices to fetch depends on the business scenario. It has the advantages of acceleration and the disadvantage of read amplification. Whether to enable it can be decided based on the business.

[0118] Figure 5 This is a flowchart of a reading process provided in an embodiment of this application, such as... Figure 5 As shown,

[0119] Step 501: Determine if there is data in the cache. If yes, proceed to step 502; otherwise, proceed to step 503.

[0120] Step 502: Determine if the offset hits the second-level cache. If so, read the data directly and return the data using Read; otherwise, proceed to step 503.

[0121] Step 503: Remotely retrieve the cache slice and execute step 504.

[0122] Step 504: Determine whether the data was successfully acquired. If yes, proceed to step 505 and read the returned data; otherwise, proceed to step 507.

[0123] Step 505: Determine if the corresponding cache slice is white. If yes, proceed to step 506; otherwise, read the data directly and return the data using Read.

[0124] Step 506: Overwrite the cache slice and read the returned data.

[0125] Step 507: Determine if the requested target slice has timed out. If so, Read returns an error; otherwise, proceed to step 504.

[0126] In some embodiments, step 101, initializing the memory pool of the first file, includes the following steps: allocating storage space for the first-level cache ring and the second-level cache unit; setting the allocation timing for the cache slices in the third-level cache slice; wherein the allocation timing is used to indicate that allocation is performed when the cache slice is used; setting all metadata in the memory pool to null; and setting all tags in the memory pool to the second identifier.

[0127] In this embodiment of the application, when initializing the memory pool, the storage space of the first-level storage and the second-level storage is allocated first, and the third-level cache is allocated when it is used. All metadata in the memory pool is set to empty, and all markers in the pool are initialized to white.

[0128] In some embodiments, the scheme to be protected by this application includes the following: if the data writing time is longer than the timeout period, or the data reading time is longer than the timeout period, an error is returned.

[0129] In this application, if the duration of a data write operation exceeds a preset timeout threshold, or if a data read operation also exceeds the set timeout limit, the data write or read operation will be deemed abnormal, and an error message will be returned. This data operation timeout detection and abnormal feedback mechanism is used to prevent system resources from being ineffectively occupied due to prolonged unresponsive operations. Simultaneously, the system can promptly notify upper-layer applications to execute retry, interrupt, or other processing logic.

[0130] The write time in this application refers to the time interval from calling the write interface to the actual completion of the write operation; the read time refers to the time interval from initiating a read request to successfully retrieving the data. The timeout can be adjusted according to different application scenarios and network environments. For example, in a high-latency network environment, system administrators or applications can appropriately extend the timeout; in a low-latency scenario, system administrators or applications can set a shorter timeout to improve response speed.

[0131] In this application, a caching framework for a single file is provided. The file inode number corresponds to a three-level storage structure for fast read / write and memory reclamation, and horizontal expansion is possible for multiple files. The first-level circular queue can store block data, colors, etc., at arbitrary offsets, while the maximum capacity of the memory pool remains constant. The second-level storage marks the cache slice offset range, color, etc., with gray and white colors indicating the storage status of each level, used to quickly determine whether the third-level cache slice is readable and writable. White indicates that it is readable, writable, and can be deleted, while gray requires synchronization to the remote cache. The addition of pre-read logic can speed up read misses. The smallest cache slices in the third level are all independently stored, storing the actual data.

[0132] In this application, prefetching is a technique to optimize read performance. When a cache hit is detected, the system proactively reads data from adjacent cache misses and preloads this data into the local cache to reduce the response time of subsequent read requests. However, prefetching may lead to read amplification, and its activation should be selected based on the specific business scenario. For example, in a video player, when a user is watching a video segment, the data processing method can predict what the user will watch next and preload the corresponding cache segments, thereby reducing stuttering during playback.

[0133] Embodiments of this application provide a data processing apparatus that can be used to implement... Figure 1 A corresponding embodiment provides a data processing method, referring to... Figure 6 As shown, the data processing device 600 includes:

[0134] Processing unit 601 is used to initialize the memory pool of the first file; wherein, the memory pool corresponds to a three-level storage structure, the three-level storage structure includes a first-level cache ring using a circular queue, a second-level cache unit using contiguous storage space, and a third-level cache slice using contiguous storage space; the first-level cache ring includes multiple second-level cache units; the second-level cache unit includes multiple third-level cache slices;

[0135] The processing unit 601 is also used to write data to the memory pool, read data from the memory pool, or remotely synchronize data in the memory pool.

[0136] In other embodiments of this application, the ring element of the first-level cache ring is the starting address of each second-level cache unit and the first storage identifier of each second-level cache unit;

[0137] The second-level cache unit stores the starting address of each third-level cache slice, the timeout period of each third-level cache slice, the second storage identifier of each third-level cache slice, the storage offset range of the target data in the first file, and the first parameter; the first parameter is used to check whether it is written repeatedly.

[0138] The third-level cache contains the data for the first file.

[0139] In other embodiments of this application, the processing unit 601 is used to scan the first-level cache ring in a first direction from the starting position of writing data in the memory pool, and determine the target second-level cache unit; wherein, the target second-level cache unit is the second-level cache unit with the first storage identifier as the first identifier among a plurality of second-level cache units; the first direction is the read / write direction or the data growth direction;

[0140] Processing unit 601 is used to traverse the target second-level cache unit and determine the target third-level cache slice; wherein, the target third-level cache slice is the third-level cache slice whose second storage identifier is the first identifier among a plurality of third-level cache slices;

[0141] The sending unit 602 is used to synchronize the cache slices included in the target third-level cache slice to the remote location.

[0142] In other embodiments of this application, the processing unit 601 is used to determine the target third-level cache based on the starting address corresponding to each third-level cache slice and the starting address corresponding to each second-level cache unit.

[0143] Processing unit 601 is configured to modify the second storage identifier corresponding to the target third-level cache to the first identifier if the second storage identifier corresponding to the target third-level cache is the second identifier, and write the first data into the target third-level cache.

[0144] The processing unit 601 is configured to write the first data into the target third-level cache if the second storage identifier corresponding to the target third-level cache is the first identifier and the starting address corresponding to the target third-level cache matches the preset address.

[0145] In other embodiments of this application, the acquisition unit 603 is used to acquire the first length of the first data;

[0146] Processing unit 601 is configured to write bytes of a second length into a target third-level cache if the first length is greater than a first threshold, and return the second length if the target third-level cache is successfully updated; wherein the second length is the difference between the first length and the first threshold.

[0147] Processing unit 601 is configured to write bytes of the first threshold into the target third-level cache if the first length is equal to the first threshold, and return the first threshold if the target third-level cache is successfully updated.

[0148] The processing unit 601 is used to increment the value corresponding to the first parameter and save it to the target third-level cache.

[0149] In other embodiments of this application, the processing unit 601 is used to determine the timing of writing the last byte based on the byte changes in the memory pool during each write process;

[0150] The processing unit 601 is used to perform security protection during the write operation.

[0151] In other embodiments of this application, processing unit 601 is used to determine a target third-level cache slice based on the storage offset range of the target data in the first file;

[0152] The processing unit 601 is used to read target data from the target third-level cache if the target third-level cache is located in the second-level cache unit;

[0153] The processing unit 601 is used to obtain target data from remote storage space if the target third-level cache is not located in the second-level cache unit, and determine whether to write the target data into the memory pool based on the second storage identifier corresponding to the target third-level cache.

[0154] In other embodiments of this application, the processing unit 601 is used to allocate storage space for the first-level cache ring and the second-level cache unit;

[0155] Processing unit 601 is configured to set allocation timing for cache slices in the third-level cache slice; wherein, allocation timing is used to indicate that allocation should be performed when the cache slice is used;

[0156] Processing unit 601 is used to empty all metadata in the memory pool;

[0157] Processing unit 601 is used to set all tags in the memory pool to the second identifier.

[0158] In other embodiments of this application, the processing unit 601 is configured to return an error if the data writing time is greater than the timeout period or the data reading time is greater than the timeout period.

[0159] The descriptions of the above device embodiments are similar to those of the above method embodiments, and have similar beneficial effects. For technical details not disclosed in the device embodiments of this application, please refer to the descriptions of the method embodiments of this application for understanding.

[0160] It should be noted that, in the embodiments of this application, if the above-described data processing method is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiments of this application, or the part that contributes to the related technology, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a terminal device to execute all or part of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, mobile hard drives, read-only memory (ROM), magnetic disks, or optical disks. Thus, the embodiments of this application are not limited to any specific hardware and software combination.

[0161] Figure 7 This is a schematic structural diagram of an electronic device provided in an embodiment of this application. The electronic device may be a data processing device. Figure 7 The illustrated electronic device 700 includes a processor 710, which can call and run computer programs from memory to implement the methods in the embodiments of this application.

[0162] Optionally, such as Figure 7 As shown, the electronic device 700 may further include a memory 720. The processor 710 can retrieve and run computer programs from the memory 720 to implement the methods described in the embodiments of this application.

[0163] The memory 720 can be a separate device independent of the processor 710, or it can be integrated into the processor 710.

[0164] Optionally, such as Figure 7 As shown, the electronic device 700 may also include a transceiver 730, which the processor 710 can control to communicate with other devices. Specifically, it can send information or data to other devices or receive information or data sent by other devices.

[0165] The transceiver 730 may include a transmitter and a receiver. The transceiver 730 may further include antennas, and the number of antennas may be one or more.

[0166] Optionally, the electronic device 700 may specifically be a data processing device in the embodiments of this application, and the electronic device 700 may implement the corresponding processes implemented by the data processing device in the various methods of the embodiments of this application. For the sake of brevity, it will not be described in detail here.

[0167] This application also provides a computer program product, including a computer program that can be executed by the processor 710 of the electronic device 700 to perform the steps described in any of the foregoing methods.

[0168] It should be understood that the processor in the embodiments of this application may be an integrated circuit chip with signal processing capabilities. In implementation, the steps of the above method embodiments can be completed by integrated logic circuits in the processor's hardware or by instructions in software form. The processor described above can be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this application can be directly embodied in the execution of a hardware decoding processor, or executed by a combination of hardware and software modules in the decoding processor. The software modules can be located in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. The storage medium is located in memory, and the processor reads information from the memory and, in conjunction with its hardware, completes the steps of the above method.

[0169] As one embodiment, the processor may include one or more general-purpose central processing units (CPUs). Each of these processors may be a single-core processor or a multi-core processor. Here, "processor" may refer to one or more devices, circuits, and / or processing cores used for processing data (e.g., executing instructions).

[0170] It is understood that the memory in the embodiments of this application can be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. The non-volatile memory can be ROM, Programmable Read-Only Memory (PROM), Erasable Programmable Read-Only Memory (EPROM), Electrically Erasable Programmable Read-Only Memory (EEPROM), or flash memory. The volatile memory can be Random Access Memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), Synchronous DRAM (SDRAM), Double Data Rate Synchronous DRAM (DDR SDRAM), Enhanced Synchronous DRAM (ESDRAM), Synchronous Link DRAM (SLDRAM), and Direct Rambus RAM (DR RAM). It should be noted that the memory used in the systems and methods described herein is intended to include, but is not limited to, these and any other suitable types of memory.

[0171] This application also provides a computer-readable storage medium for storing computer programs.

[0172] Optionally, the computer-readable storage medium can be applied to the data processing apparatus / electronic device in the embodiments of this application, and the computer program causes the computer to execute the corresponding processes implemented by the data processing apparatus / electronic device in the various methods of the embodiments of this application. For the sake of brevity, it will not be described in detail here.

[0173] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product.

[0174] A computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the flow or function according to the embodiments of this application is generated. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, computer instructions may be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that a computer can store or a data storage device such as a server or data center that integrates one or more available media. The available media may be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., DVDs), or semiconductor media (e.g., solid-state drives (SSDs)).

[0175] The above provides a detailed description of the service processing method and electronic device provided in the embodiments of this application. Specific examples have been used to illustrate the principles and implementation methods of this application. The description of the above embodiments is only for the purpose of helping to understand the method and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

[0176] It should be understood that the phrases "an embodiment," "an embodiment," "an embodiment of this application," "the foregoing embodiment," "some implementations," or "some embodiments" mentioned throughout the specification mean that a specific feature, structure, or characteristic related to the embodiment is included in at least one embodiment of this application. Therefore, the phrases "an embodiment," "an embodiment," "an embodiment of this application," "the foregoing embodiment," "some implementations," or "some embodiments" appearing throughout the specification do not necessarily refer to the same embodiment. Furthermore, these specific features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. It should be understood that in the various embodiments of this application, the sequence numbers of the above-described processes do not imply a sequential order of execution; the execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application. The sequence numbers of the above-described embodiments of this application are merely descriptive and do not represent the superiority or inferiority of the embodiments.

[0177] Unless otherwise specified, any step performed by the data processing device / electronic device in the embodiments of this application may be performed by the processor of the data processing device / electronic device. Unless otherwise specified, the embodiments of this application do not limit the order in which the data processing device / electronic device performs the following steps. Furthermore, the methods used to process data in different embodiments may be the same or different methods.

[0178] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods, such as: multiple units or components can be combined, or integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the various components shown or discussed can be through some interfaces, and the indirect coupling or communication connection between devices or units can be electrical, mechanical, or other forms.

[0179] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units. They may be located in one place or distributed across multiple network units. Some or all of the units may be selected to achieve the purpose of this embodiment according to actual needs.

[0180] In addition, each functional unit in the various embodiments of this application can be integrated into one processing unit, or each unit can be a separate unit, or two or more units can be integrated into one unit; the integrated unit can be implemented in hardware or in the form of hardware plus software functional units.

[0181] The methods disclosed in the several method embodiments provided in this application can be arbitrarily combined without conflict to obtain new method embodiments.

[0182] The features disclosed in the several product embodiments provided in this application can be arbitrarily combined without conflict to obtain new product embodiments.

[0183] The features disclosed in the several method or device embodiments provided in this application can be arbitrarily combined without conflict to obtain new method or device embodiments.

[0184] Those skilled in the art will understand that all or part of the steps of the above method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer storage medium. When the program is executed, it performs the steps of the above method embodiments. The aforementioned storage medium includes various media that can store program code, such as mobile storage devices, ROMs, magnetic disks, or optical disks.

[0185] Alternatively, if the integrated units described above are implemented as software functional modules and sold or used as independent products, they can also be stored in a computer storage medium. Based on this understanding, the technical solutions of the embodiments of this application, or the parts that contribute to related technologies, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as mobile storage devices, ROMs, magnetic disks, or optical disks.

[0186] The singular forms “a,” “the,” and “the” used in the embodiments of this application and the appended claims are also intended to include the plural forms, unless the context clearly indicates otherwise.

[0187] It should be noted that in the various embodiments involved in this application, all steps or some steps may be performed, as long as a complete technical solution can be formed.

[0188] The above description is merely an embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A data processing method, characterized in that, The method includes: Initialize the memory pool for the first file; wherein, the memory pool corresponds to a three-level storage structure, the three-level storage structure includes a first-level cache ring using a circular queue, a second-level cache unit using contiguous storage space, and a third-level cache slice using contiguous storage space; the first-level cache ring includes multiple second-level cache units; the second-level cache unit includes multiple third-level cache slices; Write data to the memory pool, read data from the memory pool, or remotely synchronize data in the memory pool.

2. The method according to claim 1, characterized in that, The ring elements of the first-level cache ring are the starting address of each second-level cache unit and the first storage identifier of each second-level cache unit; The second-level cache unit stores the starting address of each third-level cache slice, the timeout period of each third-level cache slice, the second storage identifier of each third-level cache slice, the storage offset range of the target data in the first file, and a first parameter; the first parameter is used to verify whether the data is written repeatedly. The third-level cache contains the data of the first file.

3. The method according to claim 2, characterized in that, Remote synchronization of data in the memory pool includes: Starting from the position where data is written from the memory pool, the first-level cache ring is scanned in a first direction, and the target second-level cache unit is determined; wherein, the target second-level cache unit is the second-level cache unit among the plurality of second-level cache units whose first storage identifier is a first identifier; the first direction is the read / write direction or the data growth direction; Traverse the target second-level cache units and determine the target third-level cache slice; wherein, the target third-level cache slice is the third-level cache slice whose second storage identifier is the first identifier among the plurality of third-level cache slices; Synchronize the cache slices included in the target level 3 cache slice to the remote location.

4. The method according to claim 2, characterized in that, Writing data to the memory pool includes: The target third-level cache slice is determined based on the starting address of each third-level cache slice and the starting address of each second-level cache unit. If the second storage identifier corresponding to the target third-level cache is the second identifier, modify the second storage identifier corresponding to the target third-level cache to the first identifier, and write the first data into the target third-level cache; If the second storage identifier corresponding to the target third-level cache is the first identifier, and the starting address corresponding to the target third-level cache matches the preset address, the first data is written to the target third-level cache.

5. The method according to claim 4, characterized in that, The step of writing the first data into the target third-level cache includes: Get the first length of the first data; If the first length is greater than the first threshold, the bytes of the second length are written to the target third-level cache, and if the target third-level cache is successfully updated, the second length is returned; wherein, the second length is the difference between the first length and the first threshold; If the first length is equal to the first threshold, write the bytes of the first threshold into the target third-level cache, and return the first threshold if the target third-level cache is successfully updated; The value corresponding to the first parameter is incremented and saved to the target third-level cache.

6. The method according to claim 4, characterized in that, The method further includes: The timing of writing the last byte is determined based on the changes in bytes in the memory pool during each write operation. Security protection is implemented during the write operation.

7. The method according to claim 2, characterized in that, Reading the data from the memory pool includes: Based on the storage offset range of the target data in the first file, the target third-level cache slice is determined; If the target third-level cache slice is located in the second-level cache unit, read the target data from the target third-level cache slice; If the target third-level cache slice is not located in the second-level cache unit, the target data is obtained from the remote storage space, and based on the second storage identifier corresponding to the target third-level cache slice, it is determined whether to write the target data into the memory pool.

8. The method according to claim 1, characterized in that, The initialization of the memory pool for the first file includes: Allocate storage space for the first-level cache ring and the second-level cache unit; An allocation timing is set for the cache slices in the third-level cache slice; wherein, the allocation timing is used to indicate that allocation should be performed when the cache slice is used; Set all metadata in the memory pool to null; Set all tags in the memory pool to the second identifier.

9. The method according to claim 2, characterized in that, The method further includes: If the data writing time exceeds the timeout period, or the data reading time exceeds the timeout period, an error is returned.

10. An electronic device, characterized in that, The electronic device includes: A memory for storing computer-executable instructions or computer programs; a processor for executing the computer-executable instructions or computer programs stored in the memory to implement the data processing method according to any one of claims 1 to 9.

Images