Method for accelerating data migration of device, electronic device, medium and program product

Abstract

This application discloses a data migration method, electronic device, medium, and program product for an acceleration device, relating to the field of storage technology. The method includes: determining data information of target cached data to be migrated from the internal storage space of the acceleration device, wherein the acceleration device is connected to multiple sets of external storage spaces, and the external storage spaces include multiple external storage regions with the same region information, and the external storage regions in different external storage spaces have different region information; searching for a target storage space whose region information matches the data information from the multiple sets of external storage spaces; and migrating the target cached data from the internal storage space to the target storage region within the multiple external storage regions included in the target storage space. This application solves the technical problem of low data storage efficiency for cached data in acceleration devices, achieving the technical effect of improving the data storage efficiency of cached data in acceleration devices.

Description

Technical Field

[0001] This application relates to the field of storage technology, and in particular to a method for accelerating data migration of devices, electronic devices, media, and program products. Background Technology

[0002] In large-scale model inference architectures, the internal storage space of acceleration devices can be used to store cached data that can be reused in subsequent inference processes, thereby improving the overall inference speed. However, the internal storage space of acceleration devices is limited. In scenarios involving extremely long sequence inference (such as generating text of tens of thousands of words) and high-concurrency inference, inference tasks are prone to interruption due to insufficient internal storage space. Blindly increasing the number of acceleration devices to meet internal storage space requirements will lead to an exponential increase in hardware deployment costs, becoming a core technical bottleneck for the large-scale deployment of large models.

[0003] In related technologies, Flash memory or distributed memory caches are used to store cached data. However, when using these media to store cached data, frequent I / O (Input / Output) operations are required, which affects the efficiency of inference operations; moreover, the random data storage method adopted when storing data easily leads to storage space fragmentation, resulting in serious waste of storage space. Summary of the Invention

[0004] This application provides a data migration method, electronic device, medium, and program product for accelerating devices, to at least solve the problem of low data storage efficiency for cached data in accelerating devices in related technologies.

[0005] This application provides a data migration method for an acceleration device, comprising: determining the data information of target cached data to be migrated in the internal storage space of the acceleration device, wherein the internal storage space is used to store cached data generated during the execution of computing tasks by the acceleration device, the acceleration device is connected to multiple sets of external storage spaces, the external storage spaces include: multiple external storage areas with the same area information, and the external storage areas in different external storage spaces have different area information; searching for a target storage space whose area information matches the data information from the multiple sets of external storage spaces; and migrating the target cached data from the internal storage space to the target storage area in the multiple external storage areas included in the target storage space.

[0006] This application also provides an electronic device, including: a memory for storing a computer program; and a processor for implementing the data migration method of any of the above-described acceleration devices when executing the computer program.

[0007] This application also provides a computer-readable storage medium storing a computer program, wherein when the computer program is executed by a processor, it implements the steps of the data migration method of any of the above-described acceleration devices.

[0008] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the data migration method of any of the above-described acceleration devices.

[0009] This application connects the acceleration device to multiple sets of external storage spaces. These external storage spaces include multiple external storage regions with the same region information, but different regions within different external storage spaces have different region information. The application determines the target cache data to be migrated from the internal storage space of the acceleration device. The internal storage space stores cache data generated during the acceleration device's computational tasks. Using the internal storage space as a buffer for cache data reduces I / O interactions with external storage spaces. Then, the application searches among the multiple sets of external storage spaces for a target storage space whose region information matches the data information. The target cache data is then migrated from the internal storage space to the target storage region within the multiple external storage regions included in the target storage space. Migrating the target cache data to a matching target storage region avoids wasting storage space caused by using unsuitable storage spaces. Therefore, this application solves the technical problem of low data storage efficiency for cache data in acceleration devices in related technologies, achieving the technical effect of improving the data storage efficiency of cache data in acceleration devices. Attached Figure Description

[0010] To more clearly illustrate the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0011] Figure 1 This is a hardware structure block diagram of the data migration method of the acceleration device according to an embodiment of this application;

[0012] Figure 2 This is a flowchart of a data migration method for an acceleration device according to an embodiment of this application;

[0013] Figure 3 This is an architecture diagram of a KV Pool based on Flash storage according to an embodiment of this application;

[0014] Figure 4 This is an overall flowchart of a cache data storage method based on a key-value cache pool architecture according to an embodiment of this application;

[0015] Figure 5 This is a flowchart illustrating the writing process of a cache data storage method based on a key-value cache pool architecture according to an embodiment of this application.

[0016] Figure 6 This is a flowchart illustrating the reading process of a cache data storage method based on a key-value cache pool architecture according to an embodiment of this application.

[0017] Figure 7 This is a flowchart illustrating the recycling process of a cache data storage method based on a key-value cache pool architecture according to an embodiment of this application.

[0018] Figure 8 This is a structural block diagram of a data migration apparatus for an acceleration device according to an embodiment of this application. Detailed Implementation

[0019] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the protection scope of this application.

[0020] It should be noted that, in the description of this application, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. The terms "first," "second," etc., in this application are used to distinguish similar objects and are not used to describe a specific order or sequence.

[0021] To enable those skilled in the art to better understand the present application, the present application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0022] The specific application environment architecture or specific hardware architecture on which the data migration method for the accelerated device depends is described here.

[0023] The methods and embodiments provided in this application can be executed on a server device or a similar computing device. Taking running on a server device as an example, Figure 1 This is a hardware structure block diagram of the data migration method of the acceleration device according to an embodiment of this application. Figure 1 As shown, the server device may include one or more ( Figure 1Only one is shown in the image. A processor 102 (which may include, but is not limited to, a central processing unit (CPU), microprocessor (MCU), or programmable logic device (FPGA), etc.) and a memory 104 for storing data are also shown. The server device may further include a transmission device 106 for communication functions and an input / output device 108. Those skilled in the art will understand that... Figure 1 The structure shown is for illustrative purposes only and does not limit the structure of the server equipment described above. For example, the server equipment may also include components that are more... Figure 1 The more or fewer components shown, or having the same Figure 1 The different configurations shown.

[0024] The memory 104 can be used to store computer programs, such as application software programs and modules, like the computer program corresponding to the data migration method of the acceleration device in this embodiment. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, thus implementing the above-described method. The memory 104 may include high-speed random access memory and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include memory remotely located relative to the processor 102, and these remote memories can be connected to server devices via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0025] The transmission device 106 is used to receive or send data via a network. Specific examples of the network described above may include a wireless network provided by a communication provider for the server device. In one example, the transmission device 106 includes a Network Interface Controller (NIC), which can connect to other network devices via a base station to communicate with the Internet. In another example, the transmission device 106 may be a Radio Frequency (RF) module used for wireless communication with the Internet.

[0026] The embodiments of this application provide a method for accelerating data migration of a device. The method is described in detail below in conjunction with the execution flow of the method for accelerating data migration of a device.

[0027] This embodiment provides a method for accelerating device data migration. Figure 2 This is a flowchart of a data migration method for an acceleration device according to an embodiment of this application, such as... Figure 2 As shown, the method includes the following steps:

[0028] Step S202: Determine the data information of the target cache data to be migrated in the internal storage space of the acceleration device. The internal storage space is used to store cache data generated during the execution of computing tasks by the acceleration device. The acceleration device is connected to multiple external storage spaces. The external storage spaces include: multiple external storage areas with the same area information, and the external storage areas in different external storage spaces have different area information.

[0029] Step S204: Search for the target storage space that matches the corresponding region information and data information from multiple sets of external storage spaces;

[0030] Step S206: Migrate the target cache data from the internal storage space to the target storage area among the multiple external storage areas included in the target storage space.

[0031] Through the above, the acceleration device connects to multiple sets of external storage spaces. These external storage spaces include multiple external storage regions with the same regional information, but different external storage regions have different regional information. The data information of the target cached data to be migrated from the internal storage space of the acceleration device is determined. The internal storage space is used to store cached data generated during the acceleration device's computational tasks. Using the internal storage space as a buffer for cached data reduces I / O interactions with external storage spaces. Then, the target storage space matching the corresponding regional information and data information is searched from the multiple sets of external storage spaces. The target cached data is migrated from the internal storage space to the target storage region within the multiple external storage regions included in the target storage space. Migrating the target cached data to the matching target storage region avoids the waste of storage space caused by using unsuitable storage spaces to store the target cached data. Therefore, this solves the technical problem of low data storage efficiency for cached data in related technologies, achieving the technical effect of improving the data storage efficiency of cached data in acceleration devices.

[0032] Optionally, in this embodiment, the acceleration device may refer to, but is not limited to, hardware components specifically designed to offload and accelerate specific computing tasks of the host CPU (Central Processing Unit). Optionally, the acceleration device may include, but is not limited to, GPUs (Graphics Processing Units), ASICs (Application-Specific Integrated Circuits) (such as TPUs (Tensor Processing Units), NPUs (Neural Processing Units), and DSAs (Data-Structure Accelerators).

[0033] In the embodiment provided in step S202, the internal storage space of the acceleration device may refer to, but is not limited to, the physical storage medium integrated inside the acceleration device. Optionally, the internal storage space of the acceleration device may include, but is not limited to, VRAM (Video Random Access Memory) integrated in the GPU (Graphics Processing Unit) (such as HBM (High Bandwidth Memory)).

[0034] Optionally, in this embodiment, the data information of the target cached data may be, but is not limited to, data attributes used to indicate the target cached data, such as data volume, the size relationship between the data volume and the preset amount, data type, etc.

[0035] Optionally, in this embodiment, the data information of the target cache data may also include, but is not limited to, task attributes used to indicate the target computing task that generates the target cache data, such as whether the task has been completed, the urgency of the task, the frequency of task reuse, etc.

[0036] Optionally, in the embodiments, a two-layer storage space may be provided for the acceleration device, but is not limited to. The first layer may be, but is not limited to, the internal storage space of the acceleration device, serving as a buffer for storing cached data generated during the execution of computing tasks by the acceleration device in real time. The second layer may be, but is not limited to, the external storage space of the acceleration device, used to store the cached data accumulated in the buffer.

[0037] Optionally, in this embodiment, the acceleration device may, but is not limited to, connecting to multiple sets of external storage spaces. Optionally, these multiple sets of external storage spaces may, but are not limited to, be on the same external storage device, or they may, but are not limited to, different external storage devices; this application does not limit this.

[0038] Optionally, in this embodiment, the external storage device connected to the acceleration device may include, but is not limited to, SSD (Solid State Drive), HBF (High Bandwidth Flash), etc.

[0039] Optionally, in this embodiment, each group of external storage space may include, but is not limited to, multiple external storage regions with the same region information, but the external storage regions in different groups of external storage space have different region information.

[0040] Optionally, in this embodiment, the area information may be used, but is not limited to, to indicate the storage medium attributes of the storage area, such as storage capacity, storage rate, storage medium type, storage medium security level, etc.

[0041] Optionally, in this embodiment, the acceleration device may be connected to an external storage device, but is not limited to that connected to one. The storage space in the external storage device may be divided into multiple groups of external storage spaces, and each group of external storage spaces may include multiple external storage regions. The storage capacity of the external storage regions included in each group of external storage spaces is equal, while the storage capacity of the external storage regions included in different groups of external storage spaces is not equal.

[0042] Optionally, in this embodiment, the acceleration device may also be connected to multiple external storage devices, but is not limited to. Each external storage device may correspond to a set of external storage spaces. Multiple external storage devices may be equally divided into multiple external storage regions, but the granularity of the division may be different for different external storage devices.

[0043] Optionally, in this embodiment, the acceleration device may also be connected to, but is not limited to, multiple external storage devices. Each external storage device may correspond to, but is not limited to, a set of external storage spaces. Each external storage device may be divided into multiple external storage regions in various ways.

[0044] Optionally, in this embodiment, the data information of the target cache data to be migrated can be determined first, that is, the data attributes of the target cache data to be migrated can be determined first, so as to prepare for selecting a suitable storage space and storage area to match the data attributes.

[0045] In the embodiment provided in step S204, it is possible, but not limited to, to search for the target storage space that matches the corresponding area information and data information from multiple sets of external storage spaces. That is, it is possible, but not limited to, to match the target cache data with a certain data attribute with the storage medium attribute that is compatible with the data attribute.

[0046] Optionally, in this embodiment, the target storage space matching the corresponding region information and data information can be found from multiple groups of external storage spaces in the following ways, but not limited to: matching the target security level to the target requirement parameter among the security requirement parameters and media security levels that have a corresponding relationship, wherein the target requirement parameter is used to indicate the security importance requirements of the target computing task that generates the target cache data, the data information includes the target requirement parameter, the region information includes the media security level of the external storage space, and external storage spaces with different media security levels are configured with different security policies, the higher the media security level, the more stringent the security policy requirements; and determining the external storage space whose corresponding media security level is equal to or higher than the target security level as the target storage space.

[0047] In the embodiment provided in step S206, a storage area may be selected from multiple external storage areas included in the target storage space as the target storage area for storing the target cache data, and then the target cache data may be migrated from the internal storage space to the target storage area.

[0048] Optionally, in this embodiment, when selecting a storage region as the target storage region from among the multiple external storage regions included in the target storage space, the selection can be made based on, but is not limited to, different media properties among the multiple external storage regions included in the target storage space. For example, the selection can be made based on whether the external storage region has been used, or based on, but is not limited to, the storage capacity of the external storage region, etc.

[0049] Optionally, in this embodiment, when migrating target cache data from internal storage space to target storage area, CPU-based IO writing can be used, or, but not limited to, DMA (Direct Memory Access) can be used to directly write the target cache data to the target storage area.

[0050] As an optional implementation, this application proposes a KV Pool architecture based on Flash storage, using a data migration method for accelerated devices. This architecture designs and implements a KVPool management mechanism on Flash storage devices such as HBF. While fully exploiting the hardware bandwidth potential of the storage device, it uniformly schedules and manages the KV (Key-Value) data throughout the entire lifecycle of inference requests (i.e., computational tasks), significantly improving the read / write efficiency of KV data, effectively expanding GPU memory capacity, and alleviating the GPU memory resource pressure for large model inference.

[0051] Figure 3This is an architecture diagram of a KV Pool based on Flash storage, according to an embodiment of this application. Figure 3 As shown, the overall logical architecture is divided into three layers: the computing layer, which includes general-purpose computing devices such as CPU, GPU / NPU / LPU, and HBM (i.e., internal storage space), responsible for core inference computing; the storage layer, which includes storage devices such as HBF (i.e., external storage space) and a KV cache pool management module, which acts as a KV Pool key-value cache pool; and the network layer, which includes a large model inference cross-node transmission protocol interface module (i.e., protocol Agent) and a large model inference cross-node transmission protocol Switch device, responsible for cross-node KVCache (key-value cache) addressing, transmission, and access control.

[0052] Optionally, in this embodiment, the overall architecture may, but is not limited to, follow a two-level cache structure of HBM (Level 1 cache) + HBF / SSD (Level 2 storage). The core is to use HBM memory connected to GPUs as Level 1 cache and HBF / SSD as Level 2 storage. On the HBF / SSD side, all key-value data is managed uniformly through the KV Pool, making full use of HBM's high bandwidth and low latency as a write buffer. SSD / HBF serves as massive persistent storage, maximizing SSD throughput through strict sequential append writes, and using object-oriented metadata management to achieve efficient garbage collection and sequence retrieval.

[0053] Level 1 cache can be, but is not limited to, the HBM memory within the GPU. Its main function is to cache key-value data (i.e., target cache data) generated during inference (computation task execution) in real time, reducing the frequency of data flushing to disk and reducing I / O overhead. A dedicated buffer area is set up in the HBM memory for temporary storage of key-value data. The core management threshold of the buffer is 1GB. All key-value data is written to the buffer in tokens, one token at a time, in sequence. Random writes are not allowed to ensure the continuity of data storage.

[0054] The secondary storage uses SSD / HBF as the core persistent carrier for KV data, and manages all physical storage blocks uniformly through a key-value buffer pool mechanism. This key-value buffer pool employs a hybrid physical block strategy, pre-dividing six contiguous physical segments (i.e., external storage areas) of different granularities: 1GB, 500MB, 200MB, 100MB, 50MB, and 10MB. All physical blocks are organized in a contiguous storage manner, providing a foundation for sequential writing. This multi-granularity pre-division mechanism can flexibly adapt storage space according to the actual length of different inference sequences, effectively reducing internal fragmentation while ensuring the performance of large-block contiguous writes, achieving a balance between storage efficiency and write performance.

[0055] The caching mechanism is implemented through the KV cache pool management module, which is interconnected with other modules via an internal high-speed bus.

[0056] As an optional implementation, the target storage space that matches the corresponding region information and data information can be found from multiple sets of external storage spaces in the following ways: finding at least one first storage space from multiple sets of external storage spaces where the storage amount of the corresponding external storage region is greater than or equal to the data amount of the target cached data, wherein the region information includes: storage amount, and the data information includes: data amount; and determining the target storage space from at least one first storage space.

[0057] Optionally, in this embodiment, the multiple sets of external storage spaces may be divided according to, but are not limited to, the storage capacity of the included external storage regions.

[0058] Optionally, in this embodiment, it is possible, but not limited to, searching for at least one first storage space from multiple sets of external storage spaces where the storage capacity of the included external storage area is greater than or equal to the data capacity of the target cache data, and then determining a target storage space from at least one first storage space, thereby avoiding the selection of a storage space with insufficient storage capacity and avoiding data migration failure.

[0059] As an optional implementation, the target storage space can be determined from at least one first storage space by, but is not limited to, the first storage space with the smallest corresponding storage amount among at least one first storage space.

[0060] Optionally, in this embodiment, the first storage space with the smallest storage capacity, provided that the storage capacity is sufficient to store the target cache data, can be selected as the target storage space to reduce the waste of storage resources.

[0061] As an optional implementation, the target storage space can also be determined from at least one first storage space by: calculating the difference between the storage amount and the data amount corresponding to each of the first storage spaces in the at least one first storage space; and selecting the target storage space from the first storage spaces whose corresponding difference is less than the target threshold.

[0062] Optionally, in this embodiment, a first storage space with sufficient storage capacity to store the target cached data and a certain amount of spare storage capacity can be selected as the target storage space. A small amount of redundant storage resources can be reserved to adapt to new storage needs of cached data in the future (for example, to ensure the security of cached data, the hash value of the cached data should be stored together in the external storage area).

[0063] As an optional implementation, the target storage space that matches the corresponding region information and data information can also be found from multiple sets of external storage spaces in the following way: finding at least one second storage space from multiple sets of external storage spaces where the corresponding region information matches the data type of the target cached data, wherein the region information is used to indicate the media attributes of the storage medium of the corresponding external storage region, and the data information includes: data type; determining the target storage space from at least one second storage space.

[0064] Optionally, in this embodiment, the multiple sets of external storage spaces may be divided according to, but are not limited to, the media properties of the included external storage regions.

[0065] Optionally, in this embodiment, the region information may, but is not limited to, be used to indicate the media type of the corresponding external storage region. External storage regions of different media types may, but are not limited to, be suitable for storing data of different data types; that is, different media types may, but are not limited to, be matched with different data types. For example, HDD (Hard Disk Drive), due to its disk rotation and mechanical seek mechanism, has extremely high capacity density but high random access latency, making it suitable for storing massive amounts of non-critical historical log data with low access frequency; SSD (Solid State Drive) utilizes flash memory chips for electronic addressing, possessing high sequential and random read / write performance, making it suitable for storing intermediate results of regular inference processes with certain latency requirements and moderate data volumes; while HBF (High Bandwidth Flash) has extremely low access latency and extremely high throughput, suitable for storing KV (Key-Value) cache data that requires frequent, high-speed read / write operations during large model inference processes, thereby ensuring the real-time performance and smoothness of the inference process.

[0066] Optionally, in this embodiment, the acceleration device first obtains the data type of the target cache data to be migrated, and then traverses multiple connected external storage spaces to filter out at least one second storage space whose media attributes match the current data type. For example, frequently read / written hot data is matched to a high-performance cache medium, and frequently read / written cold data is matched to a large-capacity, low-speed medium, thereby determining the most suitable target storage space from the candidate set. This precise matching mechanism based on media attributes and data type enables the dynamic selection of the optimal storage path according to the actual data access pattern, avoiding read / write bottlenecks or premature wear and tear of storage media caused by blind migration, and significantly improving the overall efficiency of data migration.

[0067] As an optional implementation, the target storage space can be determined from at least one second storage space by, but not limited to, searching for at least one third storage space in the at least one second storage space whose storage size of the corresponding external storage area is greater than or equal to the data size of the target cache data; and determining the target storage space from at least one third storage space.

[0068] Optionally, in this embodiment, if at least one second storage space has been determined, a secondary screening based on capacity constraints can be performed. That is, from the second storage spaces with matched media attributes, at least one third storage space with a corresponding external storage area storage capacity greater than or equal to the data volume of the target cache data can be found. This step can exclude those storage areas that have matched media attributes but insufficient physical capacity, ensuring that the subsequently selected storage areas can fully accommodate the data to be migrated.

[0069] Optionally, in this embodiment, the method of determining the target storage space from at least one third storage space may be, but is not limited to, the method of determining the target storage space from at least one first storage space, and will not be described in detail here.

[0070] As an optional implementation method, such as Figure 3 As shown, the aforementioned architecture may include, but is not limited to, storage layer circuit logic. Through this storage layer circuit logic, the SSD KV Pool is divided into hybrid-granularity physical segments. Specifically, after the SSD device completes initialization, the overall available storage space of the SSD can be divided into multiple contiguous physical segments of different capacities (i.e., different external storage regions within different external storage spaces). This pre-partitioning method avoids the performance loss caused by repeatedly searching contiguous storage spaces during architecture operation and ensures that the data within each physical segment remains physically continuous. Each physical segment corresponds to a contiguous physical address region within the SSD and is uniformly managed using a dedicated metadata structure.

[0071] Optionally, in this embodiment, physical segments can be divided into six specifications, but not limited to: physical segments of specification level L0, with a segment size of 10MB, accounting for 3% of the total number of physical segments of all specifications; physical segments of specification level L1, with a segment size of 50MB, accounting for 7% of the total number of physical segments of all specifications; physical segments of specification level L2, with a segment size of 100MB, accounting for 10% of the total number of physical segments of all specifications; physical segments of specification level L3, with a segment size of 200MB, accounting for 15% of the total number of physical segments of all specifications; physical segments of specification level L4, with a segment size of 500MB, accounting for 25% of the total number of physical segments of all specifications; and physical segments of specification level L5, with a segment size of 1GB, accounting for 40% of the total number of physical segments of all specifications.

[0072] Optionally, in this embodiment, each SSD physical segment may be equipped with complete metadata information (struct KVSegment, Segment segment data structure) to record the physical segment's own attributes, relationships, usage status, and other content. Specifically, the Segment data structure may include, but is not limited to, the following fields: Physical Segment Unique Identifier: A 32-bit unsigned integer serving as a unique identifier for each physical segment, ensuring global uniqueness; Associated Inference Object Number: A 32-bit unsigned integer recording the inference object (i.e., computation task) number to which the current physical segment belongs; Granularity Type: Marks the physical segment as belonging to a specific specification level from L0 to L5; Physical Starting Address: A 64-bit unsigned integer recording the starting offset address of the physical segment in the SSD, measured in bytes; Used Data Capacity: A 64-bit unsigned integer representing the amount of valid data written within the physical segment, measured in bytes; Occupancy Status: A boolean identifier, where true indicates the physical segment is allocated and used, and false indicates the physical segment is idle; Write Timestamp: A 64-bit unsigned integer recording the time data was written to the physical segment, used as a basis for determining subsequent garbage collection priorities.

[0073] The Segment data structure may include, but is not limited to, the following functional logic: Maximum capacity calculation: Based on the granularity specification corresponding to the physical segment, convert the capacity unit to bytes to obtain the total storage capacity of the physical segment; Full capacity judgment: Compare the used data capacity of the physical segment with its own maximum capacity to determine whether the current physical segment is full.

[0074] Optionally, in this embodiment, all idle physical segments can be classified and managed, and a separate idle linked list can be set up for each specification, named Free_List_L0 to Free_List_L5 respectively. Free_List_L0 records idle physical segments of specification level L0, Free_List_L1 records idle physical segments of specification level L1, and so on.

[0075] Optionally, in this embodiment, when allocating storage, the estimated size of the data to be stored (i.e., the target cache data) can be considered to select the physical segment with the smallest matching size, preventing space waste caused by storing small-volume data in large-capacity physical segments. When storage space needs to be allocated for the inference object, a physical segment is selected from the free list of the corresponding granularity; when the physical segment is used up and released, it will be returned to the free list of the original granularity for reuse.

[0076] As an optional implementation, the target storage space that matches the corresponding region information and data information can be found from multiple sets of external storage spaces in the following ways, but not limited to: when the data information indicates that the amount of target cached data has reached a data amount threshold, the external storage space with the largest storage capacity among the multiple sets of external storage spaces is determined as the target storage space, wherein the target cached data is generated by executing the target computing task, and the region information includes: the storage capacity of the external storage region, and the maximum storage capacity is set according to the data amount threshold; when the data information indicates that the target computing task has been completed, the target storage space is determined from at least one fourth storage space among the multiple sets of external storage spaces whose corresponding external storage region storage capacity is greater than or equal to the data amount of the target cached data, wherein the data information is also used to indicate the data amount.

[0077] Optionally, in this embodiment, the maximum storage capacity of the external storage area in the multiple external storage spaces can be, but is not limited to, determined based on a data volume threshold that can be used to trigger data migration. Specifically, the maximum storage capacity can be, but is not limited to, directly equal to the data volume threshold, or the maximum storage capacity can be, but is not limited to, equal to the sum of the data volume threshold and a pre-set redundancy capacity value, wherein the redundancy capacity value can be, but is not limited to, determined based on the maximum data volume of the data expected to be stored together with the target cache data.

[0078] Optionally, in this embodiment, when the amount of data used to indicate the target cached data reaches a data volume threshold, i.e. when the data volume threshold is reached and data migration is triggered, the external storage space with the largest storage capacity among multiple external storage spaces can be directly selected as the target storage space.

[0079] Optionally, in this embodiment, the cached data can be stored in units of computing tasks, but is not limited to. Therefore, data migration can be triggered even when the target computing task has been completed.

[0080] Optionally, in this embodiment, when data migration is triggered after the completion of the target computing task, the target storage space can be selected from multiple sets of external storage spaces, and the storage capacity of the included external storage areas can be sufficient to store the target cached data.

[0081] By implementing the above methods, the matching logic for matching target storage spaces based on storage volume and data volume can be reduced, thereby increasing the overall speed of the data migration process.

[0082] As an optional implementation, target cache data can be migrated from internal storage space to a target storage area among multiple external storage areas included in the target storage space in the following ways, but not limited to: determining the target storage area from multiple external storage areas included in the target storage space based on storage status information recorded in the acceleration device, wherein the storage status information is used to indicate the usage status of each external storage area; and migrating the target cache data from internal storage space to the target storage area.

[0083] Optionally, in this embodiment, after determining the target storage space, the target storage area can be selected by considering the usage status (i.e., whether data has been stored) of multiple external storage areas included in the target storage space. Storage status information can be used, but is not limited to, to indicate whether data has been stored in each external storage area.

[0084] By using the above methods, data migration failures caused by data storage conflicts can be avoided, ensuring that each data migration process can be executed normally.

[0085] As an optional implementation, the target storage area can be determined from multiple external storage areas included in the target storage space based on the storage status information recorded in the acceleration device in the following manner: searching for at least one candidate storage area corresponding to the target storage space from a first information table, wherein the storage status information includes the first information table, which records external storage spaces and unused storage areas with corresponding relationships; selecting the target storage area from at least one candidate storage area, and deleting the target storage area from the first information table.

[0086] Optionally, in this embodiment, the first information table may, but is not limited to, record corresponding external storage spaces and unstored areas. Specifically, the first information table may record information in the form of, but is not limited to, storage space specification: area identifier of external storage area; or it may, but is not limited to, storage space type: area identifier of external storage area, etc.

[0087] Optionally, in this embodiment, the first information table may include, but is not limited to, the aforementioned Free_List_L0 to Free_List_L5.

[0088] Optionally, in this embodiment, selecting a target storage region from at least one candidate storage region may include, but is not limited to, randomly selecting a target storage region from at least one candidate storage region, or selecting the candidate storage region whose address is closest to that of the storage region corresponding to the previous migration write operation from at least one candidate storage region as the target storage region, etc.

[0089] Optionally, in this embodiment, the region identifier of the target storage region may be deleted from the first information table after the target storage region has been determined, but is not limited to this.

[0090] As an optional implementation, after searching for at least one candidate storage region corresponding to the target storage space from the first information table, the external storage space may be adjusted in the following ways, but not limited to: if no candidate storage region is found from the first information table, a reference storage space is obtained from other storage spaces besides the target storage space in a plurality of external storage spaces, where the first storage amount of the external storage region is greater than the second storage amount of the external storage region in the target storage space; a first storage region is obtained from at least one alternative storage region corresponding to the reference storage space recorded in the first information table; the target storage region is divided out from the first storage region, and the first storage region is deleted from the first information table.

[0091] Optionally, in this embodiment, if at least one candidate storage area cannot be found in the first information table, i.e., if the external storage area included in the target storage space is insufficient, the external storage areas included in other free storage spaces can be split to generate new external storage areas included in the target storage space. In this way, data migration interruption or failure due to a single external storage space being full can be avoided, improving the overall utilization rate of external storage space resources and ensuring the continuity and efficiency of the data migration process.

[0092] Optionally, in this embodiment, if multiple optional storage spaces are ultimately selected after obtaining storage spaces from multiple sets of external storage spaces other than the target storage space where the first storage amount of the external storage region is greater than the second storage amount of the external storage region in the target storage space, a reference storage space may be determined based on the number of external storage regions corresponding to each optional storage space recorded in the first information table. For example, the optional storage space with the largest number of corresponding external storage regions recorded in the first information table may be determined as the reference storage space.

[0093] Optionally, in this embodiment, the region identifier of the first storage region may be deleted from the first information table after the first storage region is split, but is not limited to this.

[0094] As an optional implementation, the target storage area can be divided into the first storage area in the following ways, but not limited to: dividing the first storage area into a storage area with a second storage capacity to obtain the target storage area and a remaining storage area with remaining storage capacity; dividing the remaining storage area into at least one second storage area according to the remaining storage capacity and the storage capacity of the external storage space recorded in the first information table; and recording at least one second storage area in the first information table according to the divided storage capacity.

[0095] Optionally, when specifically splitting the first storage area, it is possible, but not limited to, ensuring that the storage area with the second storage capacity that has an urgent need can be met first. That is, it is possible, but not limited to, first dividing the first storage area into a storage area with the second storage capacity to obtain the target storage area and the remaining storage area.

[0096] Optionally, in this embodiment, the remaining storage area can be further divided into at least one second storage area based on the remaining storage capacity and the storage capacity of the external storage space recorded in the first information table. The storage capacity of each of the at least one second storage area can be, but is not limited to, the storage capacity of the external storage space recorded in the first information table, and the sum of the storage capacities of the at least one second storage area can be, but is not limited to, equal to the storage capacity of the remaining storage area. This partitioning method ensures that the partitioned external storage areas can be used normally in subsequent processes, guaranteeing the orderly progress of data migration.

[0097] Optionally, in this embodiment, after obtaining at least one second storage area, the storage amount and second storage area identifier of the corresponding second storage areas can be recorded in the first information table according to the storage amount of each second storage area.

[0098] As an optional implementation, the data information of the target cache data to be migrated in the internal storage space of the acceleration device can be determined by, but is not limited to, the following methods: determining the target cache data to be migrated in a reference storage area, and the data information of the target cache data, wherein the reference storage area is an internal storage area allocated for the target computing task in the internal storage space.

[0099] Optionally, in this embodiment, a dedicated internal storage area, namely a reference storage area, may be allocated in the internal storage space for the target computing task. Then, the cached data generated during the execution of the target computing task by the acceleration device may be written into the reference storage area in sequence, that is, the cached data is accumulated in the reference storage area.

[0100] Optionally, in this embodiment, determining the data information of the target cache data to be migrated in the internal storage space of the acceleration device may include, but is not limited to, determining the target cache data to be migrated and the data information of the target cache data in the reference storage area.

[0101] Optionally, in this embodiment, by allocating an internal storage area in the internal storage space for the target computing task, it is not only convenient to store cached data in the internal storage space, but also convenient to find the cached data to be migrated when the cached data of the target computing task needs to be migrated, and avoids the storage management difficulties caused by the mixed storage of cached data generated by multiple computing tasks.

[0102] As an optional implementation, the target cache data to be migrated in the reference storage area and the data information of the target cache data can be determined by, but not limited to, the following methods: detecting the numerical relationship between the data volume of the cache data already stored in the reference storage area and the data volume threshold, and the execution progress of the target computing task; when the numerical relationship indicates that the data volume of the stored cache data has reached the data volume threshold, the stored cache data is determined as the target cache data, and data information is determined to indicate that the data volume of the target cache data has reached the data volume threshold; when the execution progress indicates that the target computing task has been completed, the stored cache data is determined as the target cache data, and data information is determined to indicate that the target computing task has been completed and the data volume of the stored cache data.

[0103] Optionally, in this embodiment, when the amount of stored cached data is detected to have reached a data volume threshold, it indicates that the current batch of data has met the migration conditions. This part of the data is identified as the target cached data, and the data information is marked as the data volume has reached the data volume threshold so that the external storage area can match according to the data size. When the target computing task is detected to have been completed, regardless of whether the current data volume has reached the threshold, all the remaining stored cached data in the reference storage area is uniformly identified as the target cached data, and the data information is marked as the target computing task has been completed and the amount of stored cached data, ensuring that the cache is cleared without omission after the task ends.

[0104] By combining the above methods with a dual determination mechanism of data accumulation and task execution progress, the data migration strategy can flexibly adapt to different scenarios such as batch migration and task completion cleanup, thereby improving the accuracy of external storage space matching and the overall efficiency of data migration.

[0105] As an optional implementation, before determining the target cache data to be migrated in the reference storage area and the data information of the target cache data, the following operations may be performed, but are not limited to: receiving the target computing task; allocating a reference storage area in the internal storage space with a storage amount greater than or equal to a data amount threshold for the target computing task; recording the target computing task in a second information table, wherein the second information table records computing tasks with corresponding relationships and migrated storage areas, and the migrated storage areas are storage areas where the cache data of the corresponding computing tasks has been migrated in multiple sets of external storage spaces.

[0106] Optionally, in this embodiment, upon receiving a target computing task, a reference storage area with a storage capacity greater than or equal to a data volume threshold may be allocated in the internal storage space for the target computing task, in preparation for temporarily storing the cached data of the target computing task in the internal storage space; and the target computing task may be recorded in the second information table, in preparation for managing the cached data generated by the acceleration device when executing the target computing task through the second information table.

[0107] Optionally, in this embodiment, the second information table may, but is not limited to, record corresponding computational tasks and migrated storage areas. Specifically, the second information table may, but is not limited to, record the task identifiers of corresponding computational tasks and the area identifiers of the storage areas where the cached data of the corresponding computational tasks has been migrated from multiple sets of external storage spaces.

[0108] As an optional implementation method, such as Figure 3As shown, the aforementioned architecture may include, but is not limited to, cache layer circuitry. This cache layer circuitry enables sequential writing of key-value (KV) data to a buffer within the HBM. Specifically, a dedicated area is allocated within the GPU's high-bandwidth memory to construct a KV sequential write buffer, serving as a high-speed read / write cache for interfacing with the SSD. A separate buffer management unit may be configured for each inference request, responsible for receiving the key-value data corresponding to each token generated by the large language model and storing it in the buffer (i.e., the reference storage area) in sequential order. When the inference task terminates, or when the buffer data volume reaches a set threshold, the management unit automatically transfers the data in the buffer to the SSD for persistent storage.

[0109] Optionally, in this embodiment, the buffer adopts a multi-slot buffering form. The maximum capacity of a single slot is set to 1GB (i.e., the data volume threshold), which matches the capacity of the L5 physical segment in the SSD (i.e., the maximum storage capacity of the external storage space). The slot can be flexibly split for use. For each inference task in a running state, an independent write pointer is configured to mark the current data write position in the buffer.

[0110] Optionally, in this embodiment, the data writing process may, but is not limited to, adopting a token-by-token sequential append writing method. Specific steps may include, but are not limited to: Step a-1. Generating KV vectors: For each token output by the large language model, the corresponding KV vector cache data is synchronously calculated. Step b-1. HBM cache writing: The generated KV data is written sequentially to the HBM buffer corresponding to the current inference task according to the generation order. This process ensures that data in memory is stored continuously and does not generate storage space fragmentation. Step c-1. Capacity threshold verification: The total amount of data stored in the buffer is counted in real time, and two different data write-to-disk logics are executed based on the verification result:

[0111] Trigger Condition 1: Buffer Full for Flush; When the total data volume in the buffer reaches 1GB, or the number of tokens reaches a preset value, a forced disk write operation is immediately executed. The entire 1GB of data is packaged and written to a 1GB physical segment within the SSD in one go using DMA technology. After writing is complete, the buffer write pointer is switched to a new location (the new location can be a new slot or the initial location of the current slot), and a new physical segment of the same specification is requested from the SSD to continue accepting subsequent data.

[0112] Triggering condition two: Inference task completion and flushing; When the inference task terminates normally, regardless of the size of the remaining data in the buffer, a final disk write operation is performed. Based on the actual size of the remaining data, the smallest free physical segment that meets the storage requirements of the actual data size is matched for data writing; the unused remaining space in the physical segment is marked as an internal fill area.

[0113] After completing the data migration for a certain slot, the object mapping table (i.e., the second information table mentioned above) may be updated, but is not limited to.

[0114] Optionally, in this embodiment, the entire lifecycle management of the buffer can be achieved through a dedicated buffer management unit, which records the inference object number (i.e., the task identifier of the computation task), the HBM memory address, the current data storage size, and is configured with a mutex lock to ensure multi-threaded access safety. Optionally, the main functions of the buffer management unit include: Initialization and Release: Upon receiving an inference task (i.e., a computation task), a management unit is created, and 1GB of storage space (i.e., a reference storage area) is automatically allocated in HBM; the unit is destroyed when the inference task ends, and the corresponding HBM memory resources are released synchronously upon unit destruction. Data Append: KV data of a single token is received, locking is used to ensure thread safety, the remaining space is first checked for sufficiency, and if space is insufficient, disk write is performed first, then the data is sequentially copied to the buffer, and the size of the stored data is updated. Task Closure and Disk Write: After inference, if data remains in the buffer, an appropriate SSD physical segment is matched based on the data volume, and then the data is transferred to disk. Data write-to-disk execution: Request the corresponding SSD physical segment from the global index table, write the buffer data to the target physical segment, and then clear the buffer and reset the data statistics size.

[0115] Optionally, in this embodiment, the entire KV data writing process can, but is not limited to, follow the principle of caching first, then persisting, and sequential appending throughout. It leverages a tiered caching architecture composed of HBM and SSD, combined with a tiered disk-to-disk strategy, to improve write efficiency. During inference, scattered KV data is first accumulated and integrated in the HBM buffer (i.e., the reference storage area in the internal storage space). When the data volume reaches the target or the task is completed, it is then written to the SSD (i.e., the external storage space) in large blocks. All data is written sequentially within the SSD physical segment, completely avoiding the write amplification problem caused by fine-grained random writes, and fully utilizing the high bandwidth characteristics of sequential writes of flash media such as SSD, NVMe, and HBF. Simultaneously, matching the corresponding SSD physical segment size according to the actual data volume effectively reduces storage space waste, ultimately achieving dual optimization of storage bandwidth and space utilization.

[0116] As an optional implementation method, such as Figure 3 As shown, the aforementioned architecture may include, but is not limited to, management layer circuitry. Management layer circuitry may be used for key-value (KV) data management based on inference sequence objects.

[0117] Optionally, in this embodiment, an inference sequence object can be constructed based on the storage logic oriented towards inference objects. This simplifies KV data management and shields the complex details of underlying physical segment fragmentation and multi-segment storage. Specifically, a single complete inference sequence can be used as the basic logical unit to associate and integrate multiple SSD physical segments, forming a unified data view. This not only achieves fine-grained and modular management of KV data but also deeply binds the data lifecycle to inference requests, laying the foundation for subsequent garbage collection and storage space reuse.

[0118] Optionally, in this embodiment, each inference request may, but is not limited to, correspond to an independent inference sequence object. Due to limitations in single-segment storage capacity, or to adapt to different data volumes, an inference object may be associated with multiple SSD physical segments of different specifications simultaneously. The combination of all associated physical segments carries all the key-value data of the inference request.

[0119] Optionally, in this embodiment, an inference sequence object structure (structKVCacheObject) (also known as an object mapping table) (i.e., the second information table) may be set, but is not limited to. This is the core unit for managing single inference request KV data, corresponding to a complete inference request KV data set. The inference sequence object structure may include, but is not limited to, the following information: Globally Unique Identifier: A 64-bit integer used as the unique identifier for this inference object (i.e., the task identifier for the inference task). Session Identifier: Records the user and session information corresponding to the inference request, distinguishing different business dialogues. Total Token Count: Counts the total number of tokens generated in the current inference sequence. Total Data Size: Records the total capacity of all KV data under this object, in bytes. Number of Associated Physical Segments: Counts the total number of SSD physical segments bound to the current object. Physical Segment Number List (i.e., migrated storage areas): Records the numbers of all associated physical segments in the order of data writing. Creation Timestamp: Marks the generation time of the inference object. Running Status: A boolean identifier; true indicates that the inference task is running, false indicates that the task has ended and resources can be reclaimed.

[0120] Optionally, in this embodiment, the object may have two built-in operation functions at the same time: one is to add an associated physical segment, which can add the newly assigned physical segment number to the list; the other is to remove a physical segment, which is mainly used in the garbage collection stage to remove the specified physical segment number from the list.

[0121] Optionally, in this embodiment, a global index table (classKVObjectIndexTable) can be designed to uniformly manage the mapping between all inference objects and segments. The global index table can be, but is not limited to, the core control structure of the architecture, running as a global singleton, responsible for maintaining the mapping relationship between inference objects and SSD physical segments, supporting bidirectional fast queries, and can both find all physical segments associated with an inference object and locate the inference object to which it belongs based on the physical segment. Specifically, the global index table can include, but is not limited to, the following internal storage structures: (1) Object mapping table (i.e., the second information table): with the inference object number as the index, it stores the complete information of the corresponding inference object, inference object ID → KVCacheObject, and the data structure is std::map.<uint64_t,KVCacheObject> object_map; (2) Physical segment mapping table: Using the physical segment number as the index, it records all attribute data of each SSD physical segment, Segment ID→KVSegment, and the data structure is std::map<uint64_t,KVSegment> segment_map. (3) Idle segment resource pool (i.e., the first information table): Classified according to physical segment specifications, it stores the physical segment numbers that are in an idle state. (4) Mutex lock: Ensures data safety during concurrent access by multiple threads and avoids read-write conflicts.

[0122] Optionally, in this embodiment, the core functions and operating logic related to the global index table may include, but are not limited to: Creating inference objects: Upon receiving a new inference request, a unique object number is automatically generated, the object's information is initialized and marked as active, and the number is returned after object registration is completed. The entire process is protected by a mutex lock to ensure the security of concurrent creation. Allocating physical segments: According to the specified physical segment specifications, an idle physical segment is retrieved from the corresponding category's idle resource pool; if there are no idle resources, a new physical segment is created and its basic information is configured. After allocation, the physical segment's owner object, usage status, and write time are updated, and the physical segment number is returned. Data appending: When writing data to a specified physical segment, the existence and fullness of the physical segment are first checked. After successful verification, the used capacity of the physical segment is updated, and the total data volume of the corresponding inference object is accumulated synchronously. Marking the task as finished: After the inference request is completed, the running status of the corresponding inference object is changed to a reclaimable state, and the storage space is not immediately released. Garbage collection: For completed inference objects, batch unbind all associated physical segments, reset the physical segment status, return them to the corresponding size's idle resource pool, and finally delete all records of the inference object from the object mapping table. Size matching filtering: Based on the actual size of the data to be written, filter out the smallest size physical segment that can accommodate the data.

[0123] Based on the above, the overall operation flow of the cache data storage method based on the key-value cache pool architecture includes: creating an inference object and completing registration → matching the corresponding specification and allocating physical segments according to the data size → continuously writing data and synchronously updating metadata → marking the object status after the task is completed → batch reclaiming physical segments and reusing storage space when garbage collection is triggered. Thread safety is ensured throughout the process using mutex locks, achieving closed-loop management of the entire lifecycle of inference objects and physical segments.

[0124] Figure 4 This is an overall flowchart of a cache data storage method based on a key-value cache pool architecture according to an embodiment of this application. Figure 4 As shown, the complete inference scenario and the complete KV cache write process, i.e., the overall operation process of the cache data storage method based on the key-value cache pool architecture, may include, but is not limited to, the following steps: Step a-2, call the global index table to create an inference object for the new session and obtain a unique object number; Step b-2, initialize the HBM buffer for the inference object to temporarily store KV data; Step c-2, simulate the inference process, generate 1KB tokens one by one and write them to the buffer. When the data accumulates to 1GB, trigger automatic disk flushing to transfer the data to the SSD; Step d, continue to generate and write 80MB of token data. At this time, the buffer has not reached the threshold; Step e, the inference task terminates, and a final disk flushing operation is performed to write the remaining 80MB of data in the buffer to the SSD physical segment of the appropriate specification; Step f, mark the inference object as the task completed in the index table; Step g, perform garbage collection to release all SSD physical segments associated with the object in batches and return them to the idle resource pool for reuse.

[0125] First, an inference object (e.g., session_12345) is created using a global singleton KVObjectIndexTable to obtain a unique object ID. Then, an HBM Buffer is initialized to temporarily store the KV data generated per token. During inference, 1GB of data is simulated being written (appended per token, 1KB per token). When the HBM Buffer reaches its 1GB threshold, a flush operation is automatically triggered, writing the data to the SSD. Next, 80MB of data is written (before reaching the full threshold). After inference, `flush_on_finish` is called to force flushing the remaining data, ensuring all data is persisted. Finally, the object is marked as terminated and garbage collection is performed, releasing all Segment resources associated with that object.

[0126] Figure 5 This is a flowchart illustrating the write process of a cache data storage method based on a key-value cache pool architecture according to an embodiment of this application. Figure 5As shown, the write process may include, but is not limited to: the application layer generating key-value data for each token and sequentially appending it to the HBMBuffer. After each write, the buffer status is checked; if the 1GB full capacity is reached or the inference request ends, a disk flush is triggered. The most suitable idle segment is selected based on the amount of data to be written, and then the GPU initiates a DMA operation to directly write the data blocks in the HBM to a contiguous physical region of the SSD. After the write is complete, a segment is created or updated, the segment identifier is appended to the object's variable, and the Global Index Table (OIT) is synchronously updated to ensure metadata consistency, before continuing to process the next token.

[0127] As an optional implementation, after migrating the target cache data from the internal storage space to the target storage area among multiple external storage areas included in the target storage space, the cache data of a specific range of words can be read in the following ways, but not limited to: obtaining a target read request from the target application, wherein the target read request is used to request the reading of the cache data of the target words located in the target order among the multiple words corresponding to the target computing task; detecting the target storage location of the cache data of the target words according to a second information table and the target order, wherein the second information table records the migrated storage areas storing the cache data of the multiple words corresponding to the target computing task in the order of their generation; obtaining the cache data of the target words from the target storage location; and sending the cache data of the target words to the target application.

[0128] Optionally, in this embodiment, obtaining the target read request of the target application may refer to, but is not limited to, the acceleration device receiving a specific instruction from the upper-layer application, which explicitly specifies the relative position of the target token to be read in the computation task sequence, i.e., the target order, such as the 50th generated token in a large language model inference task.

[0129] Optionally, in this embodiment, since the size of the cached data corresponding to each word is fixed, and the second information table records the migrated storage area storing the cached data of the word in the order of generation of the cached data of the word corresponding to the target computing task, the target storage location of the cached data of the target word can be detected according to the second information table and the target order.

[0130] As an optional implementation, the target storage location of the cached data of the target lexical can be detected according to the second information table and the target order in the following manner: searching the migrated storage area corresponding to the target computing task in the second information table to obtain the target migrated storage area; searching for the target storage location corresponding to the target order in the target migrated storage area; if no target storage location corresponding to the target order is found in the target migrated storage area, searching for the target storage location in the reference storage area, wherein the reference storage area is the internal storage area allocated for the target computing task in the internal storage space.

[0131] Optionally, in this embodiment, the second information table may, but is not limited to, record the migrated storage areas in the order in which they were used.

[0132] Optionally, in this embodiment, the target storage location may be searched in the external storage area according to the target order first. If the target storage location is not found in the external storage area, the storage location of the cached data of the target word may be searched in the internal storage space.

[0133] Figure 6 This is a flowchart illustrating the reading process of a cache data storage method based on a key-value cache pool architecture according to an embodiment of this application. Figure 6 As shown, the read path process can be, but is not limited to, as follows: When an application requests to read a specific token range of a Sequence, it first queries the Global Index Table (OIT) to obtain the target SequenceObject and its associated list of segments. It then calculates and determines the segment containing the cached data for that specific token range and its offset within that segment. If the data is still residing in the HBM Buffer and has not yet been flushed, it is directly returned via memory copy; if the data has been persisted to the SSD, an SSD read command containing the LBA (Logical Block Address) and length is constructed, and the data is read into the HBM via DMA for GPU computation. To improve read efficiency, Scatter-Gather DMA is supported, allowing a single command to read data scattered across different segments (if hardware supports it); otherwise, multiple reads are initiated in segment order.

[0134] As an optional implementation, after migrating the target cache data from the internal storage space to the target storage area among multiple external storage areas included in the target storage space, the utilized storage area can be reclaimed in the following ways, but not limited to: detecting whether the target computing task has been completed, wherein the target computing task is the computing task that generates the target cache data; if the target computing task has been completed, adjusting the task status of the target computing task to an inactive state in a second information table, wherein the second information table records the task status of each computing task, as well as the corresponding computing tasks and migrated storage areas, wherein the migrated storage area is the storage area where the cache data of the corresponding computing task has been migrated in multiple sets of external storage spaces; detecting whether the reclamation time for the migrated storage area has arrived; if the reclamation time has arrived, reclamating the migrated storage area corresponding to the computing task whose task status is marked as inactive.

[0135] By introducing a dynamic binding mechanism between task lifecycle and storage resource status, and using the clear time node of task completion as the trigger condition, combined with the precise mapping relationship provided by the second information table, it is possible to automatically and efficiently identify and reclaim no longer needed storage resources without manual intervention or full scan. This enables timely reuse of external storage space resources, effectively avoids storage waste caused by long-term resource retention, improves overall resource utilization, and accelerates device operating efficiency.

[0136] Optionally, in this embodiment, in order to avoid unnecessary migration of cached data in the external storage space during the recycling process, the storage amount of the external storage area of ​​the external storage space may be determined as an integer multiple of the minimum amount that the storage medium can erase at one time.

[0137] As an optional implementation, the migrated storage area corresponding to a computing task whose task status is marked as inactive can be reclaimed in the following ways, but not limited to: searching for the computing task whose task status is marked as inactive in a second information table to obtain the reclaimed computing task; searching for the migrated storage area corresponding to the reclaimed computing task in the second information table to obtain the reclaimed storage area; determining the external storage space corresponding to the reclaimed storage area based on the area information of the reclaimed storage area to obtain the reclaimed storage space; storing the reclaimed storage space and reclaimed storage area with corresponding relationship in a first information table, and deleting the reclaimed computing task and reclaimed storage area with corresponding relationship from the second information table, wherein the first information table records the external storage space and unused storage area with corresponding relationship.

[0138] Optionally, in this embodiment, in order to ensure that the reclaimed external storage area can be used normally during the subsequent migration of cached data, it is possible, but not limited to, to determine the external storage space corresponding to each reclaimed storage area, so that the free external storage area can be stored in the first information table according to different external storage spaces.

[0139] Optionally, in this embodiment, deleting the corresponding recycling calculation task and recycling storage area in the second information table may include, but is not limited to, deleting the task identifier of the corresponding recycling calculation task and the area identifier of the recycling storage area in the second information table.

[0140] Optionally, in this embodiment, external storage area reclamation can be, but is not limited to, using the inference object as the smallest unit of operation, rather than a single segment. When the inference request lifecycle ends and no more KV data needs to be retained, the inference object is first marked as reclaimable. After reclamation is triggered, all segments bound to the inference object are released in batches at once and returned to the KV Pool in granularity for reuse by subsequent new inference objects. This approach avoids space fragmentation caused by scattered reclamation, while reducing reclamation complexity and system overhead.

[0141] Figure 7 This is a flowchart illustrating the retrieval process of a cache data storage method based on a key-value cache pool architecture according to an embodiment of this application. Figure 7 As shown, the external storage area reclamation process may include, but is not limited to, the following: When an object corresponding to an inference request completes its inference process and its data no longer needs to be retained, a marking operation will be performed: First, the target inference object is located through the object mapping table in the global index table, and then its object status (i.e., task status) is changed from active (is_active=true) to inactive (is_active=false). This process only marks the status and does not immediately release physical space, thereby avoiding impact on ongoing read and write operations. After marking, the object will enter the candidate pool for reclamation, waiting for subsequent reclamation scheduling. The reclamation process may be automatically initiated by real-time monitoring of four types of indicators (i.e., detecting whether the reclamation time of the migrated storage area has arrived). It will be triggered when any of the following conditions are met: the proportion of free physical blocks on the SSD is less than 20% (space is urgent), the number of expired objects marked for reclamation exceeds 1000 (object backlog), more than 1 hour has passed since the last garbage collection (timed forced reclamation), or the entire architecture system is in a period of low write pressure (proactive reclamation to avoid affecting inference performance).

[0142] Optionally, in this embodiment, the LRU (Least Recently Used) algorithm can be used, but is not limited to, to select the optimal object to be reclaimed (i.e., the reclaimed computation task) to ensure cache efficiency. The specific process may include, but is not limited to: first, scanning the global index table to filter out objects that have finished inference and exceeded their keep-alive time; then, sorting these objects according to their last access time, prioritizing the reclaiming of the objects that have not been accessed for the longest time; simultaneously, this strategy supports business priority protection, with high-priority objects being directly skipped from the reclamation process and not participating in eviction. The process may include, but is not limited to, locking the objects to be reclaimed based on the filtering results and obtaining a list of all segments bound to them; subsequently, entering the batch release of segments stage, marking all segments as unoccupied, clearing their association information with the original objects, and returning them to the corresponding free queue in the KV Pool according to segment granularity (e.g., 1GB / 500MB / 100MB, etc.); next, performing index cleanup, deleting all records of the object from the global index table, updating metadata, and marking the object's metadata on the SSD as overwriteable; finally, all segments enter the reuse pool, and the object records are completely cleared, marking the completion of the reclamation process. Reclaimed segments are not deleted but retained in the KV Pool. When a new inference request needs to allocate storage blocks, these free segments are reused first, rather than directly occupying new physical space on the SSD. This mechanism realizes the recycling of space, effectively reduces storage fragmentation, lowers IO overhead, and improves overall storage efficiency.

[0143] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods according to the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method.

[0144] Embodiments of this application also provide a device for accelerating data migration of devices. Figure 8 This is a structural block diagram of a data migration apparatus for an acceleration device according to an embodiment of this application, such as... Figure 8 As shown, the device includes:

[0145] The determination module 802 is used to determine the data information of the target cache data to be migrated in the internal storage space of the acceleration device. The internal storage space is used to store cache data generated during the execution of computing tasks by the acceleration device. The acceleration device is connected to multiple sets of external storage spaces. The external storage spaces include multiple external storage areas with the same area information, and the external storage areas in different external storage spaces have different area information.

[0146] The lookup module 804 is used to find the target storage space that matches the corresponding area information and data information from multiple sets of external storage spaces;

[0147] Migration module 806 is used to migrate target cache data from internal storage space to target storage area in multiple external storage areas included in the target storage space.

[0148] Through the above-described device, the acceleration device is connected to multiple sets of external storage spaces. These external storage spaces include multiple external storage regions with the same regional information, but different external storage regions have different regional information. The device determines the target cache data to be migrated from its internal storage space. The internal storage space stores cache data generated during the acceleration device's computational tasks. Using the internal storage space as a buffer for cache data reduces I / O interactions with external storage spaces. Then, the device searches for a target storage space whose regional information matches the data information from the multiple sets of external storage spaces. The target cache data is then migrated from the internal storage space to the target storage region within the multiple external storage regions included in the target storage space. Migrating the target cache data to the matching target storage region avoids wasting storage space caused by using unsuitable storage spaces. Therefore, this addresses the technical problem of low data storage efficiency for cache data in acceleration devices in related technologies, achieving the technical effect of improving the data storage efficiency of cache data in acceleration devices.

[0149] In some embodiments, the lookup module includes: a first lookup unit, configured to look up at least one first storage space from multiple sets of external storage spaces where the storage amount of the corresponding external storage region is greater than or equal to the data amount of the target cached data, wherein the region information includes: storage amount, and the data information includes: data amount; and a first determination unit, configured to determine the target storage space from at least one first storage space.

[0150] In some embodiments, the first determining unit is further configured to: determine the first storage space with the smallest corresponding storage amount among at least one first storage space as the target storage space; calculate the difference between the storage amount and the data amount corresponding to each of the at least one first storage spaces; and select the target storage space from the first storage spaces whose corresponding difference is less than the target threshold.

[0151] In some embodiments, the lookup module includes: a second lookup unit, configured to: look up at least one second storage space from a plurality of external storage spaces whose corresponding region information matches the data type of the target cache data, wherein the region information is used to indicate the media attributes of the storage medium of the corresponding external storage region, and the data information includes: data type; and a second determination unit, configured to determine the target storage space from at least one second storage space.

[0152] In some embodiments, the second determining unit is further configured to: search for at least one third storage space from at least one second storage space where the storage amount of the corresponding external storage area is greater than or equal to the data amount of the target cache data; and determine the target storage space from at least one third storage space.

[0153] In some embodiments, the lookup module includes: a third determining unit, configured to determine the external storage space with the largest storage capacity among multiple sets of external storage spaces as the target storage space when the data information indicates that the data volume of the target cached data has reached a data volume threshold, wherein the target cached data is generated by executing the target computing task, and the region information includes: the storage volume of the external storage region, and the maximum storage volume is set according to the data volume threshold; and a fourth determining unit, configured to determine the target storage space from at least one fourth storage space among multiple sets of external storage spaces when the data information indicates that the target computing task has been completed, wherein the data information is further used to indicate the data volume.

[0154] In some embodiments, the migration module includes: a fourth determining unit, configured to determine a target storage region from a plurality of external storage regions included in the target storage space based on storage status information recorded in the acceleration device, wherein the storage status information is used to indicate the usage status of each external storage region;

[0155] The migration unit is used to migrate target cached data from internal storage space to target storage area.

[0156] In some embodiments, the fourth determining unit is further configured to: search for at least one candidate storage region corresponding to the target storage space from the first information table, wherein the storage status information includes the first information table, which records external storage spaces and unused storage regions with corresponding relationships; select the target storage region from the at least one candidate storage region, and delete the target storage region from the first information table.

[0157] In some embodiments, the migration module further includes: a first acquisition unit, configured to, after searching for at least one candidate storage region corresponding to the target storage space from a first information table, and if no candidate storage region is found from the first information table, acquire a reference storage space from other storage spaces besides the target storage space from a plurality of external storage spaces, wherein the first storage amount of the external storage region is greater than the second storage amount of the external storage region in the target storage space; a second acquisition unit, configured to acquire the first storage region from at least one alternative storage region corresponding to the reference storage space recorded in the first information table; and a first processing unit, configured to divide the target storage region from the first storage region and delete the first storage region from the first information table.

[0158] In some embodiments, the first processing unit is further configured to: divide a storage area with a second storage capacity in a first storage area to obtain a target storage area and a remaining storage area with remaining storage capacity; divide the remaining storage area into at least one second storage area according to the remaining storage capacity and the storage capacity of the external storage space recorded in the first information table; and record at least one second storage area in the first information table according to the divided storage capacity.

[0159] In some embodiments, the determining module includes: a fifth determining unit, configured to determine the target cache data to be migrated in the reference storage area, and the data information of the target cache data, wherein the reference storage area is an internal storage area allocated in the internal storage space for the target computing task.

[0160] In some embodiments, the fifth determining unit is further configured to: detect the numerical relationship between the amount of cached data stored in the reference storage area and the data amount threshold, and the execution progress of the target computing task; when the numerical relationship indicates that the amount of cached data stored has reached the data amount threshold, determine the cached data stored as the target cached data, and determine data information to indicate that the amount of cached data stored as the target has reached the data amount threshold; when the execution progress indicates that the target computing task has been completed, determine the cached data stored as the target cached data, and determine data information to indicate that the target computing task has been completed and the amount of cached data stored.

[0161] In some embodiments, the determining module further includes: a receiving unit, configured to receive a target computing task before determining the target cache data to be migrated in the reference storage area and the data information of the target cache data; an allocation unit, configured to allocate a reference storage area in the internal storage space with a storage amount greater than or equal to a data amount threshold for the target computing task; and a recording unit, configured to record the target computing task in a second information table, wherein the second information table records computing tasks with corresponding relationships and migrated storage areas, and the migrated storage areas are storage areas where the cache data of the corresponding computing tasks has been migrated in multiple sets of external storage spaces.

[0162] In some embodiments, the data migration apparatus further includes: a first acquisition module, configured to acquire a target read request of a target application after migrating target cached data from internal storage space to a target storage area among multiple external storage areas included in the target storage space, wherein the target read request is used to request the reading of cached data of target words located in the target order among multiple words corresponding to the target computing task; a first detection module, configured to detect the target storage location of the cached data of the target words according to a second information table and the target order, wherein the second information table records the migrated storage areas storing the cached data of multiple words in the generation order of the cached data of the multiple words corresponding to the target computing task; a second acquisition module, configured to acquire the cached data of the target words from the target storage location; and a sending module, configured to send the cached data of the target words to the target application.

[0163] In some embodiments, the first detection module includes: a third search unit, configured to search for the migrated storage area corresponding to the target computing task in the second information table to obtain the target migrated storage area; a fourth search unit, configured to search for the target storage location corresponding to the target order in the target migrated storage area; and a fifth search unit, configured to search for the target storage location in a reference storage area if no target storage location corresponding to the target order is found in the target migrated storage area, wherein the reference storage area is an internal storage area allocated for the target computing task in the internal storage space.

[0164] In some embodiments, the data migration apparatus further includes: a second detection module, configured to detect whether a target computing task has been completed after migrating target cache data from internal storage space to a target storage area in multiple external storage areas included in the target storage space, wherein the target computing task is a computing task that generates the target cache data; an adjustment module, configured to adjust the task status of the target computing task to an inactive state in a second information table when the target computing task has been completed, wherein the second information table records the task status of each computing task, as well as the corresponding computing tasks and migrated storage areas, wherein the migrated storage area is a storage area in multiple sets of external storage spaces where the cache data of the corresponding computing task has been migrated; a third detection module, configured to detect whether the recycling time for the migrated storage area has arrived; and a recycling module, configured to recycle the migrated storage area corresponding to the computing task whose task status is marked as inactive when the recycling time arrives.

[0165] In some embodiments, the recycling module includes: a sixth lookup unit, configured to look up computing tasks whose corresponding task status is marked as inactive in a second information table, and obtain the recycled computing tasks; a seventh lookup unit, configured to look up the migrated storage area corresponding to the recycled computing tasks in the second information table, and obtain the recycled storage area; a sixth determination unit, configured to determine the external storage space corresponding to the recycled storage area based on the area information of the recycled storage area, and obtain the recycled storage space; and a second processing unit, configured to store the corresponding recycled storage space and recycled storage area in a first information table, and delete the corresponding recycled computing tasks and recycled storage areas from the second information table, wherein the first information table records the corresponding external storage space and unused storage area.

[0166] For a description of the features in the embodiment corresponding to the data migration device of the acceleration device, please refer to the relevant description of the embodiment corresponding to the data migration method of the acceleration device, which will not be repeated here.

[0167] Embodiments of this application also provide an electronic device, including a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to perform the steps in any of the above-described embodiments of the data migration method for accelerating devices.

[0168] Embodiments of this application also provide a computer-readable storage medium storing a computer program, wherein the computer program is configured to execute the steps in any of the above-described embodiments of the data migration method for accelerating devices.

[0169] In one exemplary embodiment, the aforementioned computer-readable storage medium may include, but is not limited to, various media capable of storing computer programs, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard disk, magnetic disk, or optical disk.

[0170] Embodiments of this application also provide a computer program product, which includes a computer program that, when executed by a processor, implements the steps in the data migration method embodiments of any of the above-described acceleration devices.

[0171] Embodiments of this application also provide another computer program product, including a non-volatile computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps in any of the above-described embodiments of the data migration method for acceleration devices.

[0172] Any of the components, modules, units, parts, methods, and operations described herein can be implemented using software, firmware, hardware (e.g., fixed logic circuitry), manual processing, or any combination thereof. Alternatively or additionally, any functionality described herein can be executed at least in part by one or more hardware logic components, such as, but not limited to, a central processing unit (CPU), a field-programmable gate array (FPGA), an application-specific integrated circuit (ASIC), an application-specific standard product (ASSP), a system-on-a-chip (SoC), a complex programmable logic device (CPLD), a microprocessor (MCU), etc. The terms "system," "computing device," or "apparatus" as used herein encompass various means, devices, and machines for processing data, including, for example, one or more programmable processors, computers, SoCs, or combinations thereof. The apparatus may also include code that creates an execution environment for the computer program in question, such as code constituting processor firmware, a protocol stack, a database management system, an operating system, a cross-platform runtime environment, a virtual machine, or one or more combinations thereof. The aforementioned computer program (also known as a program, software, software application, app, script, or code) can be written in any form of programming language, including compiled or interpreted languages, declarative or procedural languages, and can be deployed in any form, including as a standalone program or as a module, component, subroutine, object, or other unit suitable for a computing environment.

[0173] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0174] The foregoing has provided a detailed description of the data migration method, electronic device, medium, and program product for accelerating equipment provided in this application. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the embodiments above are merely for the purpose of helping to understand the method and core ideas of this application. It should be noted that those skilled in the art can make various improvements and modifications to this application without departing from its principles, and these improvements and modifications also fall within the protection scope of the claims of this application.

Claims

1. A method for accelerating data migration in devices, characterized in that, include: The data information of the target cached data to be migrated in the internal storage space of the acceleration device is determined. The internal storage space is used to store cached data generated during the execution of computing tasks by the acceleration device. The acceleration device is connected to multiple sets of external storage spaces. The external storage spaces include: multiple external storage areas with the same area information, and external storage areas in different external storage spaces have different area information. Search for the target storage space that matches the corresponding region information and the data information from the multiple sets of external storage spaces; The target cache data is migrated from the internal storage space to the target storage area among the multiple external storage areas included in the target storage space.

2. The data migration method according to claim 1, characterized in that, The step of searching for the target storage space from the multiple sets of external storage spaces that matches the corresponding region information and the data information includes: From the plurality of external storage spaces, at least one first storage space is found whose corresponding external storage region has a storage amount greater than or equal to the data amount of the target cache data, wherein the region information includes: the storage amount, and the data information includes: the data amount; The target storage space is determined from the at least one first storage space.

3. The data migration method according to claim 2, characterized in that, Determining the target storage space from the at least one first storage space includes one of the following: The first storage space with the smallest storage capacity among the at least one first storage space is determined as the target storage space; Calculate the difference between the storage amount and the data amount corresponding to each of the at least one first storage spaces; The target storage space is selected from the first storage space whose difference is less than the target threshold.

4. The data migration method according to claim 1, characterized in that, The step of searching for the target storage space from the multiple sets of external storage spaces that matches the corresponding region information and the data information includes: From the plurality of external storage spaces, at least one second storage space whose corresponding region information matches the data type of the target cache data is searched, wherein the region information is used to indicate the media attributes of the storage medium of the corresponding external storage region, and the data information includes: the data type; The target storage space is determined from the at least one second storage space.

5. The data migration method according to claim 4, characterized in that, Determining the target storage space from the at least one second storage space includes: Find at least one third storage space from the at least one second storage space whose corresponding external storage area has a storage size greater than or equal to the target cache data size; The target storage space is determined from the at least one third storage space.

6. The data migration method according to claim 1, characterized in that, The step of searching for the target storage space from the multiple sets of external storage spaces that matches the corresponding region information and the data information includes: When the data information indicates that the amount of the target cached data has reached a data amount threshold, the external storage space with the largest storage capacity among the multiple groups of external storage spaces is determined as the target storage space. The target cached data is generated by executing the target computing task. The area information includes the storage capacity of the external storage area, and the maximum storage capacity is set according to the data amount threshold. When the data information is used to indicate that the target computing task has been completed, the target storage space is determined from at least one fourth storage space in the plurality of external storage spaces whose storage volume is greater than or equal to the data volume of the target cached data, wherein the data information is also used to indicate the data volume.

7. The data migration method according to claim 1, characterized in that, The process of migrating the target cache data from the internal storage space to the target storage area, which is one of the multiple external storage areas included in the target storage space, includes: The target storage area is determined from multiple external storage areas included in the target storage space based on the storage status information recorded in the acceleration device, wherein the storage status information is used to indicate the usage status of each external storage area; The target cached data is migrated from the internal storage space to the target storage area.

8. The data migration method according to claim 7, characterized in that, The step of determining the target storage region from multiple external storage regions included in the target storage space based on the storage state information recorded in the acceleration device includes: The first information table is used to find at least one candidate storage area corresponding to the target storage space, wherein the storage status information includes the first information table, which records external storage spaces and unused storage areas with corresponding relationships; Select the target storage region from the at least one candidate storage region and delete the target storage region from the first information table.

9. The data migration method according to claim 8, characterized in that, After searching the first information table for at least one candidate storage region corresponding to the target storage space, the method further includes: If no candidate storage region is found in the first information table, a reference storage space is obtained from the other storage spaces in the multiple sets of external storage spaces other than the target storage space, where the first storage amount of the external storage region is greater than the second storage amount of the external storage region in the target storage space. Obtain the first storage area from at least one alternative storage area corresponding to the reference storage space recorded in the first information table; The target storage area is divided in the first storage area, and the first storage area is deleted from the first information table.

10. The data migration method according to claim 9, characterized in that, The step of dividing the target storage region in the first storage region includes: A storage area with the second storage capacity is divided in the first storage area to obtain the target storage area and the remaining storage area with the remaining storage capacity. The remaining storage area is divided into at least one second storage area based on the remaining storage capacity and the storage capacity of the external storage space recorded in the first information table; The first information table records at least one second storage area according to the allocated storage volume.

11. The data migration method according to claim 1, characterized in that, The data information for determining the target cache data to be migrated in the internal storage space of the acceleration device includes: The target cache data to be migrated in the reference storage area and the data information of the target cache data are determined, wherein the reference storage area is an internal storage area allocated for the target computing task in the internal storage space.

12. The data migration method according to claim 11, characterized in that, The determination of the target cache data to be migrated in the reference storage area, and the data information of the target cache data, includes: The numerical relationship between the amount of cached data stored in the reference storage area and the data amount threshold is detected, as well as the execution progress of the target computing task; When the numerical relationship indicates that the amount of stored cached data has reached the data amount threshold, the stored cached data is determined as the target cached data, and the data information used to indicate that the amount of target cached data has reached the data amount threshold is determined. When the execution progress indicates that the target computing task has been completed, the stored cached data is identified as the target cached data, and the data information used to indicate that the target computing task has been completed and the amount of stored cached data are determined.

13. The data migration method according to claim 12, characterized in that, Before determining the target cache data to be migrated in the reference storage area, and the data information of the target cache data, the method further includes: Receive the target computation task; In the internal storage space, a reference storage area with a storage amount greater than or equal to the data amount threshold is allocated for the target computing task. The target computing task is recorded in the second information table, wherein the second information table records computing tasks with corresponding relationships and migrated storage areas, and the migrated storage areas are storage areas where cached data of the corresponding computing tasks have been migrated from the multiple sets of external storage spaces.

14. The data migration method according to claim 13, characterized in that, After migrating the target cache data from the internal storage space to a target storage area within the plurality of external storage areas included in the target storage space, the method further includes: Obtain the target read request of the target application, wherein the target read request is used to request to read the cached data of the target word in the target order among the multiple words corresponding to the target computing task; The target storage location of the cached data of the target word is detected according to the second information table and the target order. The second information table records the migrated storage area that stores the cached data of the multiple words in the order of generation of the cached data of the multiple words corresponding to the target computing task. Retrieve cached data of the target word from the target storage location; Send the cached data of the target lexical to the target application.

15. The data migration method according to claim 14, characterized in that, The step of detecting the target storage location of the cached data of the target word based on the second information table and the target order includes: The target migrated storage region is obtained by searching the second information table for the migrated storage region corresponding to the target computing task. Locate the target storage location corresponding to the target order in the target migrated storage area; If the target storage location corresponding to the target order is not found in the target migrated storage area, the target storage location is searched in the reference storage area, wherein the reference storage area is the internal storage area allocated for the target computing task in the internal storage space.

16. The data migration method according to claim 1, characterized in that, After migrating the target cache data from the internal storage space to a target storage area within the plurality of external storage areas included in the target storage space, the method further includes: Detect whether the target computing task has been completed, wherein the target computing task is the computing task that generates the target cached data; When the target computing task has been completed, the task status of the target computing task is adjusted to an inactive state in the second information table. The second information table records the task status of each computing task, as well as the corresponding computing tasks and migrated storage areas. The migrated storage areas are storage areas where the cached data of the corresponding computing tasks has been migrated from the multiple sets of external storage spaces. Detect whether the time for recycling the migrated storage area has arrived; When the recycling time arrives, the task state corresponding to the recycling is marked as the migrated storage area corresponding to the computing task in the inactive state.

17. The data migration method according to claim 16, characterized in that, The migrated storage area corresponding to the task state marked as inactive during the recycling includes: Search the second information table for the computing tasks whose corresponding task status is marked as inactive, and obtain the reclaimed computing tasks; The migrated storage area corresponding to the recycling calculation task is found in the second information table to obtain the recycling storage area; Based on the area information of the reclaimed storage area, the external storage space corresponding to the reclaimed storage area is determined, and the reclaimed storage space is obtained; The corresponding recycling storage space and the corresponding recycling storage area are stored in a first information table, and the corresponding recycling calculation task and the corresponding recycling storage area are deleted from the second information table. The first information table records the corresponding external storage space and unused storage area.

18. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor, configured to implement the steps of the data migration method of the acceleration device as described in any one of claims 1 to 17 when executing the computer program.

19. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, wherein the computer program, when executed by a processor, implements the steps of the data migration method of the acceleration device as described in any one of claims 1 to 17.

20. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the steps of the data migration method for the acceleration device as described in any one of claims 1 to 17.

Images