Distributed storage method and device, equipment and medium

Abstract

The invention relates to the technical field of distributed storage, can be applied to business system platforms of financial science and technology, medical treatment and health and the like, and discloses a distributed storage method, device, equipment and medium. Judging whether a reusable KV cache fragment exists in a preset shared memory pool or not according to a preset consistent hash index table, if yes, outputting a hash position of the reusable KV cache fragment in the shared memory pool, obtaining a KV cache from the shared memory pool according to the hash position, and storing the KV cache in the shared memory pool according to the KV cache. Otherwise, creating a KV cache according to the structured request parameter, confirming a data fragmentation strategy according to the KV cache and system load data acquired in real time, selecting a data transmission path according to the data fragmentation strategy, performing task scheduling according to the data transmission path, generating a task data block, and writing the task data block into a preset distributed cluster. According to the invention, the efficiency of distributed storage is improved.

Description

Technical Field

[0001] This invention relates to the field of distributed storage technology, and in particular to a distributed storage method, apparatus, device, and medium. Background Technology

[0002] In the healthcare field, the current storage architecture of medical AI systems faces a fundamental bottleneck in multimodal data collaborative processing. During the training phase, when processing massive amounts of patient images, genomes, and medical records, the static sharding strategy of traditional distributed storage leads to severe imbalances in node load. Hot nodes experience IOPS overload while cold nodes remain idle, resulting in low writing efficiency for petabyte-level intermediate features and model checkpoints, significantly delaying research and model iteration cycles. During the clinical inference phase, especially in emergency multimodal collaborative diagnosis, the system needs to process CT scans, medical records, and other data in parallel. Existing solutions independently allocate complete intermediate model states (such as feature vectors) for each concurrent request, causing severe redundancy in GPU memory and limiting the number of critical cases that a single GPU can process simultaneously. Furthermore, "data silos" exist across institutions or research projects, where the same medical prior knowledge (such as standard organ characteristics) is repeatedly calculated and stored. The storage system is completely unaware of the 3D convolution or sequential access patterns of the medical model and cannot reserve computing and I / O resources for critical emergency paths. This results in the inability to guarantee the determinism and real-time nature of the diagnostic process in emergency situations, thus hindering the development of smart healthcare systems towards large-scale, real-time, and collaborative capabilities.

[0003] In the fintech sector, existing fintech infrastructure faces severe performance and cost challenges in supporting critical businesses such as high-frequency trading and real-time risk measurement. In training scenarios, complex quantitative models used for market simulation and pricing require processing terabytes of historical market data, generating numerous intermediate checkpoints. Traditional static storage solutions with multiple replicas are costly and cannot flexibly handle instantaneous load spikes caused by market fluctuations, leading to unstable training throughput. In inference scenarios, millisecond-level trading and real-time risk control systems need to perform parallel pricing calculations on tens of thousands of derivative positions. Existing architectures force each concurrent request to completely store an independent intermediate computational state in GPU memory, quickly exhausting expensive high-bandwidth GPU memory and severely limiting the portfolio size and complexity the system can handle simultaneously under extreme market conditions. Furthermore, the storage system is completely decoupled from the computing engine, making it unable to perceive the frequent access patterns of financial models to hot data such as real-time volatility surfaces. It fails to achieve intelligent local placement of data and cannot provide a guaranteed low-latency channel for critical risk control data flow at the network layer. This makes cross-domain data synchronization a performance bottleneck and cannot meet the stringent requirements of financial business for extreme performance, high concurrency, and stability. Therefore, a high-performance distributed storage method is needed. Summary of the Invention

[0004] This invention provides a distributed storage method, apparatus, computer equipment, and medium to solve the problem of low efficiency in existing distributed storage methods on the market.

[0005] Firstly, a distributed storage method is provided, including: The pre-acquired user requests are parsed to obtain structured request parameters; The system determines whether the structured request parameters have a reusable KV cache fragment in the preset shared memory pool based on the preset consistent hash index table. If it exists, output the hash position of the reusable KV cache fragment in the shared memory pool, and retrieve the KV cache from the shared memory pool according to the hash position; If it does not exist, a KV cache is created based on the structured request parameters; The data sharding strategy is confirmed based on the KV cache and the real-time acquired system load data. Select the data transmission path according to the data fragmentation strategy; Task scheduling is performed according to the data transmission path to generate task data blocks; Write the task data block into a preset distributed cluster.

[0006] Secondly, a distributed storage device is provided, comprising: The request parsing module is used to parse pre-acquired user requests to obtain structured request parameters; The cache determination module is used to determine whether there is a reusable KV cache fragment in the preset shared memory pool based on the preset consistent hash index table. If it exists, the hash position of the reusable KV cache fragment in the shared memory pool is output, and the KV cache is obtained from the shared memory pool according to the hash position. If it does not exist, the KV cache is created according to the structured request parameters. The path selection module is used to confirm the data sharding strategy based on the KV cache and the real-time acquired system load data, and select the data transmission path according to the data sharding strategy. The task scheduling module is used to schedule tasks according to the data transmission path and generate task data blocks; The data writing module is used to write the task data block into a preset distributed cluster.

[0007] Thirdly, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the above-described distributed storage method.

[0008] Fourthly, a computer-readable storage medium is provided, which stores a computer program that, when executed by a processor, implements the steps of the above-described distributed storage method.

[0009] In the above-described distributed storage method, apparatus, computer equipment, and storage medium, structured request parameters are obtained by parsing pre-acquired user requests. A preset consistent hash index table is used to determine whether reusable key-value (KV) cache fragments exist in a preset shared memory pool. If they exist, the hash position of the reusable KV cache fragment in the shared memory pool is output, and the KV cache is retrieved from the shared memory pool based on the hash position. Otherwise, a KV cache is created based on the structured request parameters. A data sharding strategy is confirmed based on the KV cache and real-time acquired system load data. A data transmission path is selected based on the data sharding strategy, and task scheduling is performed based on the data transmission path to generate task data blocks. The task data blocks are then written to a preset distributed cluster. This improves the efficiency of distributed storage. Attached Figure Description

[0010] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the description of the embodiments of the present invention will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0011] Figure 1 This is a schematic diagram of an application environment for a distributed storage method according to an embodiment of the present invention; Figure 2 This is a flowchart illustrating a distributed storage method according to an embodiment of the present invention; Figure 3 This is a schematic diagram of a distributed storage device according to an embodiment of the present invention; Figure 4 This is a schematic diagram of the structure of a computer device according to an embodiment of the present invention; Figure 5 This is another structural schematic diagram of a computer device according to one embodiment of the present invention. Detailed Implementation

[0012] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0013] The distributed storage method provided in this invention can be applied to, for example... Figure 1 In this application environment, the client communicates with the server via a network. The server can extract core fields from user requests and construct a basic information structure based on the client's request. Based on this, it determines the request type and extracts the corresponding model slicing strategy, tensor description, or input sequence parameters. Combining this with the model registry's associated storage strategy, it ultimately calculates key derived parameters, including cache size and hash keys, and encapsulates them into unified structured request parameters. Based on this, the system quickly determines whether reusable KV cache slices exist in the shared memory pool using consistent hash indexing: extracting elements from structured parameters to generate unique cache keys, locating the target node of the hash ring, and querying metadata; if they exist, the hash position is parsed, and the cache is directly retrieved from the memory pool via RDMA connection; if they do not exist, the cache creation process is triggered: dynamically allocating computing resources, loading model weights, executing Transformer forward computation to generate a KV cache tensor set, and formatting the output. Subsequently, the system integrates KV cache information with real-time load data, dynamically formulating data slicing strategies with the help of reinforcement learning agents, clarifying the slicing granularity, storage level, and compression method. According to this strategy, the system further parses and generates a transmission task list, plans the optimal RDMA transmission path based on network topology and real-time status, reserves network resources, and generates a scheduling instruction set. Based on the instruction set, the system schedules tasks to generate task data blocks and writes them to a distributed cluster in parallel to achieve efficient and reliable distributed storage management. The client can be, but is not limited to, various personal computers, laptops, smartphones, tablets, and portable wearable devices. The server can be implemented using a standalone server or a server cluster consisting of multiple servers. The invention will be described in detail below through specific embodiments.

[0014] Please see Figure 2 As shown, Figure 2 A flowchart illustrating a distributed storage method provided in an embodiment of the present invention includes the following steps: S1. Parse the pre-acquired user request to obtain structured request parameters.

[0015] In the healthcare field, user requests typically manifest as a "multimodal patient data analysis task." The structured request parameters include: a unique patient identifier, the medical image sequence to be processed, genome sequencing data fragments, relevant electronic medical record text fragments, and a request type identifier (e.g., "emergency diagnosis" or "scientific research analysis"). This request triggers a medical AI model (such as an image segmentation model or pathology prediction model) to load the corresponding parameters and process this multimodal data. The "KV cache" generated or reused during this process may be intermediate tensors extracted by the model for specific image features or encoding results of general pathological patterns. The system, through dynamic resource awareness technology, prioritizes the data stream of emergency requests to be scheduled to high-speed storage channels and allows different diagnostic tasks to share the same patient's baseline image feature cache, thereby accelerating collaborative diagnosis and scientific research analysis.

[0016] In the fintech field, user requests typically manifest as a "high-frequency real-time risk calculation or trading decision task." Its structured request parameters include: trading account ID, thousands of market risk factor parameters to be simulated (such as volatility surface data), a snapshot identifier of the current portfolio, and millisecond-level SLO requirements. This request drives a pricing model or risk engine to perform large-scale Monte Carlo simulations. The "KV cache" generated or reused during this process may be intermediate results of parts of the computational path of a general pricing model (such as price surfaces for specific option structures) or the encoded state of real-time market data streams. The system utilizes a topology-aware RDMA (Remote Direct Memory Access) path and a distributed shared memory pool to ensure the aggregation of real-time market data on distributed nodes with extremely low latency. It also allows parallel computing tasks to reuse the same market state cache, thereby meeting the extreme performance and consistency requirements of high-frequency trading and real-time risk control.

[0017] In this embodiment of the invention, parsing the pre-acquired user request to obtain structured request parameters includes: Extract the core fields from the user request; Construct a basic request information structure based on the core fields; Based on the basic request information structure, determine whether the user request is a training request; If so, extract the model slicing strategy and parameter tensor description from the user request to form a task data block; If not, the user request is determined to be a reasoning request, and the input sequence Token ID, generation parameters and sequence unique identifier in the user request are extracted to form a task data block; Based on the basic request information structure and the request parameter structure, the associated model structure and storage strategy configuration are used to obtain the model storage configuration structure; Calculate key derived parameters based on the model storage configuration structure, the basic request information structure, and the request parameter structure; The key derived parameters are encapsulated into a unified structured object to obtain structured request parameters.

[0018] In detail, extracting the core fields from the user request involves extracting basic metadata such as request identifier, timestamp, operation type, model name, and sequence length from the user request.

[0019] In detail, constructing the basic request information structure based on the core fields involves mapping the extracted core fields to the structure members by name. Simultaneously, system default values ​​are added, such as setting the default priority to "medium," and a globally unique internal request identifier is generated.

[0020] In detail, the step of determining whether the user request is a training request based on the basic request information structure is to determine whether it is a training request based on the request identifier in the basic request information structure.

[0021] In detail, the step of obtaining the model storage configuration structure by associating the model structure and storage strategy configuration based on the basic request information structure and the request parameter structure involves querying the model registry using the basic request information structure as the key to obtain predefined model architecture configurations (including the number of layers, number of attention heads, hidden dimensions, and KVCache format preferences). Combined with the sequence length or parallel strategy in the request parameters, the corresponding storage configuration template (such as compression threshold and storage level mapping) is obtained from the strategy engine to generate the model storage configuration structure.

[0022] In detail, the calculation of key derived parameters based on the model storage configuration structure, the basic request information structure, and the request parameter structure involves generating key derived parameters through a preset calculation module based on the model layer number, attention head number, and hidden dimension in the model storage configuration structure, combined with the sequence length and request type in the basic request information structure, and the batch size and data precision in the request parameter structure. These parameters include the predicted cache size, cache reuse hash key, data priority label, suggested storage level, and compression decision threshold.

[0023] S2. Determine whether the structured request parameters have reusable KV cache fragments in the preset shared memory pool based on the preset consistent hash index table.

[0024] In this embodiment of the invention, determining whether a reusable key-value cache fragment exists in a preset shared memory pool based on a preset consistent hash index table includes: Extract the elements required for cache key generation from the structured request parameters to generate a cache key generation element set; Generate a unique cache key based on the cache key and the element set generated; The target node in the preset consistent hash ring is located based on the unique cache key, and the target storage node identifier is obtained; Based on the target storage node identifier, perform a cache metadata query in the consistent hash index table to obtain the cache metadata query result; If the cache metadata query result indicates the existence of corresponding cache metadata, then it is determined that there is a reusable KV cache fragment in the preset shared memory pool. If the cache metadata query result indicates that there is no corresponding cache metadata, then it is determined that there is no reusable KV cache shard in the preset shared memory pool.

[0025] In detail, the step of extracting the elements required for generating cache keys from the structured request parameters and generating a cache key generation element set involves extracting the key elements necessary for generating cache lookup keys from the structured request parameters, including core information such as model identifier, input sequence feature values, sequence length, and model version.

[0026] In detail, the step of generating a unique cache key based on the cache key generating feature set is to generate a globally unique cache identifier key by extracting the cache key generating feature set and using a specific hash algorithm (such as SHA-256).

[0027] In detail, the step of locating the target node in the preset consistent hash ring based on the unique cache key and obtaining the target storage node identifier involves mapping the unique cache key to the preset consistent hash ring, determining the target physical node responsible for storing the cached data corresponding to the key through a clockwise search algorithm on the ring, and obtaining the network address or logical identifier of the node.

[0028] In detail, the step of querying cache metadata in the consistent hash index table based on the target storage node identifier to obtain the cache metadata query result involves using the target storage node identifier to initiate a cache metadata query request to the corresponding node, searching through the distributed index table to see if there is a cache record that matches the unique cache key, and returning query results including information such as cache existence, storage location, size, and status.

[0029] In this embodiment of the invention, the KV cache is intermediate state data stored in real time by a large language model during text generation (inference) to accelerate self-attention calculation. Essentially, it is a collection of key and value matrices calculated and saved by the model for each generated word in the current input sequence when processing the current input sequence. By caching these historical calculation results, the model does not need to repeatedly calculate the attention of the entire sequence when generating the next word, thereby reducing the computational complexity from quadratic to linear and achieving efficient inference for long text generation. However, due to its large size and the need to store it in high-speed video memory, it has become a key resource bottleneck restricting inference concurrency and system cost. This is the core object of optimization and management of the KV cache through technologies such as sharing, compression, and hierarchical storage in this patent solution.

[0030] If it exists, then execute S3, find the hash position of the reusable KV cache fragment in the shared memory pool, and retrieve the KV cache from the shared memory pool according to the hash position.

[0031] In this embodiment of the invention, obtaining the KV cache from the shared memory pool based on the hash position includes: The hash position is parsed to obtain the parsed position; Extract the target node identifier contained in the parsed position, establish a shared memory pool connection based on the target node identifier, and construct a cache key generation element set; Construct an access request structure based on the network connection handle; The KV cache is read based on the network connection handle and the access request structure.

[0032] In detail, parsing the hash position to obtain the parsed position involves deserializing the hash position according to a predefined format (such as JSON, Protocol Buffers, or a custom binary format) to extract structured fields, which mainly include the network identifier of the target storage node (such as IP:Port), the precise offset address of the KV cache data block in the node's memory pool, the data block length, and metadata used for subsequent integrity verification.

[0033] In detail, the process of extracting the target node identifier contained in the parsed location, establishing a shared memory pool connection based on the target node identifier, and constructing a cache key generation element set involves checking whether an available persistent connection (such as an RDMA QP connection) to the target node exists based on the target node identifier. If an existing connection exists and is healthy, it is directly reused and its handle is returned. If a new connection needs to be established, the underlying network library is invoked to initiate the connection establishment process, which includes exchanging transmission parameters, establishing queue pairs, and registering memory windows. The established connection handle encapsulates all necessary network context, such as the QP number, access key, and port information, for use in subsequent operations.

[0034] In detail, the step of constructing the access request structure based on the network connection handle involves using the transmission context contained in the network connection handle, combined with the memory offset address and data length obtained from the parsed location, to construct a specific Remote Direct Memory Access (RDMA) read request structure.

[0035] In detail, the step of reading the KV cache based on the network connection handle and the access request structure involves submitting the constructed access request structure to the send queue of the local RDMA device via the network connection handle. The pre-defined smart network interface card (NIC) directly parses the request without remote CPU intervention and initiates a read operation to the target node's memory via the data center network. Data is directly transferred from the address specified in the remote shared memory pool to the local receive buffer. The local driver confirms the completion of the RDMA read operation through polling or completion queue notification. Subsequently, the system verifies whether the number of bytes transferred matches the request and may selectively use the metadata obtained during location parsing to perform a fast integrity check on the data in the buffer. Finally, the data located in the local buffer is the original content of the acquired KV cache.

[0036] If it does not exist, then execute S4 to create a KV cache based on the structured request parameters.

[0037] In this embodiment of the invention, creating a KV cache based on the structured request parameters includes: Resource allocation is performed based on the structured request parameters to obtain the resource allocation result; Extract the model identifier contained in the structured request parameters, and load the preset large speech model's required model calculation context data according to the weight path corresponding to the model identifier and the resource allocation result; The context data is sequence-embedded and position-encoded based on the input sequence Token ID in the structured request parameters to generate an embedded sequence representation; The large speech model is used to perform forward computation of the embedded sequence representation and the resource allocation result using the Transformer layer, and outputs a set of KV buffer tensors. The KV cache tensor set is formatted to obtain the KV cache.

[0038] In detail, resource allocation is performed based on the structured request parameters to obtain the resource allocation result. This is achieved by parsing the computational requirement specifications in the structured request parameters, including the peak memory usage for a single inference run corresponding to the model identifier, the batch size (and the sequence length). Then, a resource request is sent to a preset global resource scheduler. This scheduler dynamically allocates the optimal combination of computing devices (e.g., specifying four A100 GPUs interconnected via NVSwitch) based on the real-time cluster status (GPU memory fragmentation, NVLink topology, node load) and reserves contiguous memory blocks of the appropriate size. Simultaneously, storage space is pre-registered and allocated for the upcoming KV cache in the shared memory pool, and its virtual address is obtained, ultimately yielding the resource allocation result.

[0039] In detail, the step of extracting the model identifier contained in the structured request parameters and loading the model computation context data required by the preset large speech model according to the weight path corresponding to the model identifier and the resource allocation result involves querying a preset central model registry using the model identifier as the key to obtain the distributed storage path of the corresponding model configuration file and the sharded weight file. Based on the number of GPUs and the tensor parallelism strategy specified in the resource allocation result, multiple data loading tasks are initiated in parallel using the scheduler: each task loads the weight shards required by the corresponding GPU directly from distributed object storage to the allocated video memory via a high-speed network. Simultaneously, the computation context data of the model on each GPU (such as a PyTorch DistributedModel instance or a TensorRT inference engine) is initialized, process group communication is established, and finally, a parallel-executable distributed model computation context data holding weight parameters is assembled.

[0040] In detail, the step of performing sequence embedding and positional encoding on the context data based on the input sequence Token ID in the structured request parameters to generate an embedded sequence representation involves feeding the input Token ID sequence into the embedding layer of the large speech model. This embedding layer, as part of the loaded model context, has its weight matrix loaded onto the GPU via a query. The system performs matrix multiplication, mapping each Token ID to a corresponding high-dimensional vector (e.g., for a model with a hidden dimension of 8192, each Token is converted into an 8192-dimensional FP16 vector). Subsequently, according to the model configuration, the system calls the corresponding positional encoding kernel function to inject positional information into the vector at each position in the sequence, ultimately generating the embedded sequence representation.

[0041] In detail, the process of performing Transformer layer forward computation on the embedded sequence representation and the resource allocation result using the large speech model, and outputting a KV cache tensor set, involves inputting the embedded sequence representation into the model computation context data to initiate the forward propagation process of multiple Transformers. In the self-attention computation of each layer, the Key (K) and Value (V) projection matrices of that layer are calculated and stored. This computation is performed in parallel on the allocated GPU by a highly optimized operator library. For models employing tensor parallelism, the computation of the K and V matrices is also synchronized and aggregated across GPUs. The system collects the K and V matrices output from all layers in layer order, forming a temporary tensor set, i.e., the KV cache tensor set.

[0042] In detail, the process of formatting the KV cache tensor set to obtain the KV cache involves standardizing and encapsulating the KV cache tensor set. Specifically, tensors distributed across different GPUs and potentially having different views are logically reorganized through set communication and uniformly converted into a preset global data format. Then, a descriptive header structure is added, containing the format version, model identifier, creation timestamp, tensor dimension layout, and data checksum, ultimately resulting in the KV cache.

[0043] S5. Confirm the data sharding strategy based on the KV cache and the real-time acquired system load data.

[0044] In this embodiment of the invention, the process of determining the data sharding strategy based on the KV cache and real-time acquired system load data involves fusing the structured information of the KV cache (including cache size, data type, access priority, and associated model layer identifier) ​​with real-time acquired multi-dimensional system load data (covering the current IOPS of each storage node, remaining memory / GPU memory capacity, network link bandwidth utilization, GPU computing queue depth, and cross-rack latency metrics) to form a dynamic environment state vector. This state vector is then input into the decision network of a pre-trained reinforcement learning agent (RL-Agent). This network outputs a multi-dimensional decision vector through policy evaluation, which is then transformed into a specific executable data sharding strategy by the policy parsing engine. This strategy includes determining the sharding granularity (e.g., partitioning by attention head or sequence block), the target storage level for sharding (e.g., high-speed memory pool, NVMe SSD, or SCM persistent memory), the compression encoding method for each shard (e.g., lossless compression, INT8 quantization, or selective sparsity), and the data placement location (selecting nodes on the same rack or low-latency path based on topology awareness).

[0045] S6. Select the data transmission path according to the data fragmentation strategy.

[0046] In this embodiment of the invention, selecting the data transmission path according to the data sharding strategy includes: The data fragmentation strategy is parsed to generate a transmission task list; Obtain the network topology and network status of a preset transmission network, and construct a network status view based on the network topology and network status; The optimal transmission path is calculated based on the transmission task list and the network status view to obtain the initial path planning result; Based on the initial path planning results, transmission resources are reserved on the preset transmission network to obtain a list of reserved resources; Based on the reserved resource list, scheduling instructions are generated to obtain an instruction set; The transmission path is selected according to the instruction set to form the data transmission path.

[0047] In detail, the process of parsing the data sharding strategy to generate a transport task list involves identifying core instructions in the data sharding strategy, such as the source storage node, target storage node, shard size, priority label (e.g., "hot", "cold"), and SLO requirement for each data shard. Subsequently, it instantiates each shard's move or copy operation as an independent transport task object with a unique ID.

[0048] In detail, the step of obtaining the network topology and network status of the preset transmission network, and constructing a network status view based on the network topology and network status, is to construct a dynamic network status view with real-time costs by assigning dynamic weights (such as "delay cost" or "inverse of available bandwidth") to each edge in the network topology based on the network status.

[0049] In detail, the step of calculating the optimal transmission path based on the transmission task list and the network state view to obtain the initial path planning result involves inputting the transmission task list and the dynamic network state view into a path calculation engine. This engine runs a path optimization algorithm on the network state view for each task's node pair. For low-latency sensitive tasks, the K-shortest path algorithm is used to find the path with the fewest hops and lowest latency; for high-throughput tasks, the maximum bandwidth path algorithm is used.

[0050] In detail, the step of reserving transmission resources on a preset transmission network based on the initial path planning results to obtain a reserved resource list is a critical task requiring guaranteed performance (such as bandwidth and low jitter) in the initial path planning results. The system initiates a resource reservation request through the SDN controller. For example, a minimum guaranteed bandwidth queue is configured on the egress port of the relevant switch using the NETCONF / YANG model for data flows of a specific priority.

[0051] In detail, the step of generating scheduling instructions based on the reserved resource list to obtain an instruction set involves using an instruction generator to generate precise scheduling instructions for each transmission task that can directly drive the network card or network stack, based on the updated path planning (which tasks have been marked as having reserved resources) and the reserved resource list.

[0052] In detail, the step of selecting a transmission path based on the instruction set to form the data transmission path involves reading the instruction set and extracting the physical path information of each task, which is explicitly specified and bound to specific network resources. It associates the complete description of each path with the corresponding task ID, reserved ID, and scheduling instruction to generate a data transmission path.

[0053] S7. Perform task scheduling according to the data transmission path and generate task data blocks.

[0054] In this embodiment of the invention, the step of scheduling tasks according to the data transmission path and generating task data blocks includes: Generate a priority task queue based on the data transmission path; Based on the priority task queue, perform inter-task dependency analysis and construct a task dependency graph; The priority task queue is split into subtask queues based on the task dependency graph. A scheduling instruction set is generated based on the subtask queue; Obtain task data blocks according to the scheduling instruction set.

[0055] In detail, generating a priority task queue based on the data transmission path involves creating an empty task queue, extracting the transmission task definitions from the data transmission path, and then iterating through all task definitions. For each task, the scheduler calculates a dynamic priority score based on its associated data shard metadata (such as access popularity, whether it is a critical checkpoint, and the requested SLO), and encapsulates the task and score together into a schedulable task object and inserts it into the queue to obtain the priority task queue.

[0056] In detail, the step of analyzing inter-task dependencies and constructing a task dependency graph based on the priority task queue involves reading all tasks from the priority task queue and performing two rounds of scanning. The first round analyzes data dependencies: by comparing the source and destination addresses of tasks, producer-consumer relationships such as "the source address of task B is the destination address of task A" are identified. The second round analyzes resource dependencies: it checks whether the network ports and RDMA queues used by the task plans overlap. After obtaining the analysis results, a directed acyclic graph (DAG) is constructed, where each node represents a task and each directed edge represents a dependency relationship.

[0057] In detail, the subtask splitting of the priority task queue according to the task dependency graph to obtain the subtask queue is achieved by identifying all task clusters in the task dependency graph that have no direct dependencies and can be executed in parallel. Then, for each independent large-scale task within a cluster, the engine logically splits it into multiple smaller, data-independent subtasks according to the optimal partitioning dimension (e.g., along the "sequence" dimension or the "head" dimension) based on the currently available parallel resources (such as multi-channel RDMA links) and the data characteristics of the task itself (such as the shape of the KVCache tensor [layer, head, sequence, dimension]).

[0058] In detail, the step of generating a scheduling instruction set based on the subtask queue means that for each subtask, the compiler generates one or more low-level hardware-executable instructions based on its bound source / target node, memory address, and path information.

[0059] In detail, the step of obtaining task data blocks according to the scheduling instruction set involves parsing the scheduling instruction set and extracting the source data memory address (located in the shared memory pool) corresponding to each data transmission operation. Subsequently, it uses a memory management unit or a direct memory access engine to prefetch the original data blocks pointed to by these addresses into a temporary contiguous buffer.

[0060] S8. Write the task data block into a preset distributed cluster.

[0061] In this embodiment of the invention, writing the task data block to a preset distributed cluster involves routing the task data block to a target storage node group based on preset distributed cluster metadata (including node topology, storage tier policies, and real-time load), and then performing a parallel write pipeline on each data block: On the client side, the data block is divided into fixed-size stripes and a checksum block is generated by a GPU-accelerated erasure coding encoder; the encoded data stripes are transmitted in batches and concurrently to the persistent memory buffer of the corresponding storage node via an RDMA network; each storage node, upon receiving the data, immediately performs local integrity verification and writes it to the log structure, then asynchronously flushes it to the NVMe SSD and updates the local metadata index through atomic operations; finally, the coordinator collects write confirmations from all nodes and atomically commits the write transaction in the global metadata service, completing the persistent storage and index update of the data block in the distributed cluster.

[0062] In the healthcare field, this solution can be applied to a "multimodal medical AI diagnostic and research analysis platform." By using a large number of intermediate feature vectors generated by medical image segmentation models and genome analysis models as "KV caches," these vectors can be reused in a distributed shared memory pool. For example, different hospitals can share the same feature encoding cache for CT image analysis of the same typical case. In emergency scenarios, RL-Agent dynamic slicing and RDMA path selection technologies ensure that multimodal data streams of critically ill patients (such as real-time vital signs streams and historical images in the ICU) are prioritized for high-speed channels, achieving sub-second diagnostic inference. Simultaneously, erasure coding technology securely and cost-effectively stores research checkpoints of massive patient queues, accelerating model training iterations in new drug development, thereby significantly improving the response speed, diagnostic consistency, and large-scale research efficiency of medical AI systems.

[0063] In the fintech field, this solution can build a "high-performance real-time risk calculation and quantitative trading engine." With millisecond-level latency requirements, the system uses intermediate states generated by high-frequency pricing models (such as Monte Carlo simulations of options) as a shared "KV cache" in a memory pool. This allows a large number of similar derivative pricing requests to reuse some calculation results, increasing concurrent processing capabilities several times over. Through topology-aware scheduling and dynamic sharding, the system ensures high-speed synchronization of core risk factor calculation data along the optimal RDMA path between computing nodes and the NVMe-oF storage cluster during peak trading periods, achieving second-level full reassessment of portfolio risk values. Simultaneously, dynamic resource awareness based on reinforcement learning can intelligently respond to sudden loads such as "surges in market volatility," ensuring stable and low-latency operation of the risk control system under extreme market conditions.

[0064] As can be seen, in the above scheme, core fields are extracted from user requests and a basic information structure is constructed. Based on this, the request type is determined, and then the corresponding model slicing strategy, tensor description, or input sequence parameters are extracted. Combined with the model registry and associated storage strategy, key derived parameters, including cache size and hash key, are calculated and encapsulated into unified structured request parameters. Based on this, the system quickly determines whether there are reusable KV cache slices in the shared memory pool through consistent hash indexing: extracting elements from structured parameters to generate unique cache keys, locating the target node of the hash ring, and querying metadata; if they exist, the hash position is parsed, and the cache is directly obtained from the memory pool via RDMA connection; if they do not exist, the cache creation process is triggered: dynamically allocating computing resources, loading model weights, executing Transformer forward computation to generate a KV cache tensor set, and formatting the output. Subsequently, the system integrates KV cache information with real-time load data, dynamically formulates data slicing strategies with the help of reinforcement learning agents, and clarifies the slicing granularity, storage level, and compression method. According to this strategy, the system further parses and generates a transmission task list, plans the optimal RDMA transmission path based on network topology and real-time status, reserves network resources, and generates a scheduling instruction set. Based on the instruction set, the system schedules tasks to generate task data blocks and writes them to a distributed cluster in parallel to achieve efficient and reliable distributed storage management.

[0065] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0066] In one embodiment, a distributed storage device is provided, which corresponds one-to-one with the distributed storage method described in the above embodiments. For example... Figure 3 As shown, the distributed storage device includes a request parsing module 101, a cache judgment module 102, a path selection module 103, a task scheduling module 104, and a data writing module 105. Detailed descriptions of each functional module are as follows: The request parsing module 101 is used to parse the pre-acquired user request to obtain structured request parameters; The cache judgment module 102 is used to determine whether there is a reusable KV cache fragment in the preset shared memory pool based on the preset consistent hash index table. If it exists, the hash position of the reusable KV cache fragment in the shared memory pool is output, and the KV cache is obtained from the shared memory pool according to the hash position. If it does not exist, the KV cache is created according to the structured request parameters. The path selection module 103 is used to confirm the data sharding strategy based on the KV cache and the real-time acquired system load data, and select the data transmission path according to the data sharding strategy. Task scheduling module 104 is used to schedule tasks according to the data transmission path and generate task data blocks; The data writing module 105 is used to write the task data block into a preset distributed cluster.

[0067] In one embodiment, the request parsing module 101, when performing the parsing of the pre-acquired user request to obtain structured request parameters, is specifically used for: Extract the core fields from the user request; Construct a basic request information structure based on the core fields; Based on the basic request information structure, determine whether the user request is a training request; If so, extract the model slicing strategy and parameter tensor description from the user request to form a task data block; If not, the user request is determined to be a reasoning request, and the input sequence Token ID, generation parameters and sequence unique identifier in the user request are extracted to form a task data block; Based on the basic request information structure and the request parameter structure, the associated model structure and storage strategy configuration are used to obtain the model storage configuration structure; Calculate key derived parameters based on the model storage configuration structure, the basic request information structure, and the request parameter structure; The key derived parameters are encapsulated into a unified structured object to obtain structured request parameters.

[0068] In one embodiment, the cache determination module 102, when performing the step of determining whether a reusable KV cache fragment exists in a preset shared memory pool based on a preset consistent hash index table, is specifically used for: Extract the elements required for cache key generation from the structured request parameters to generate a cache key generation element set; Generate a unique cache key based on the cache key and the element set generated; The target node in the preset consistent hash ring is located based on the unique cache key, and the target storage node identifier is obtained; Based on the target storage node identifier, perform a cache metadata query in the consistent hash index table to obtain the cache metadata query result; If the cache metadata query result indicates the existence of corresponding cache metadata, then it is determined that there is a reusable KV cache fragment in the preset shared memory pool. If the cache metadata query result indicates that there is no corresponding cache metadata, then it is determined that there is no reusable KV cache shard in the preset shared memory pool.

[0069] In one embodiment, the cache determination module 102, when performing the step of retrieving the KV cache from the shared memory pool based on the hash position, is specifically used for: The hash position is parsed to obtain the parsed position; Extract the target node identifier contained in the parsed position, establish a shared memory pool connection based on the target node identifier, and construct a cache key generation element set; Construct an access request structure based on the network connection handle; The KV cache is read based on the network connection handle and the access request structure.

[0070] In one embodiment, the cache determination module 102, when performing the creation of the KV cache based on the structured request parameters, is specifically used for: Resource allocation is performed based on the structured request parameters to obtain the resource allocation result; Extract the model identifier contained in the structured request parameters, and load the preset large speech model's required model calculation context data according to the weight path corresponding to the model identifier and the resource allocation result; The context data is sequence-embedded and position-encoded based on the input sequence Token ID in the structured request parameters to generate an embedded sequence representation; The large speech model is used to perform forward computation of the embedded sequence representation and the resource allocation result using the Transformer layer, and outputs a set of KV buffer tensors. The KV cache tensor set is formatted to obtain the KV cache.

[0071] In one embodiment, the path selection module 103, when performing the step of selecting a data transmission path according to the data fragmentation strategy, is specifically used for: The data fragmentation strategy is parsed to generate a transmission task list; Obtain the network topology and network status of a preset transmission network, and construct a network status view based on the network topology and network status; The optimal transmission path is calculated based on the transmission task list and the network status view to obtain the initial path planning result; Based on the initial path planning results, transmission resources are reserved on the preset transmission network to obtain a list of reserved resources; Based on the reserved resource list, scheduling instructions are generated to obtain an instruction set; The transmission path is selected according to the instruction set to form the data transmission path.

[0072] In one embodiment, when the task scheduling module 104 performs task scheduling based on the data transmission path and generates task data blocks, it is specifically used for: Generate a priority task queue based on the data transmission path; Based on the priority task queue, perform inter-task dependency analysis and construct a task dependency graph; The priority task queue is split into subtask queues based on the task dependency graph. A scheduling instruction set is generated based on the subtask queue; Obtain task data blocks according to the scheduling instruction set.

[0073] This invention provides a distributed storage device that can extract core fields from user requests and construct a basic information structure. Based on this, the request type is determined, and then the corresponding model slicing strategy, tensor description, or input sequence parameters are extracted. Combined with the model registry and associated storage strategy, key derived parameters, including cache size and hash keys, are calculated and encapsulated into unified structured request parameters. Based on this, the system quickly determines whether reusable KV cache slices exist in the shared memory pool using a consistent hash index: extracting elements from structured parameters to generate unique cache keys, locating the target node of the hash ring, and querying metadata; if they exist, the hash position is parsed, and the cache is directly obtained from the memory pool via RDMA connection; if they do not exist, a cache creation process is triggered: dynamically allocating computing resources, loading model weights, executing Transformer forward computation to generate a KV cache tensor set, and formatting the output. Subsequently, the system integrates KV cache information with real-time load data, dynamically formulates a data slicing strategy using a reinforcement learning agent, and clarifies the slicing granularity, storage level, and compression method. According to this strategy, the system further parses and generates a transmission task list, plans the optimal RDMA transmission path based on network topology and real-time status, reserves network resources, and generates a scheduling instruction set. Based on the instruction set, the system schedules tasks to generate task data blocks and writes them to a distributed cluster in parallel to achieve efficient and reliable distributed storage management.

[0074] For specific limitations regarding distributed storage devices, please refer to the limitations on distributed storage methods described above, which will not be repeated here. Each module in the aforementioned distributed storage device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device in hardware form, or stored in the memory of a computer device in software form, so that the processor can call and execute the operations corresponding to each module.

[0075] In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 4 As shown, the computer device includes a processor, memory, network interface, and database connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile and / or volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and database. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The network interface is used to communicate with external clients via a network connection. When the computer program is executed by the processor, it implements the functions or steps of a distributed storage method on the server side.

[0076] In one embodiment, a computer device is provided, which may be a client, and its internal structure diagram may be as follows: Figure 5 As shown, the computer device includes a processor, memory, network interface, display screen, and input devices connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The network interface is used to communicate with an external server via a network connection. When the computer program is executed by the processor, it implements the functions or steps on the client side of a distributed storage method.

[0077] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to perform the following steps: The pre-acquired user requests are parsed to obtain structured request parameters; The system determines whether the structured request parameters have a reusable KV cache fragment in the preset shared memory pool based on the preset consistent hash index table. If it exists, output the hash position of the reusable KV cache fragment in the shared memory pool, and retrieve the KV cache from the shared memory pool according to the hash position; If it does not exist, a KV cache is created based on the structured request parameters; The data sharding strategy is confirmed based on the KV cache and the real-time acquired system load data. Select the data transmission path according to the data fragmentation strategy; Task scheduling is performed according to the data transmission path to generate task data blocks; Write the task data block into a preset distributed cluster.

[0078] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, the computer program performing the following steps when executed by a processor: The pre-acquired user requests are parsed to obtain structured request parameters; The system determines whether the structured request parameters have a reusable KV cache fragment in the preset shared memory pool based on the preset consistent hash index table. If it exists, output the hash position of the reusable KV cache fragment in the shared memory pool, and retrieve the KV cache from the shared memory pool according to the hash position; If it does not exist, a KV cache is created based on the structured request parameters; The data sharding strategy is confirmed based on the KV cache and the real-time acquired system load data. Select the data transmission path according to the data fragmentation strategy; Task scheduling is performed according to the data transmission path to generate task data blocks; Write the task data block into a preset distributed cluster.

[0079] It should be noted that the functions or steps that can be implemented by the computer-readable storage medium or computer device described above can be referred to the relevant descriptions on the server side and client side in the foregoing method embodiments. To avoid repetition, they will not be described one by one here.

[0080] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0081] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is used as an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.

[0082] Finally, it should be noted that if any software tools or components not belonging to this company appear in the embodiments of the application, they are merely illustrative examples and do not represent actual use. The embodiments described above are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included within the protection scope of the present invention.

Claims

1. A distributed storage method, characterized in that, include: The pre-acquired user requests are parsed to obtain structured request parameters; The system determines whether the structured request parameters have a reusable KV cache fragment in the preset shared memory pool based on the preset consistent hash index table. If it exists, output the hash position of the reusable KV cache fragment in the shared memory pool, and retrieve the KV cache from the shared memory pool according to the hash position; If it does not exist, a KV cache is created based on the structured request parameters; The data sharding strategy is confirmed based on the KV cache and the real-time acquired system load data. Select the data transmission path according to the data fragmentation strategy; Task scheduling is performed according to the data transmission path to generate task data blocks; Write the task data block into a preset distributed cluster.

2. The distributed storage method as described in claim 1, characterized in that, The step of parsing the pre-acquired user request to obtain structured request parameters includes: Extract the core fields from the user request; Construct a basic request information structure based on the core fields; Determine whether the user request is a training request based on the basic request information structure; If so, extract the model slicing strategy and parameter tensor description from the user request to form a task data block; If not, the user request is determined to be a reasoning request, and the input sequence Token ID, generation parameters and sequence unique identifier in the user request are extracted to form a task data block; Based on the basic request information structure and the request parameter structure, the associated model structure and storage strategy configuration are used to obtain the model storage configuration structure; Calculate key derived parameters based on the model storage configuration structure, the basic request information structure, and the request parameter structure; The key derived parameters are encapsulated into a unified structured object to obtain structured request parameters.

3. The distributed storage method as described in claim 1, characterized in that, The step of determining whether the structured request parameters have a reusable KV cache fragment in a preset shared memory pool based on a preset consistent hash index table includes: Extract the elements required for cache key generation from the structured request parameters to generate a cache key generation element set; Generate a unique cache key based on the cache key and the element set generated; The target node in the preset consistent hash ring is located based on the unique cache key, and the target storage node identifier is obtained; Based on the target storage node identifier, perform a cache metadata query in the consistent hash index table to obtain the cache metadata query result; If the cache metadata query result indicates the existence of corresponding cache metadata, then it is determined that there is a reusable KV cache fragment in the preset shared memory pool. If the cache metadata query result indicates that there is no corresponding cache metadata, then it is determined that there is no reusable KV cache shard in the preset shared memory pool.

4. The distributed storage method as described in claim 1, characterized in that, The step of retrieving the KV cache from the shared memory pool based on the hash position includes: The hash position is parsed to obtain the parsed position; Extract the target node identifier contained in the parsed position, establish a shared memory pool connection based on the target node identifier, and construct a cache key generation element set; Construct an access request structure based on the network connection handle; The KV cache is read based on the network connection handle and the access request structure.

5. The distributed storage method as described in claim 1, characterized in that, The step of creating a KV cache based on the structured request parameters includes: Resource allocation is performed based on the structured request parameters to obtain the resource allocation result; Extract the model identifier contained in the structured request parameters, and load the preset large speech model's required model calculation context data according to the weight path corresponding to the model identifier and the resource allocation result; The context data is sequence-embedded and position-encoded based on the input sequence Token ID in the structured request parameters to generate an embedded sequence representation; The large speech model is used to perform forward computation of the embedded sequence representation and the resource allocation result using the Transformer layer, and outputs a set of KV buffer tensors. The KV cache tensor set is formatted to obtain the KV cache.

6. The distributed storage method as described in claim 1, characterized in that, The step of selecting a data transmission path according to the data fragmentation strategy includes: The data fragmentation strategy is parsed to generate a transmission task list; Obtain the network topology and network status of a preset transmission network, and construct a network status view based on the network topology and network status; The optimal transmission path is calculated based on the transmission task list and the network status view to obtain the initial path planning result; Based on the initial path planning results, transmission resources are reserved on the preset transmission network to obtain a list of reserved resources; Based on the reserved resource list, scheduling instructions are generated to obtain an instruction set; The transmission path is selected according to the instruction set to form the data transmission path.

7. The distributed storage method as described in claim 1, characterized in that, The step of scheduling tasks according to the data transmission path and generating task data blocks includes: Generate a priority task queue based on the data transmission path; Based on the priority task queue, perform inter-task dependency analysis and construct a task dependency graph; The priority task queue is split into subtask queues based on the task dependency graph. A scheduling instruction set is generated based on the subtask queue; Obtain task data blocks according to the scheduling instruction set.

8. A distributed storage device, characterized in that, include: The request parsing module is used to parse pre-acquired user requests to obtain structured request parameters; The cache determination module is used to determine whether there is a reusable KV cache fragment in the preset shared memory pool based on the preset consistent hash index table. If it exists, the hash position of the reusable KV cache fragment in the shared memory pool is output, and the KV cache is obtained from the shared memory pool according to the hash position. If it does not exist, the KV cache is created according to the structured request parameters. The path selection module is used to confirm the data sharding strategy based on the KV cache and the real-time acquired system load data, and select the data transmission path according to the data sharding strategy. The task scheduling module is used to schedule tasks according to the data transmission path and generate task data blocks; The data writing module is used to write the task data block into a preset distributed cluster.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the distributed storage method as described in any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the distributed storage method as described in any one of claims 1 to 7.

Images