Data writing method and apparatus, data reading method and apparatus, storage controller

By optimizing physical address allocation based on co-activation relationships in solid-state drives, the problem of dispersed distribution of expert weights in hybrid expert models is solved, thereby improving data reading speed.

CN122633131BActive Publication Date: 2026-11-10MAXIO TECHNOLOGY (HANGZHOU) CO LTD
View PDF 4 Cites 0 Cited by

Patent Information

Application Number
CN202611133559.0
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2026-07-28
Publication Date
2026-11-10
Estimated Expiration
2046-07-28

Smart Images

  • Figure CN122633131B_ABST
    Figure CN122633131B_ABST
Patent Text Reader

Abstract

This invention relates to the field of storage technology and discloses a data writing method and apparatus, a data reading method and apparatus, and a storage controller, applied to solid-state drives (SSDs). It addresses the problem in existing solutions where physical addresses are allocated only according to the writing order, resulting in physically dispersed expert weights and low read speeds. Specifically, the invention discloses: in response to a weight write request for any expert, obtaining the identifier and weight of any expert; determining the target physical address of any expert in the SSD based on the identifier and co-activation relationship of any expert; storing the weights of at least two experts with co-activation relationships in a region with adjacent physical addresses; and writing the weight of any expert, starting from the same page in multiple storage dies, in parallel to the target physical address, so that the weights of co-activated experts are ordered and adjacent in the physical storage space, thereby reducing random read operations and improving data read speed.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention relates to the field of storage technology, and in particular to a data writing method and apparatus, a data reading method and apparatus, and a storage controller. Background Technology

[0002] Currently, when storing data on SSDs (Solid State Drives), a physical address allocation scheme based on write order is typically used. This means that the flash translation layer allocates physical addresses to the data according to the order in which write requests arrive, based on the currently available physical storage space, and then writes the data to the corresponding storage chip.

[0003] While the above scheme can achieve basic data storage, it only considers the writing order during storage. Therefore, for scenarios that require multiple rounds of writing, such as inference scenarios using hybrid expert models, the above scheme will not be aware of the correlation between different experts between multiple rounds of writing because it only considers the writing order. This results in the weights of multiple experts that need to be read simultaneously in one inference being distributed in the physical storage space, leading to a large number of discrete random read operations during reading and reducing the data reading speed. Summary of the Invention

[0004] The main objective of this invention is to provide a data writing method and apparatus, a data reading method and apparatus, and a storage controller, which aims to solve the problem that existing physical address allocation schemes based on writing order only consider the writing order and do not perceive the correlation between different experts, resulting in the weights of multiple experts that need to be read simultaneously in a single inference being scattered in the physical storage space and the reading speed being low.

[0005] In a first aspect, embodiments of this disclosure provide a data writing method applied to a solid-state drive (SSD), wherein the SSD is used to store the weights of multiple experts in a hybrid expert model; the SSD includes multiple storage dies that can be accessed in parallel, and the method includes: In response to a weight write request for any expert, obtain the identifier and weight of that expert. Based on the identifier and co-activation relationship of any expert, the target physical address of any expert in the solid-state drive is determined; wherein, the co-activation relationship is determined based on at least two experts among the plurality of experts who are simultaneously activated in the same inference iteration; the weights of the at least two experts with the co-activation relationship are stored in a region where the physical addresses are adjacent. The weight of any expert is written in parallel to the target physical address, starting from the same page in the plurality of storage dies; wherein, the identifier of any expert and the target physical address are used to update a preset mapping table; the preset mapping table is used to record the correspondence between expert identifiers and physical addresses.

[0006] Secondly, embodiments of this disclosure provide a data reading method applied to a solid-state drive (SSD), wherein the SSD is used to store the weights of multiple experts in a hybrid expert model; the SSD includes multiple storage dies that can be accessed in parallel; the method includes: Based on the obtained weight read request, determine the identifiers of multiple target experts whose weights are to be read; Based on a preset mapping table and the identifiers of the multiple target experts, multiple target physical addresses of the multiple target experts are obtained; wherein, the preset mapping table is used to record the correspondence between expert identifiers and physical addresses; According to the physical address order, the multiple target physical addresses are arranged, and adjacent or overlapping target physical addresses after arrangement are merged to obtain a reading plan; wherein, the merged target physical addresses store the weights of at least two experts with co-activation relationship; the co-activation relationship indicates that at least two experts among the multiple experts are simultaneously activated in the same inference iteration; Weights are read from the solid-state drive according to the read plan.

[0007] Thirdly, embodiments of this disclosure provide a data writing device applied to a solid-state drive (SSD), the SSD being used to store the weights of multiple experts in a hybrid expert model; the SSD includes multiple storage dies accessible in parallel, and the device includes: The processing module is used to obtain the identifier and weight of any expert in response to a weight write request for any expert. A determination module is used to determine the target physical address of any expert in the solid-state drive based on the identifier and co-activation relationship of any expert; wherein, the co-activation relationship is determined based on at least two experts that are simultaneously activated in the same inference iteration among the plurality of experts; the weights of the at least two experts with the co-activation relationship are stored in a region where the physical addresses are adjacent. The writing module is used to write the weight of any expert to the target physical address in parallel, starting from the same page in the plurality of storage dies; wherein, the identifier of any expert and the target physical address are used to update a preset mapping table; the preset mapping table is used to record the correspondence between expert identifiers and physical addresses.

[0008] Fourthly, embodiments of this disclosure provide a data reading device applied to a solid-state drive (SSD), the SSD being used to store the weights of multiple experts in a hybrid expert model; the SSD includes multiple storage dies that can be accessed in parallel; the device includes: The analysis module is used to determine the identifiers of multiple target experts whose weights need to be read based on the obtained weight read requests; The acquisition module is used to acquire multiple target physical addresses of the multiple target experts according to a preset mapping table and the identifiers of the multiple target experts; wherein, the preset mapping table is used to record the correspondence between expert identifiers and physical addresses; A generation module is used to arrange the plurality of target physical addresses according to the physical address order, and merge adjacent or overlapping target physical addresses after arrangement to obtain a reading plan; wherein, the merged target physical addresses store the weights of at least two experts with a co-activation relationship; the co-activation relationship indicates that at least two experts among the plurality of experts are simultaneously activated in the same inference iteration; A read module is used to read weights from the solid-state drive according to the read plan.

[0009] Fifthly, embodiments of this disclosure provide a storage controller, which includes at least a processor, firmware memory, a host interface, and a flash memory interface; The firmware memory stores a physical layout engine and a read execution engine. The physical layout engine determines the target physical address of expert weights in a non-volatile storage medium based on expert identifiers and co-activation relationships. The co-activation relationship is determined based on at least two experts who are simultaneously activated in the same inference iteration among multiple experts. The read execution engine maps discrete expert weight read requests to continuous physical read plans according to a preset mapping table. The preset mapping table records the correspondence between expert identifiers and physical addresses. The host interface is used to receive weight write requests and weight read requests; The flash memory interface is used to issue read and write commands to the non-volatile storage medium; The processor, the firmware memory, the host interface, and the flash memory interface are configured to collaboratively implement the steps of the method as described in the first or second aspect above.

[0010] In a sixth aspect, embodiments of this disclosure provide a storage device, including a storage controller as described in the fifth aspect above, and a non-volatile storage medium.

[0011] In a seventh aspect, embodiments of this disclosure provide a computing device including a storage controller as described in the sixth aspect above, and a processor.

[0012] Eighthly, embodiments of this disclosure provide a computer-readable storage medium for storing computer-executable instructions that, when executed by a processor, implement the steps of the method described in the first or second aspect above.

[0013] Ninthly, embodiments of this disclosure provide a computer program product, the computer program product including a computer program, which, when executed by a processor, implements the steps of the method described in the first or second aspect above.

[0014] The at least one technical solution provided by the embodiments of the present invention can achieve the following technical effects: In this embodiment of the invention, in response to a weight write request for any expert, the expert's identifier and weight are obtained. Based on the expert's identifier and co-activation relationship, the target physical address of the expert in the solid-state drive (SSD) is determined. The co-activation relationship is determined based on at least two experts from multiple experts in a hybrid expert model stored on the SSD who are simultaneously activated in the same inference iteration; the weights of these at least two experts with a co-activation relationship are stored in a region with adjacent physical addresses. Then, the expert's weight is written in parallel to the target physical address, starting from the same page in multiple parallel-accessible storage dies of the SSD. The expert's identifier and target physical address are used to update a preset mapping table, which records the correspondence between expert identifiers and physical addresses.

[0015] In this embodiment of the invention, when writing expert weights, the target physical address is determined based on the co-activation relationship, ensuring that the weights of at least two experts that need to be read simultaneously in the same inference are stored in regions with adjacent physical addresses. Simultaneously, parallel writing is performed starting from the same page in multiple storage dies, and a preset mapping table is updated to record the correspondence between expert identifiers and physical addresses. This changes the passive approach of prior art, which only allocates physical addresses according to the writing order without considering the correlation between experts. It ensures that the weights of co-activated experts are arranged in an ordered manner in the physical storage space rather than being scattered, effectively overcoming the technical defect of requiring a large number of discrete random read operations when reading co-activated expert weights, and effectively improving data reading speed. Attached Figure Description

[0016] Figure 1 This is one of the flowcharts illustrating the data writing method provided in an embodiment of the present invention; Figure 2 This is a schematic diagram of the arrangement provided in an embodiment of the present invention; Figure 3 A second schematic flowchart of the data writing method provided in an embodiment of the present invention; Figure 4This is one of the flowcharts illustrating the data reading method provided in an embodiment of the present invention; Figure 5 A second schematic flowchart of the data reading method provided in an embodiment of the present invention; Figure 6 The third flowchart illustrating the data reading method provided in this embodiment of the invention; Figure 7 The fourth flowchart illustrating the data reading method provided in this embodiment of the invention; Figure 8 A schematic diagram of the module composition of the data writing device 800 provided in an embodiment of the present invention; Figure 9 A schematic diagram of the module composition of the data reading device 900 provided in an embodiment of the present invention; Figure 10 This is a schematic diagram of the hardware structure of a storage device provided in one embodiment of the present invention. Detailed Implementation

[0017] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this invention, and not all of them. Based on the embodiments of this invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this invention.

[0018] The technical solutions provided by the various embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0019] Please see Figure 1 , Figure 1 This is one of the flowcharts illustrating the data writing method provided in this embodiment of the invention. It is applied to a solid-state drive (SSD), which stores the weights of multiple experts in a hybrid expert model and includes multiple storage dies that can be accessed in parallel, such as... Figure 1 As shown, the method includes the following steps: Step 102: In response to a weight write request for any expert, obtain the identifier and weight of any expert.

[0020] Step 104: Determine the target physical address of any expert in the solid-state drive based on the identifier and co-activation relationship of any expert; wherein, the co-activation relationship is determined based on at least two experts that are simultaneously activated in the same inference iteration among multiple experts; the weights of at least two experts with co-activation relationship are stored in the region where the physical addresses are adjacent.

[0021] Step 106: Write the weight of any expert to the target physical address in parallel, starting from the same page in multiple memory chips; wherein, the identifier of any expert and the target physical address are used to update the preset mapping table; the preset mapping table is used to record the correspondence between expert identifiers and physical addresses.

[0022] In this embodiment of the invention, in response to a weight write request for any expert, the identifier and weight of that expert can be obtained.

[0023] In this model, any expert can be an expert in a Mixed Experts (MOE) model. The MOE model is a neural network architecture that dynamically selects a subset of experts for computation using a gating network, with each expert having independent weight parameters. During inference, the gating network calculates the activation score for each expert based on the input data, selecting only the experts with the highest scores to participate in the inference calculation; for example, activating only 2 to 8 experts out of 64. Since the total number of expert weight parameters in an MOE model can reach hundreds of gigabytes or even terabytes, it is usually impossible for them all to reside in GPU (Graphics Processing Unit) memory. Therefore, the expert weights need to be stored on an SSD and loaded onto the GPU memory on demand during inference. This on-demand loading characteristic makes SSD read performance one of the key bottlenecks to the inference efficiency of the MOE model.

[0024] In the above scenario, the upper-layer (Host-side) MOE model inference engine issues expert weight write requests to the SSD via a dedicated interface or by attaching metadata to standard NVMe (Non-Volatile Memory Express) write commands. This write request may contain the expert's unique identifier and complete weight data. The write request may also include the size of the expert's weight data, such as the total length of the weight data in bytes; this information can be used to subsequently determine the size of the NAND physical storage space to be allocated.

[0025] In one example, the write request may also include the starting logical address of the expert's weight data, which can be used by the SSD firmware to establish a mapping from logical address to physical address in the standard FTL (Flash Translation Layer) mapping table to be compatible with the standard NVMe read / write path.

[0026] By responding to a weight write request for any expert and obtaining the expert's identifier and weight, the SSD firmware can acquire expert identifier and weight data during the write phase, providing an input basis for subsequent physical layout optimization. Since the expert identifier is defined during the write phase, the SSD firmware can plan the physical location before data is written to disk, controlling the distribution of expert weights in the NAND physical space from the source, avoiding the passive situation of writing first and then discovering problems and trying to remedy them.

[0027] In this embodiment of the invention, after obtaining the identifier and weight of any expert, the target physical address of any expert in the SSD can be determined based on the identifier and co-activation relationship of the expert.

[0028] The co-activation relationship can be determined based on at least two experts who are simultaneously activated in the same inference iteration among multiple experts. The weights of the at least two experts with a co-activation relationship can be stored in a region with adjacent physical addresses.

[0029] In one example, during the execution of the MOE model, the gating network might simultaneously activate expert 0, expert 1, and expert 2 in each inference iteration, indicating a co-activation relationship among these three experts. Since the input data differs across inference iterations, the gating network may activate different sets of experts. Therefore, by statistically analyzing the outputs of the gating network across a large number of inference iterations, it is possible to determine which experts tend to be activated simultaneously in the same inference iteration. This co-activation relationship reflects the actual access patterns during MOE model inference and can be used for subsequent physical layout optimization.

[0030] Co-activation relationships can originate from various sources. For example, they can be pre-provided by the upper-layer MOE model inference framework during model deployment based on offline statistical analysis and sent to the SSD along with the write request; alternatively, they can be pre-stored in the SSD and maintained locally by the SSD firmware. Therefore, regardless of whether the co-activation relationship is carried with the write request or pre-stored in the SSD, the SSD firmware can obtain the co-activation relationship, providing a basis for subsequent physical arrangement.

[0031] In one example, the physical layout engine in the SSD firmware layer can be used to determine the target physical address of any expert in the SSD based on the expert's identifier and co-activation relationship. Before explaining the specific method of determining the target physical address using the physical layout engine, it's helpful to explain the SSD's NAND flash memory architecture to facilitate understanding of the subsequent method for determining the target physical address: SSDs use NAND flash memory chips as their storage medium. NAND flash memory chips are organized in a hierarchical structure: an SSD contains multiple NAND channels, each channel has multiple memory dies, each die contains multiple memory blocks, and each block contains multiple memory pages. A memory page is the smallest unit of reading and writing in NAND flash memory, typically 16 kilobytes (KB). A memory block is the smallest unit of erasure, consisting of hundreds to thousands of memory pages. A key characteristic of NAND flash memory is that when sequentially reading adjacent pages within the same block, row buffers can be reused, reducing addressing overhead. Therefore, sequential read performance is significantly better than random read performance, with the former reaching 3 to 5 gigabytes per second (GB / s) compared to the latter's approximately 0.5 to 1 GB / s. This characteristic allows us to determine that if expert weights can be neatly arranged rather than randomly scattered in the NAND physical space, read efficiency will be improved by orders of magnitude.

[0032] Based on the aforementioned NAND flash memory architecture, the physical layout engine can follow four physical layout principles when determining the target physical address. The following combines... Figure 2 Please provide a detailed explanation. Figure 2 This is a diagram illustrating the physical arrangement of expert weights. Figure 2 The left side shows the situation where expert weights are randomly distributed under the traditional arrangement method. Figure 2 The right side shows the effect of striping the same expert across memory die and memory page under this example layout.

[0033] The first principle is: striping of the same expert across memory dies and memory pages. All weight parameters of the same expert are distributed across the same memory page rows on all memory dies, i.e., striping across memory dies and memory pages. For example... Figure 2 As shown on the right, expert 0's data is simultaneously located on storage page 0 of storage die 0, storage page 0 of storage die 1, storage page 0 of storage die 2, and storage page 0 of storage die 3; expert 1's data is simultaneously located on storage page 1 of storage die 0, storage page 1 of storage die 1, storage page 1 of storage die 2, and storage page 1 of storage die 3. Since expert weight data is typically much larger than the capacity of a single storage page—for example, expert weights can reach 2 MB or even larger—while storage page sizes are typically 16 KB, the weight data for a single expert needs to span multiple storage pages. Based on this principle, the contents of multiple storage pages for the same expert are allocated to the same storage page row across all storage dies. The SSD controller can simultaneously read all the weight data of that expert from all storage dies in parallel using a single multi-plane read command, fully utilizing the multi-die parallel bandwidth during single-expert reads.

[0034] The second principle is: physical proximity of co-activated experts. The weights of at least two experts with co-activation relationships are stored in physically adjacent regions. For example... Figure 2As shown on the right, experts 0, 1, and 2, which have a co-activation relationship, occupy storage page rows 0, 1, and 2 respectively (each row spanning all memory dies), as well as storage page rows 3, 4, and 5. Since these three experts are physically adjacent, when aggregating the reading of these three experts, the NAND controller only needs to sequentially move the storage page row pointers to continuously complete the reading of all experts, minimizing addressing overhead. This principle complements the first principle: the first principle guarantees parallel bandwidth across memory dies for experts with the same characteristics, while the second principle guarantees the sequential reading continuity between co-activation experts. In MOE model inference, a single inference iteration typically requires loading multiple co-activation experts; therefore, the physical proximity effect of the second principle is particularly prominent in practical inference scenarios. In traditional arrangement methods, such as... Figure 2 As shown on the left, taking fragments from Expert 0 to Expert 3 as an example, these fragments are randomly scattered across different memory page rows on various memory chips. Reading requires multiple random addressings, making it impossible to utilize the advantage of parallel reading across multiple memory chips, resulting in a read throughput of only about 0.5 to 1 GB / s. The arrangement in this example, however, increases the read throughput to 3 to 5 GB / s.

[0035] The third principle is storage block alignment. Each expert's weight data is stored in NAND physical space aligned to storage block boundaries. By storing expert weight data aligned to storage block boundaries, it ensures that a single expert's read will not cross storage block boundaries, avoiding the additional addressing latency caused by switching target storage blocks when reading across storage blocks. Simultaneously, during the reclamation mechanism, since expert data within a storage block has the same access characteristics (i.e., co-activation attribute), it can be processed as a whole during reclamation and migration, reducing the complexity of data migration. The reclamation mechanism can refer to a space reclamation mechanism within the SSD: when the proportion of effective storage pages in a storage block falls below a certain threshold, resulting in insufficient available storage blocks, the SSD firmware will migrate the effective data in that storage block to a new free storage block, and then erase the original storage block to reclaim the storage space within it. Details will be provided later.

[0036] The fourth principle is: storage page alignment. The starting address of each expert weight data is aligned to the storage page boundary, ensuring that read operations always begin from the beginning of the storage page, avoiding additional addressing latency caused by reading across storage pages. Simultaneously, the end of each expert weight data is also aligned to the storage page boundary through data padding. Specifically, the size of the expert weight data is not always an integer multiple of the storage page size. If the expert weight data is 2100 KB and the storage page size is 16 KB, then the expert weight data occupies 131.25 storage pages, that is, 131 complete storage pages plus 4 KB of remaining data. According to this principle, the remaining 4 KB needs to be padded to 16 KB, aligning its end to the storage page boundary. In this way, the starting address of the next expert falls exactly at the beginning of the next storage page. The storage pages of the entire weight storage area are uniformly utilized, and the physical transition between adjacent experts is always the boundary of a complete storage page. During reading, there is no need to handle the boundary cases of partial storage pages, eliminating fragmentation overhead.

[0037] The four physical arrangement principles described above work together to form a complete expert-weighted physical arrangement scheme. The first principle guarantees the parallel bandwidth of multiple storage chips during a single expert's read; the second principle guarantees the sequentiality of multiple experts' aggregated reads; and the third and fourth principles guarantee the neatness of storage space management and the stability of read efficiency. These four principles work together from different dimensions, enabling SSDs to achieve performance close to the theoretical upper limit of NAND sequential reads when performing expert-weighted reads.

[0038] In one example, the target physical address can be determined not only based on expert identifiers and co-activation relationships, but also simultaneously based on storage block alignment constraints and storage page alignment constraints. Specifically, the target physical address of any expert in the SSD can be determined based on the expert's identifier, co-activation relationship, storage block alignment constraints, and storage page alignment constraints. The storage block alignment constraint limits the starting physical address of the expert's weights to be aligned to the boundary of the storage block; the storage page alignment constraint limits the physical space occupied by the expert's weights to an integer multiple of the storage page size. When determining the physical address, the physical layout engine ensures that the selected starting physical address is exactly located in the first storage page of a storage block. When the amount of expert weight data is less than one storage block, the physical layout engine still allocates a complete storage block for it, leaving unused storage pages empty or for future data filling, to ensure that the expert's weight data does not share the same storage block with the weight data of other experts. When calculating the number of storage pages required for expert weight data, the physical layout engine can use a rounding-up method: if the weight data size is S bytes and the storage page size is P bytes, then the required number of storage pages is the rounding-up function ceil(S / P). During writing, if the last storage page is not full, it is padded with data to fill the entire storage page.

[0039] In one example, besides using storage pages as the alignment granularity, a coarser-grained mapping cell alignment constraint can be used to determine the target physical address. Specifically, the target physical address of any expert can be determined based on the expert's identifier, co-activation relationship, and mapping cell alignment constraint. The mapping cell alignment constraint can be used to limit the alignment of the starting physical address of the expert weight to the mapping cell boundary, and the expert weight is stored in units of an integer number of mapping cells. The mapping cell can be the preset minimum management granularity of the logical address to physical address mapping table maintained by the flash translation layer of the solid-state drive. The granularity of the mapping cell can be larger than the granularity of the storage page, so that the number of mapping entries required to record a single expert in the preset mapping table is less than the number of mapping entries required when managing at the storage page granularity. After aligning the starting address and length of the expert weight data according to the mapping cell boundary, NAND read operations are always on the integer boundary of the mapping cell, eliminating the need to handle boundary cases across mapping cells, thereby reducing logical-to-physical address translation latency. While coarser alignment granularity may introduce a small amount of data padding waste, the waste is extremely low for most expert weights (typically in the MB to GB range), usually less than one percent, and is acceptable in engineering practice.

[0040] In one example, the storage space of the solid-state drive (SSD) can be divided into multiple partitions, and the target partition corresponding to any expert can be determined based on the identifier and co-activation relationship of any expert. At least two experts with a co-activation relationship are assigned to the same partition. The current write position within the target partition can be determined as the target physical address of any expert on the SSD. Sequential writing is used within the target partition so that the weights of experts written to the target partition sequentially are adjacent in physical storage location.

[0041] One specific implementation of the above partitioning scheme is as follows: Utilizing the NVMe ZNS (Zoned Namespace) standard, the namespace is divided into multiple partitions, requiring sequential writes within each partition. Each expert or each group of co-activated experts is assigned to an independent partition, and the sequential write mechanism within the partition naturally ensures the physical continuity of the weight data for the same expert. This example effectively utilizes the existing NVMe standard without requiring custom SSD firmware extensions.

[0042] In one example, the weights of at least two experts with a co-activation relationship can be assigned the same reclaimed region identifier. Based on the identifier of any expert and the co-activation relationship, the target reclaimed region identifier corresponding to any expert is determined, and the write position in the reclaimed region corresponding to the target reclaimed region identifier is determined as the target physical address of any expert in the solid-state drive, so that the weights of at least two experts with a co-activation relationship are physically assigned to the same reclaimed region through the reclaimed region identifier.

[0043] One specific implementation of the above-mentioned reclamation area scheme can be achieved using the NVMe FDP (Flexible Data Placement) standard. FDP allows the host to control the physical location of data in the NAND flash memory through reclamation unit handles, where the reclamation unit handle in the FDP corresponds to the aforementioned reclamation area identifier. An independent reclamation unit handle can be assigned to each group of co-activated experts, and their physical location in the NAND flash memory is controlled by specifying the reclamation unit handle when writing expert weights. This example utilizes the existing NVMe FDP standard, offering good compatibility. The granularity of the reclamation unit is typically at the storage block or superblock level. While it cannot be as fine as the storage page level, read optimization can be achieved by combining it with the preset mapping table proposed in this example to compensate for the shortcomings of coarse-grained control.

[0044] In another example, to balance performance and resource consumption, a hybrid layout strategy can be adopted based on characteristics such as the amount of expert weight data, access frequency, and whether there is a co-activation relationship. Specifically, the target layout granularity can be determined from multiple layout granularities based on at least one of the following: the amount of weight data of any expert, access frequency, and whether there is a co-activation relationship. These multiple layout granularities can include storage page-level aligned layout and storage block-level aligned layout. Then, the target physical address of any expert can be determined based on the expert's identifier, co-activation relationship, and alignment constraints corresponding to the target layout granularity. For example, for expert combinations with small weight data and co-activation relationships, a fine-grained storage page-level aligned layout is used to pursue the highest continuous read efficiency; for experts with large weight data but low access frequency, a coarse-grained storage block-level aligned layout is used to reduce the number of mapping table entries and management overhead; for rarely activated "cold" experts, no physical layout optimization is performed, and standard FTL storage is used directly to save the management resources of the physical layout engine. The hybrid layout strategy is applicable to ultra-large-scale MOE models with a very large number of experts and highly unbalanced access patterns.

[0045] Having clarified the four physical layout principles, alignment constraints, and alternative layout methods mentioned above, the following details the specific method by which the physical layout engine determines the target physical address: The physical layout engine first queries the preset mapping table to determine whether the expert already has a storage location. The preset mapping table records the correspondence between expert identifiers and physical addresses. The preset mapping table differs from the logical address mapping mentioned above; they are two independent mapping mechanisms, but both are associated through physical addresses and jointly serve the storage and retrieval of expert weights. If the expert already has a storage location—for example, if the expert's weight has been previously written to the SSD and this is an update or overwrite of the weight—the existing location is adjusted based on co-activation relationships to ensure that the expert's physical location remains adjacent to the physical locations of other experts with co-activation relationships. If the expert does not yet have a storage location—for example, if this is the first time the expert is written—the physical layout engine first determines whether other experts with co-activation relationships already exist. If an expert's weight exists, it will be allocated to a free area near existing co-activated expert weights and adjacent to their physical addresses. If no expert's weight exists, a storage block with sufficient free pages will be selected as the starting location based on the storage block's availability. During physical address allocation, the physical layout engine can also consider the storage block's availability, including the current number of used pages, the number of free pages, the number of erase / write cycles, and whether it is a dedicated storage area for expert weights. The physical layout engine prioritizes selecting available storage blocks within the dedicated storage area for expert weights to ensure physical isolation between expert weights and regular user data.

[0046] In the above layout scheme, the expert weight storage area is a dedicated region within the NAND physical space for storing expert weights. The expert weights in this area can be neatly arranged according to four physical arrangement principles. The regular data storage area stores regular user data that does not contain expert weights, managed using standard FTL mapping, and is physically isolated from the expert weight storage area.

[0047] In one example, the overall architecture can be as follows: Figure 3 As shown. Figure 3 The architecture diagram shown contains three layers: the upper application layer (Host side), the SSD firmware layer, and the NAND flash memory layer.

[0048] On the host side, this can include an MOE model inference engine and a standard NVMe read driver. The MOE model inference engine is responsible for the inference computation of the MOE model, including gated network computation, expert selection decision-making, expert weight loading, and inference forward propagation. The standard NVMe read driver can utilize the SGL (Scatter Gather List) mechanism to assemble the target memory addresses corresponding to the expert identifier list output by the inference engine into an SGL entry list on the host side, construct a standard NVMe read command, and send it to the SSD. SGL is a standard data transfer mechanism in the NVMe protocol, which allows a single NVMe read command to specify multiple non-contiguous target memory address segments. This example uses this mechanism to complete the data transfer of multiple expert weights in a single NVMe read command, and is fully compatible with the existing NVMe protocol stack.

[0049] The SSD firmware layer can include an expert weight management module, FTL, and NAND read / write drivers. The expert weight management module is a core module added to the SSD firmware in this example, and it can contain three sub-modules: a physical layout engine, a physical layout-based read execution engine, and a preset mapping table. The physical layout engine is responsible for determining the target physical address for expert weights during write operations and implementing four physical layout principles. The read execution engine is responsible for generating optimized NAND physical read plans during read operations, mapping discrete logical requests to continuous physical reads. The preset mapping table records the correspondence between expert identifiers and physical addresses, serving as a core data bridge connecting the write and read phases. FTL is an existing functional layer of the SSD, responsible for standard mapping from logical addresses to physical addresses, recycling mechanisms, and wear leveling. This example extends FTL with dedicated mapping and layout functions for expert weights. The NAND read / write driver is responsible for the low-level read / write operations of the NAND flash memory chip, including basic operations such as memory page programming, memory page reading, and memory block erasure.

[0050] The NAND flash memory layer can include the aforementioned expert weight storage area and regular data storage area. The PCIe (Peripheral Component Interconnect Express) interface connects the host side and the SSD firmware layer, handling the transmission of NVMe commands, data, and completion notifications.

[0051] It should be noted that the storage devices in the embodiments of the present invention may include, but are not limited to: computing solid-state drives, enterprise-grade solid-state drives, and general-purpose flash storage.

[0052] In this embodiment of the invention, after determining the target physical address, the weight of any expert can be written in parallel to the target physical address, starting from the same page in multiple memory dies. The identifier of any expert and the target physical address can be used to update a preset mapping table. The preset mapping table can be used to record the correspondence between expert identifiers and physical addresses.

[0053] At the write execution level, the SSD controller's NAND read / write driver can simultaneously initiate write commands to each memory die based on the target physical address determined by the physical layout engine, writing expert weight data in parallel to the same memory page line across multiple memory dies. For example, if the SSD has four memory dies, each with a memory page size of 16 KB, a single parallel write operation can write 16 KB of data to each of the four memory dies simultaneously, totaling 64 KB. If the expert weight data is 2 MB, approximately 32 parallel write operations are required, sequentially writing 32 memory page lines. These 32 memory page lines can be physically contiguous, meaning they are consecutive memory page lines located within the same memory block, thus achieving good sequential write performance during the write process.

[0054] After the write operation is complete, the physical layout engine records the correspondence between the expert's identifier and the target physical address in a preset mapping table, thus updating the preset mapping table. The physical address information recorded in the preset mapping table may include detailed information such as the starting storage block address of the expert's weight data, the storage page offset, the data length, the storage channel number, and the storage die number. This information can be queried and used by the read execution engine in the subsequent read phase.

[0055] By using parallel writing, the first principle mentioned above (i.e., striping of the same expert across memory dies and memory pages) can be physically implemented. Since the writing process uses a parallel writing method that starts from the same memory page across multiple memory dies, subsequent readings can utilize multi-plane read commands to read all the weight data of the expert in parallel from all memory dies at once, thereby achieving efficient parallel reading at the NAND physical layer.

[0056] In one example, during the long-term operation of an SSD, due to the aforementioned reclamation mechanism, the physical arrangement of expert weights may gradually degrade. That is, when the proportion of effective storage pages in a certain storage block falls below a certain threshold, resulting in insufficient available storage blocks, the SSD firmware will migrate the effective data in that storage block to a new free storage block, and then erase the original storage block to reclaim the storage space in that block. In addition, the SSD also has a wear leveling mechanism to evenly distribute erase and write operations across all storage blocks, preventing some storage blocks from prematurely failing due to frequent erase and write operations.

[0057] Since expert weight data is typical of "hot-read, cold-write" data in inference scenarios—meaning it is frequently read but rarely updated—it may be relocated during the aforementioned recycling mechanism or wear leveling process. After relocation, its originally neatly arranged physical positions may be disrupted. Furthermore, as the MOE model continues to run, the co-activation patterns of experts may change; for example, after model fine-tuning, the activation distribution of the gating network may change. Therefore, after completing the parallel data writing, a dynamic maintenance mechanism for the physical arrangement can be provided.

[0058] In one example, the dynamic maintenance mechanism may include layout degradation detection and rearrangement. Specifically, the SSD firmware may periodically or upon triggering a preset event detect the physical dispersion of weights for multiple experts. Physical dispersion characterizes the degree to which the weights of the same expert are dispersed across physical addresses and can be measured by one or more of the following metrics: the number of storage blocks spanned by the expert's weight data, the discontinuity of the storage pages spanned (i.e., the number of physical address jumps between adjacent storage pages), and the number of independent NAND read operations required to read all the expert's weight data. Higher physical dispersion indicates that the expert's weight data is more dispersed in physical space, requiring more independent random read operations and resulting in lower read efficiency. When the physical dispersion of an expert exceeds a preset threshold, that expert is identified as needing to have its weights rewritten. The preset threshold can be set empirically, for example, triggering rearrangement when the physical dispersion exceeds twice the initial write state. Then, based on the current co-activation relationship, the physical address of the expert weight to be rewritten is redefined, and the expert weight to be rewritten is written to the redefined physical address, so that the weights of at least two experts with a co-activation relationship are restored to the region of adjacent physical addresses. After rewriting, the physical address entry corresponding to the expert in the preset mapping table is updated synchronously. Since the rewrite operation will generate additional write overhead and storage block erase, the physical layout engine can prioritize performing the resizing during SSD idle periods (e.g., when the inference load is low) to reduce the impact on normal inference services.

[0059] In another example, dynamic maintenance may include reclamation-aware migration. When a preset reclamation mechanism is triggered, the expert weights in the storage block that triggered the mechanism are identified as the expert weights to be migrated, and these expert weights are then migrated. This reclamation mechanism can be used to trigger data migration when there are insufficient valid pages in a storage block. Unlike the conventional reclamation mechanism that simply migrates valid data, the migration process in this example maintains or even optimizes the physical layout during migration: after migration, the weights of at least two experts with co-activation relationships are still stored in a region with adjacent physical addresses, and the physical addresses corresponding to the expert weights to be migrated in the preset mapping table are updated synchronously. Thus, even during unavoidable internal SSD operations such as reclamation mechanisms, the optimized physical layout of expert weights can be maintained.

[0060] In another example, the SSD firmware also performs self-learning of co-activation relationships during runtime. Specifically, it can acquire multiple weighted read requests for multiple experts and record the expert identifier set involved in each read request. Through statistical analysis of a large number of read requests, it counts the frequency of multiple experts being requested simultaneously in multiple read requests, identifies expert combinations that frequently appear in the same read request, and updates the co-activation relationship based on this frequency. For example, if expert A and expert B appear simultaneously in more than 800 of the past 1000 read requests, it can be determined that expert A and expert B have a strong co-activation relationship. As runtime progresses, the self-learned co-activation relationship increasingly accurately reflects the actual access patterns of the model, further optimizing the physical layout. This self-learning mechanism allows the SSD to establish and optimize co-activation relationships even without upper-layer co-activation information. In cloud inference scenarios with multi-tenant shared SSDs, different tenants may load different MOE models or different expert combinations of the same model. The self-learning mechanism can automatically adapt to the access patterns of each tenant, reduce IO resource contention between tenants, and improve service quality.

[0061] In this embodiment of the invention, in response to a weight write request for any expert, the expert's identifier and weight are obtained. Based on the expert's identifier and co-activation relationship, the target physical address of the expert in the solid-state drive (SSD) is determined. The co-activation relationship is determined based on at least two experts from multiple experts in a hybrid expert model stored on the SSD who are simultaneously activated in the same inference iteration; the weights of these at least two experts with a co-activation relationship are stored in a region with adjacent physical addresses. Then, the expert's weight is written in parallel to the target physical address, starting from the same page in multiple parallel-accessible storage dies of the SSD. The expert's identifier and target physical address are used to update a preset mapping table, which records the correspondence between expert identifiers and physical addresses.

[0062] In this embodiment of the invention, when writing expert weights, the target physical address is determined based on the co-activation relationship, ensuring that the weights of at least two experts that need to be read simultaneously in the same inference are stored in regions with adjacent physical addresses. Simultaneously, parallel writing is performed starting from the same page in multiple storage dies, and a preset mapping table is updated to record the correspondence between expert identifiers and physical addresses. This changes the passive approach of prior art, which only allocates physical addresses according to the writing order without considering the correlation between experts. It ensures that the weights of co-activated experts are arranged in an ordered manner in the physical storage space rather than being scattered, effectively overcoming the technical defect of requiring a large number of discrete random read operations when reading co-activated expert weights, and effectively improving data reading speed.

[0063] Please see Figure 4 , Figure 4 This is one of the flowcharts illustrating a data reading method provided in an embodiment of the present invention. It is applied to a solid-state drive (SSD), which stores the weights of multiple experts in a hybrid expert model and includes multiple storage dies that can be accessed in parallel, such as... Figure 4 As shown, the method includes the following steps: Step 402: Based on the obtained weight reading request, determine the identifiers of the multiple target experts whose weights are to be read.

[0064] Step 404: Obtain multiple target physical addresses of multiple target experts based on the preset mapping table and the identifiers of multiple target experts; wherein, the preset mapping table is used to record the correspondence between expert identifiers and physical addresses.

[0065] Step 406: Arrange multiple target physical addresses according to the physical address order, and merge adjacent or overlapping target physical addresses after arrangement to obtain a reading plan; wherein, the merged target physical addresses store the weights of at least two experts with co-activation relationship; co-activation relationship indicates that at least two experts among multiple experts are activated simultaneously in the same inference iteration.

[0066] Step 408: Read the weights from the solid-state drive according to the read plan.

[0067] In this embodiment of the invention, the identifiers of multiple target experts whose weights are to be read can be determined based on the obtained weight reading request.

[0068] The target expert can be an expert in the MoE model. Since the MoE model has been described in detail in the above embodiments, it will not be described again in the embodiments of the present invention.

[0069] In one example, such as Figure 5As shown, in one inference iteration, the gated network of the MoE model can receive input data, calculate the activation scores of all experts, and determine the K experts that need to be activated in this inference based on the activation scores, thus obtaining a list of K expert identifiers. Here, K is usually 2 to 8, which is much smaller than the total number of experts.

[0070] After the host-side MOE model inference engine determines the K expert identifiers, it can pass them to the standard NVMe read driver. The standard NVMe read driver utilizes the SGL mechanism in the NVMe protocol to assemble the target memory addresses corresponding to each of the K expert weights (e.g., the buffer address reserved for that expert weight in GPU memory) into an SGL entry list, constructing a standard NVMe read command. SGL is the standard data transfer mechanism in the NVMe protocol, allowing a single NVMe read command to specify multiple non-contiguous target memory address segments. Each entry describes the start address and length of a host-side memory region. Subsequently, this standard NVMe read command is sent to the SSD controller through the NVMe commit queue, requiring only one command commit and one doorbell write operation.

[0071] The read execution engine on the SSD firmware side can receive the standard NVMe read command and parse the logical address range in the read request. Based on the logical address range, the read execution engine determines the identifiers of multiple target experts whose weights are to be read.

[0072] In one example, the SSD firmware can maintain a predefined mapping between logical address ranges and expert identifiers. The expert weight storage area can be divided into multiple fixed-size logical address segments, each corresponding to one expert, allowing the read execution engine to directly determine the corresponding list of expert identifiers based on the logical address range.

[0073] In another example, the host side can directly include a list of expert identifiers in the command payload using a custom NVMe vendor-specific command. After receiving this vendor-specific command, the SSD firmware's read execution engine can directly parse the identifiers of multiple target experts for the weight to be read from the command payload, without needing to perform a reverse lookup through logical address ranges. This method eliminates the step of looking up expert identifiers from logical addresses, further simplifying the internal processing flow of the SSD.

[0074] Therefore, regardless of whether the read request is issued to the SSD in the form of a standard NVMe read command (carrying a logical address range) or a custom vendor-specific command (directly carrying a list of expert identifiers), the SSD firmware can identify the identifiers of multiple target experts to determine the weight to be read, providing input for subsequent processing.

[0075] Through the aforementioned SGL mechanism, read requests from K experts can be sent with only a single command submission on the host side. Compared to the traditional approach where K experts require K independent NVMe read commands, resulting in K times the command overhead (approximately K times 3 to 5 microseconds), this example reduces the command overhead to 1 / K. Taking K equal to 8 as an example, the command overhead decreases from approximately 24 to 40 microseconds to approximately 3 to 5 microseconds, a reduction of approximately 87.5%. This optimization utilizes the existing standard SGL mechanism of NVMe, requiring no custom commands or dedicated drivers, and is fully compatible with the existing NVMe protocol, resulting in low software deployment costs. In edge MOE model inference scenarios, where SSD capacity and performance are limited, the reduction in command overhead allows for more efficient utilization of limited IO resources.

[0076] In one example, to achieve batch reads without modifying the SSD firmware, batch reads can be implemented at the file system layer. The host provides a dedicated file system interface that receives a list of expert identifiers. The file system merges read requests from multiple experts into a batch submission of multiple standard NVMe read commands, reducing the command interaction overhead between the host and the SSD. No physical layout optimization is performed on the SSD side; standard FTL is still used for address mapping. This example is simple to implement and easy to deploy, but it cannot solve the problem of NAND physical read efficiency within the SSD.

[0077] In this embodiment of the invention, after determining the identifiers of multiple target experts, multiple target physical addresses of the multiple target experts can be obtained according to a preset mapping table and the identifiers of the multiple target experts. The preset mapping table can be used to record the correspondence between expert identifiers and physical addresses.

[0078] Unlike the logical address to physical address mapping table in the standard FTL of SSD firmware, the default mapping table is a newly added dedicated mapping table. The default mapping table uses expert identifiers as indexes, and each entry records the complete physical location information of an expert. In one example, each entry in the default mapping table may contain the following fields: Expert Identifier field, used to uniquely identify an expert; Storage Block Address field, used to record the NAND storage block address from which the expert's weight data begins; Storage Page Offset field, used to record the storage page offset within the starting storage block; Data Length field, used to record the total number of bytes of the expert's weight data; Storage Page Count field, used to record the total number of storage pages occupied by the expert's weight data; Storage Channel Number field, used to record the NAND channel number storing the expert; Storage Die Number field, used to record the storage die number storing the expert; and Co-Activation Group Identifier field, used to record the co-activation group number to which the expert belongs. Using these fields, the read execution engine can completely obtain the accurate location of each expert in the NAND physical space.

[0079] The default mapping table and the standard FTL's logical address to physical address mapping table can work in parallel, with clearly defined responsibilities. The standard logical address to physical address mapping table maintains the mapping from logical block addresses to physical block addresses for all data, handling regular input / output requests; the default mapping table specifically maintains the mapping from expert identifiers to physical block addresses, handling expert weight read requests. The two tables are linked by physical addresses: the physical addresses recorded in the default mapping table and the corresponding physical addresses within the logical address range in the logical address to physical address mapping table point to the same NAND physical location. During the write phase, after the expert weights are written to the NAND flash memory, the physical layout engine creates or updates the entry for that expert in the default mapping table, and the standard FTL also records the corresponding logical address to physical address mapping in its mapping table. During the read phase, the read execution engine queries the default mapping table to retrieve the physical location information of each expert one by one. During the recycling and migration phase, after the expert weight data is migrated, the corresponding physical address entry for that expert in the default mapping table is updated synchronously.

[0080] In one example, a pre-defined mapping table can be stored in random access memory (RAM) while the storage controller is running, allowing for high-speed lookups and updates by the physical layout engine and read execution engine. Furthermore, a backup copy of the pre-defined mapping table can be stored in non-volatile storage media to prevent loss of mapping relationships due to unexpected power failures of the storage controller. During storage controller power-on initialization, the backup copy of the pre-defined mapping table can be loaded from the non-volatile storage media into RAM.

[0081] In one example, see Figure 6 , Figure 6 The logical-physical mapping diagram provided in this example visually presents a discrete list of expert identifiers across three layers: the logical layer, the mapping layer, and the physical layer. It demonstrates how these identifiers can be mapped to a physically sequential or parallel read plan using a pre-defined mapping table. For example... Figure 6As shown, at the logical layer, the expert identifier list (e.g., expert 3, expert 7, expert 1, expert 5) included in the batch read requests issued by the host side is logically discrete, and these expert identifiers are not continuous. At the mapping layer, by querying the preset mapping table, the SSD firmware obtains the NAND physical address corresponding to each expert identifier: expert 1 corresponds to storage block 5, storage page offset 0, 2 MB, channel 0, and co-active group A; expert 3 corresponds to storage block 6, storage page offset 0, 2 MB, channel 0, and co-active group A; expert 5 corresponds to storage block 7, storage page offset 0, 2 MB, channel 1, and co-active group A; expert 7 corresponds to storage block 8, storage page offset 0, 2 MB, channel 1, and co-active group A. At the physical layer, due to the aforementioned physical arrangement optimization (i.e., the second principle described in the above embodiment: physical proximity of co-active experts), expert 1 (storage block 5) and expert 3 (storage block 6) are physically continuous on the same channel CH0, and expert 5 (storage block 7) and expert 7 (storage block 8) are physically continuous on the same channel CH1. The read execution engine generates a read plan accordingly: CH0 performs continuous reads from storage block 5 to storage block 6, and CH1 performs continuous reads from storage block 7 to storage block 8, with CH0 and CH1 executing in parallel. Starting from the originally logically discrete four expert identifiers, two physically continuous parallel read paths are ultimately formed, realizing the mapping from discrete logical requests to continuous physical reads.

[0082] Through a pre-defined mapping table, the upper-layer MOE model inference engine does not need to be aware of the physical details of the NAND flash memory; the SSD automatically completes the optimized mapping. This significantly simplifies the interaction complexity between the upper-layer inference framework and the SSD, reduces the implementation and maintenance costs of the host-side software, and pushes the co-activation semantics of the MOE model down to the SSD firmware layer. The SSD automatically completes the conversion from discrete logical requests to continuous physical reads according to the pre-defined mapping table, providing deterministic performance guarantees for MOE model inference at the storage device level, without relying on the accuracy of the upper-layer prefetch strategy.

[0083] In this embodiment of the invention, after obtaining multiple target physical addresses, the multiple target physical addresses can be arranged according to the physical address order, and adjacent or overlapping target physical addresses after arrangement can be merged to obtain a reading plan. The merged target physical addresses store the weights of at least two experts with a co-activation relationship; a co-activation relationship indicates that at least two experts among the multiple experts are simultaneously activated in the same inference iteration. Since the details of the co-activation relationship have been described in the above embodiments, they will not be repeated here.

[0084] In one example, the step of obtaining the read plan can be executed by the read execution engine, and can specifically include the following three sub-steps: The first sub-step is: sorting by physical address. The read execution engine arranges the K target physical addresses obtained in the previous steps in ascending order of their NAND physical addresses. The physical addresses are sorted using the storage channel number as the primary sort key, the storage block address as the secondary sort key, and the storage page offset as the tertiary sort key. After sorting, expert weights that are in the same storage channel and have adjacent or close physical addresses will be grouped together. Since the physical arrangement engine has already stored experts with co-activation relationships in adjacent physical address regions during the write phase, the physical addresses of these co-activation experts are naturally adjacent or close after sorting. The computational complexity of the sorting operation is O(K log K), which is negligible when K typically does not exceed 8.

[0085] The second sub-step is to merge adjacent or overlapping intervals. Adjacent or overlapping target physical addresses are merged after arrangement. When the intervals of two target physical addresses are physically adjacent (i.e., the end address of the previous interval is continuous with the start address of the next interval) or overlap (i.e., the two intervals have an intersection), they are merged into a larger continuous read interval. A typical example of the merging operation is as follows: If the physical address interval of expert A is from page 0 to page 127 of storage block 5, and the physical address interval of expert B is from page 128 to page 255 of storage block 5, meaning the end address of expert A is continuous with the start address of expert B, then the merged read interval is from page 0 to page 255 of storage block 5, forming a complete continuous read interval. Due to physical layout optimization, co-activated experts are physically stored in adjacent or nearby locations, so a large number of adjacent or overlapping read intervals after physical address arrangement can be merged. In the best case, all weight data of K experts are merged into one or a few continuous read intervals, thus transforming K random reads into one or a few continuous reads. Figure 6 For example, Expert 1 (block 5) and Expert 3 (block 6) are merged into one continuous read interval on channel CH0, and Expert 5 (block 7) and Expert 7 (block 8) are merged into another continuous read interval on channel CH1. The original four expert reads are merged into two continuous read intervals.

[0086] The third sub-step is: multi-channel parallel read scheduling. In one example, when generating the read plan, different parallel read tasks are allocated to the merged target physical addresses belonging to different storage channels, based on the storage channels corresponding to multiple target physical addresses. After sorting and merging, the read execution engine allocates the read intervals on different channels to different parallel read tasks according to the NAND storage channels to which each merged read interval belongs. Each parallel read task is responsible for reading data from its corresponding channel. Because the physical layout optimization distributes co-activation experts as much as possible across different storage channels, this parallel scheduling can fully utilize the multi-channel parallel bandwidth of the SSD. Figure 6 As shown, the two merged consecutive read intervals are located in channel CH0 (blocks 5 to 6) and channel CH1 (blocks 7 to 8), respectively. The read execution engine allocates a parallel read task to channel CH0 and channel CH1 respectively. The two tasks simultaneously send read commands to their respective channels, achieving dual-channel parallel reading. In actual SSDs, the number of channels is usually 4 to 16, and the bandwidth improvement brought by multi-channel parallel scheduling is approximately proportional to the number of channels.

[0087] After the above three sub-steps, the read execution engine can generate the final read plan. The read plan can contain one or more sets of read tasks, each specifying the starting physical address, read length, target storage channel, and parallel execution strategy. Through the synergistic effect of the three sub-steps of sorting, merging, and parallel scheduling, discrete expert-weighted read requests are transformed into an optimized read plan that utilizes NAND sequential read and multi-channel parallel read capabilities, maximizing the efficiency of NAND physical reads.

[0088] In this embodiment of the invention, after obtaining the read plan, the weight can be read from the SSD according to the read plan.

[0089] After receiving the read plan, the SSD controller can initiate physical read commands to each NAND channel and memory die according to the read plan. For portions that are merged into consecutive read intervals, the SSD controller can perform consecutive NAND read operations, utilizing the sequential read characteristics of NAND flash memory and multiplexing row buffers to achieve a high throughput of 3 to 5 GB / s. For different memory channels allocated with parallel read tasks, the SSD controller can initiate read commands simultaneously to achieve multi-channel parallel reading.

[0090] After the read operation is complete, the read expert weight data can be transferred to the host-side memory via the PCIe interface. The SSD controller, according to the SGL in the standard NVMe read command, writes the weight data for each expert to the corresponding target memory address via direct memory access. Once all expert weight data has been transferred, the SSD sends a completion notification to the host side via the NVMe completion queue, informing them that all K expert weight data have been loaded. Upon receiving the completion notification, the host-side MOE model inference engine's GPU can then retrieve the expert weight data from the host memory and execute MOE model inference calculations.

[0091] In one example, when reading weights from the SSD according to the read plan, multiple non-contiguous target memory address ranges can be specified using the SGL field in the standard NVMe read command. Each target memory address range indicates the target storage location of an expert weight to be read in host memory. Then, the expert weights to be read are read from the SSD's NAND flash memory according to the read plan, and the read expert weights are transferred to their corresponding target memory address ranges according to the SGL field. This example utilizes the existing standard SGL mechanism of the NVMe protocol, eliminating the need for custom commands or dedicated drivers. On the host side, the driver allocates an independent GPU memory buffer for each expert, fills the physical addresses of each buffer into the SGL entry list, and then constructs a standard NVMe read command to send to the SSD. On the SSD side, after the read execution engine reads the expert weight data from the NAND flash memory according to the read plan, it transfers each expert weight to its corresponding host memory address sequentially according to the SGL order. By combining the physical layout optimizations established during the aforementioned write phase, a collaborative optimization is achieved where the upper-layer SGL addresses the discontinuity issue on the target side (Host memory or GPU memory), and the SSD physical layout addresses the discontinuity issue on the source side (NAND flash memory). Unlike existing solutions that can only solve the discontinuity issue on the target side through SGL, while the physical dispersion issue of NAND on the source side still exists, requiring random reads to be performed within the SSD, this example fundamentally solves the source-side problem through physical layout, making the upper-layer SGL and the lower-layer physical layout complementary.

[0092] See Figure 7 , Figure 7 This is a comparison chart of traditional reading and optimized reading provided for embodiments of the present invention. Figure 7 The left side illustrates the traditional read-by-read scheme: the host constructs an independent NVMe read command for each expert, requiring K command submissions and K independent random NAND reads for K experts. Due to the random physical locations of the experts, the NAND read throughput is only about 0.5 to 1 GB / s, and the end-to-end latency is K times the command overhead plus the time for K random reads. Figure 7The right side shows the optimized read scheme of this invention: the host side only needs one standard NVMe read command with SGL and one command submission; after the SSD firmware queries the preset mapping table, it generates a continuous NAND physical read plan and performs continuous or parallel NAND reads, with a throughput close to 3 to 5 GB / s of sequential read performance; the end-to-end latency is greatly reduced and the command overhead is reduced to one-Kth of the original.

[0093] In one example, the performance comparison between this example and the traditional solution in a typical MOE model inference scenario is as follows: Taking the activation of 8 experts (K equals 8) and each expert's weight of 2 MB (totaling 16 MB) as an example. Regarding the number of input / output commands, the traditional solution requires 8 independent NVMe read commands, while this example, utilizing the SGL mechanism, requires only 1 command, reducing the number of input / output commands to one-eighth. Regarding host-side command overhead, the traditional solution accumulates approximately 24 to 40 microseconds (8 commands multiplied by 3 to 5 microseconds each), while this example reduces it to approximately 3 to 5 microseconds, a reduction of approximately 5 to 8 times. Regarding NAND read throughput, the traditional solution is limited by physical distribution, with a throughput of approximately 0.5 to 1 GB / s; this example, through physical layout optimization to achieve continuous or parallel reads, increases the throughput to 3 to 5 GB / s, an improvement of 3 to 5 times. Regarding end-to-end expert loading latency, the traditional solution is approximately 100 microseconds, while this example reduces it to approximately 20 to 30 microseconds, a reduction of approximately 3 to 5 times. This example optimizes both command interaction overhead and NAND physical read efficiency, and has significant practical application value for real-time inference scenarios that are sensitive to inference latency (such as cloud-based MOE model multi-tenant inference services) and edge-based MOE model deployment scenarios.

[0094] In one example, to further reduce PCIe data transfer volume, a computing unit can be integrated into the SSD controller to directly perform expert weight loading and partial inference computation within the SSD. The SSD not only utilizes the physical layout proposed in this example to optimize the aggregation and reading of expert weights, but also directly executes the expert's feedforward network computation through the integrated AI computing unit. Only the computation results, rather than the complete weight data, are transmitted to the GPU via the PCIe interface, which is particularly advantageous for bandwidth-constrained edge computing scenarios.

[0095] In another example, to eliminate the overhead of explicit input / output commands, a persistent memory mapping can be established between the SSD and the GPU. The expert weights, physically stored in the SSD, are mapped to the GPU's address space. The GPU directly reads the expert weights via memory access instructions and loads them into the GPU cache as needed, resulting in extremely low software stack overhead. In the checkpoint storage and recovery scenario during MOE model training, the physical arrangement optimization in this example maintains a neat arrangement during writes, enabling efficient saving and recovery of checkpoints.

[0096] In this embodiment of the invention, based on the acquired weight read request, the identifiers of multiple target experts whose weights are to be read are determined, and multiple target physical addresses of the multiple target experts are obtained according to a preset mapping table and the identifiers of the multiple target experts. The preset mapping table records the correspondence between expert identifiers and physical addresses. Then, according to the physical address order, the multiple target physical addresses are arranged, and adjacent or overlapping target physical addresses are merged to obtain a read plan. The merged target physical addresses store the weights of at least two experts with a co-activation relationship, where co-activation indicates that at least two of the multiple experts are simultaneously activated in the same inference iteration. According to this read plan, the weights are read from the solid-state drive.

[0097] In this embodiment of the invention, when reading expert weights, after determining the target expert identifier based on the obtained weight read request, the corresponding target physical address is obtained through a preset mapping table. Adjacent or overlapping physical addresses are then merged according to their physical address order to generate a read plan. Since the weights of at least two experts with co-activation relationships are stored in adjacent physical address regions based on co-activation relationships during the write phase, the merged read plan corresponds to a continuous physical interval. This changes the passive approach in the prior art where expert weights are physically dispersed, resulting in discrete random reads for each operation. It transforms multiple logically discrete read requests into a small number of continuous physical read operations, effectively overcoming the technical defect of requiring a large number of discrete random read operations when reading co-activated expert weights, and significantly improving data read speed.

[0098] Figure 8 The data writing device 800 shown can achieve Figure 1 The method described in the embodiment achieves the same technical effect, and can be specifically referred to in the above description. Figure 1 The data writing method of the illustrated embodiment will not be described in detail here. The data writing device 800 is applied to a solid-state drive (SSD), which is used to store the weights of multiple experts in a hybrid expert model; the SSD includes multiple storage dies that can be accessed in parallel, and the data writing device 800 includes: Processing module 801 is used to obtain the identifier and weight of any expert in response to a weight write request for any expert. The determination module 802 is used to determine the target physical address of any expert in the solid-state drive based on the identifier and co-activation relationship of any expert; wherein, the co-activation relationship is determined based on at least two experts among the plurality of experts who are simultaneously activated in the same inference iteration; the weights of the at least two experts with the co-activation relationship are stored in a region where the physical addresses are adjacent. The writing module 803 is used to write the weight of any expert in parallel to the target physical address, starting from the same page in the plurality of storage dies; wherein, the identifier of any expert and the target physical address are used to update a preset mapping table; the preset mapping table is used to record the correspondence between expert identifiers and physical addresses.

[0099] Optionally, the data writing device 800 may further include other functional modules for implementing any of the method embodiments described above, such as a module for detecting physical dispersion and rewriting, a module for performing perceptual migration when the recycling mechanism is triggered, a module for updating co-activation relationships based on read frequency, and a module for determining the target physical address based on mapping unit alignment constraints, partitioning mechanisms, recycling region identifiers, or hybrid arrangement strategies. The specific processing flow of each of the above functional modules has been described in [the original text]. Figure 1 The method embodiments shown are described in detail and will not be repeated here.

[0100] The functional modules in the embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0101] It should be noted that if the modules in the embodiments of this application are implemented in the form of software functional units, they can be stored in a computer-readable storage medium and executed by a processor to achieve the corresponding functions. The specific processing flow executed by each module in this virtual device has been described in detail in the foregoing method embodiments. This device embodiment only describes the logical connection relationship between modules. For the specific program execution flow, please refer to the foregoing method embodiments.

[0102] Figure 9 The data reading device 900 shown can achieve Figure 4 The method described in the embodiment achieves the same technical effect, and can be specifically referred to in the above description. Figure 4 The data reading method of the illustrated embodiment will not be described in detail here. The data reading device 900 is applied to a solid-state drive (SSD), which is used to store the weights of multiple experts in a hybrid expert model; the SSD includes multiple storage dies that can be accessed in parallel, and the data reading device 900 includes: Analysis module 901 is used to determine the identifiers of multiple target experts whose weights are to be read based on the acquired weight read request; The acquisition module 902 is used to acquire multiple target physical addresses of the multiple target experts according to a preset mapping table and the identifiers of the multiple target experts; wherein, the preset mapping table is used to record the correspondence between expert identifiers and physical addresses; The generation module 903 is used to arrange the plurality of target physical addresses according to the physical address order, and merge adjacent or overlapping target physical addresses after arrangement to obtain a reading plan; wherein, the merged target physical addresses store the weights of at least two experts with co-activation relationship; the co-activation relationship indicates that at least two experts among the plurality of experts are simultaneously activated in the same inference iteration; The read module 904 is used to read weights from the solid-state drive according to the read plan.

[0103] Optionally, the data reading device 900 may further include other functional modules for implementing any of the method embodiments described above, such as a module for multi-channel parallel read scheduling, and a module for specifying multiple non-contiguous target memory address segments through the SGL field in the standard NVMe read command. The specific processing flow of each of the above functional modules has been described in [the document / document / etc.]. Figure 4 The method embodiments shown are described in detail and will not be repeated here.

[0104] The functional modules in the embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0105] It should be noted that if the modules in the embodiments of this application are implemented in the form of software functional units, they can be stored in a computer-readable storage medium and executed by a processor to achieve the corresponding functions. The specific processing flow executed by each module in this virtual device has been described in detail in the foregoing method embodiments. This device embodiment only describes the logical connection relationship between modules. For the specific program execution flow, please refer to the foregoing method embodiments.

[0106] Figure 10 This is a schematic diagram of the structure of a storage device provided in one embodiment of the present invention. Please refer to it. Figure 10 At the hardware level, the storage device includes a storage controller and non-volatile storage media. In one software implementation, the storage controller includes a processor and optionally an internal bus, network interface, and memory. The memory may include main memory, such as high-speed random-access memory (RAM), or non-volatile memory, such as at least one disk drive. Of course, the storage device may also include other hardware required for other business operations.

[0107] The processor, network interface, and memory can be interconnected via an internal bus, which can be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus, or an EISA (Extended Industry Standard Architecture) bus, etc. This bus can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 10 The symbol is represented by a single double-headed arrow, but this does not mean that there is only one bus or one type of bus.

[0108] Memory is used to store programs. Specifically, programs may include program code, which includes computer operation instructions. Memory may include main memory and non-volatile memory, and provides instructions and data to the processor.

[0109] The processor reads the corresponding computer program from non-volatile memory into main memory and then executes it, forming a data writing device at the logical level. This data writing device operates in the memory controller, performing the physical arrangement and writing of expert weights on the non-volatile storage medium. The processor executes the program stored in memory and specifically performs the following operations: In response to a weight write request for any expert, obtain the identifier and weight of that expert. Based on the identifier and co-activation relationship of any expert, the target physical address of any expert in the solid-state drive is determined; wherein, the co-activation relationship is determined based on at least two experts among the plurality of experts that are simultaneously activated in the same inference iteration; the weights of the at least two experts with the co-activation relationship are stored in a region where the physical addresses are adjacent. The weight of any expert is written in parallel to the target physical address, starting from the same page in multiple storage dies; wherein, the identifier of any expert and the target physical address are used to update a preset mapping table; the preset mapping table is used to record the correspondence between expert identifiers and physical addresses.

[0110] The processor reads the corresponding computer program from non-volatile memory into main memory and then executes it, forming a data read / write device at the logical level. This data read / write device operates in the memory controller, performing aggregated readings of expert weights on the non-volatile storage medium. The processor executes the program stored in memory and specifically performs the following operations: Based on the obtained weight read request, determine the identifiers of multiple target experts whose weights are to be read; Based on a preset mapping table and the identifiers of the multiple target experts, multiple target physical addresses of the multiple target experts are obtained; wherein, the preset mapping table is used to record the correspondence between expert identifiers and physical addresses; According to the physical address order, the multiple target physical addresses are arranged, and adjacent or overlapping target physical addresses after arrangement are merged to obtain a reading plan; wherein, the merged target physical addresses store the weights of at least two experts with co-activation relationship; the co-activation relationship indicates that at least two experts among the multiple experts are simultaneously activated in the same inference iteration; According to the read plan, the weights are read from the solid-state drive.

[0111] The above is as described in the present invention. Figure 1 or Figure 4 The methods disclosed in the illustrated embodiments can be applied to or implemented by a processor. The processor may be an integrated circuit chip with signal processing capabilities. During implementation, each step of the above methods can be completed by integrated logic circuits in the processor's hardware or by instructions in software form. The processor can be a general-purpose processor, including a Central Processing Unit (CPU), a Network Processor (NP), etc.; it can also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. The methods, steps, and logic block diagrams disclosed in one or more embodiments of this invention can be implemented or executed. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the methods disclosed in one or more embodiments of this invention can be directly embodied in the execution of a hardware decoding processor, or can be executed by a combination of hardware and software modules in the decoding processor. The software module can reside in a mature storage medium in the field, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, or registers. This storage medium is located in memory, and the processor reads information from the memory and, in conjunction with its hardware, completes the steps of the above method.

[0112] This storage device can also perform... Figure 1 or Figure 4 The method of the illustrated embodiment will not be described in detail here.

[0113] This invention also provides a storage controller configured to implement Figure 1 or Figure 4 The method of the illustrated embodiment. Specifically, the storage controller includes at least a processor, firmware memory, a host interface, and a flash memory interface. The firmware memory stores a physical layout engine and a read execution engine. The physical layout engine determines the target physical address of expert weights in the non-volatile storage medium based on expert identifiers and co-activation relationships. The read execution engine maps discrete expert weight read requests to continuous physical read plans according to a preset mapping table. The preset mapping table records the correspondence between expert identifiers and physical addresses. The host interface receives weight write requests and weight read requests. The flash memory interface issues read and write commands to the non-volatile storage medium. Both the physical layout engine and the read execution engine are software modules stored in the firmware memory and executed by the processor. The specific operation of this storage controller can be referred to the above description. Figure 1 or Figure 4 The method described in the illustrated embodiment will not be repeated here.

[0114] This invention also provides a computer-readable storage medium that stores one or more programs, the programs including instructions that, when executed by a portable electronic device including multiple applications, enable the portable electronic device to perform the steps of any one of the above-described data writing method embodiments or data reading method embodiments, which will not be elaborated further here.

[0115] This invention also provides a computer program product, which is stored in a storage medium and executed by at least one processor to implement the steps of any one of the above-described data writing method embodiments or data reading method embodiments, which will not be elaborated further here.

[0116] Of course, in addition to the software implementation, the storage device of the present invention does not exclude other implementation methods, such as logic devices or a combination of hardware and software, etc. That is to say, the execution subject of the following processing flow is not limited to each logic unit, but can also be hardware or logic devices.

[0117] In summary, the above description is merely a preferred embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of one or more embodiments of the present invention should be included within the scope of protection of one or more embodiments of the present invention.

[0118] The systems, devices, modules, or units described in the above embodiments can be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a storage device. Specifically, storage devices include, but are not limited to, cSSD (Client Solid State Drive), eSSD (Enterprise Solid State Drive), UFS (Universal Flash Storage), solid-state drives, embedded storage devices, memory cards, or any combination of these devices.

[0119] Computer-readable media, including both permanent and non-permanent, removable and non-removable media, can store information using any method or technology. Information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined in this embodiment of the invention, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0120] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0121] The various embodiments in this invention are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the system embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments.

Claims

1. A data writing method, characterized in that, It is applied to a solid-state drive, which is used to store the weights of multiple experts in a hybrid expert model; The solid-state drive includes multiple storage dies that can be accessed in parallel, and the method includes: In response to a weight write request for any expert, obtain the identifier and weight of that expert. Based on the identifier and co-activation relationship of any expert, the target physical address of any expert in the solid-state drive is determined; wherein, the co-activation relationship is determined based on at least two experts among the plurality of experts who are simultaneously activated in the same inference iteration; the weights of the at least two experts with the co-activation relationship are stored in a region where the physical addresses are adjacent. The weight of any expert is written in parallel to the target physical address, starting from the same page in the plurality of storage dies; wherein, the identifier of any expert and the target physical address are used to update a preset mapping table; the preset mapping table is used to record the correspondence between expert identifiers and physical addresses; Determining the target physical address of any expert in the solid-state drive based on the expert's identifier and co-activation relationship includes: Based on at least one of the following factors—the amount of weighted data, access frequency, and whether a co-activation relationship exists for any expert—a target layout granularity is determined from multiple layout granularities. These multiple layout granularities include storage page-level aligned layout and storage block-level aligned layout. For expert combinations with small weighted data and co-activation relationships, storage page-level aligned layout is used. For experts with large weighted data but low access frequency, storage block-level aligned layout is used. For rarely activated "cold" experts, standard flash translation layer (FTL) storage is used.

2. The method according to claim 1, characterized in that, After writing the weight of any expert to the target physical address in parallel, starting from the same page in the plurality of memory dies, the method further includes at least one of the following: The physical dispersion of the weights of the multiple experts is detected. Experts whose physical dispersion exceeds a preset threshold are identified as experts whose weights need to be rewritten. Based on the co-activation relationship, the physical address of the expert weights to be rewritten is re-determined, and the expert weights to be rewritten are written to the re-determined physical address, so that the weights of at least two experts with a co-activation relationship are stored in a region with adjacent physical addresses; wherein, the physical dispersion is used to characterize the degree of dispersion of the same expert weights on the physical address. When the preset reclamation mechanism is triggered, the expert weights in the storage block that triggered the preset reclamation mechanism are determined as expert weights to be migrated, and the expert weights to be migrated are migrated; wherein, the reclamation mechanism is used to trigger data migration when there are insufficient valid pages in the storage block; after migration, the weights of at least two experts with co-activation relationship are still stored in the region of adjacent physical addresses, and the physical addresses corresponding to the expert weights to be migrated in the preset mapping table are updated synchronously. Obtain multiple weighted read requests for the multiple experts, count the frequency of the multiple experts being requested simultaneously in the multiple read requests, and update the co-activation relationship based on the frequency.

3. The method according to claim 1, characterized in that, Determining the target physical address of any expert in the solid-state drive based on the expert's identifier and co-activation relationship includes: Based on the identifier of any expert, the co-activation relationship, the storage block alignment constraint, and the storage page alignment constraint, the target physical address of any expert in the solid-state drive is determined; wherein, the storage block alignment constraint is used to limit the starting physical address of the expert weight to be aligned to the boundary of the storage block; the storage page alignment constraint is used to limit the starting physical address of the expert weight to be aligned to the boundary of the storage page, and the end of the expert weight is aligned to the boundary of the storage page through data padding.

4. The method according to claim 1, characterized in that, Determining the target physical address of any expert in the solid-state drive based on the expert's identifier and co-activation relationship includes: Based on the identifier of any expert, the co-activation relationship, and the mapping unit alignment constraint, determine the target physical address of any expert in the solid-state drive; The mapping unit alignment constraint is used to limit the starting physical address of the expert weight to be aligned to the boundary of the mapping unit, and the expert weight is stored in units of an integer number of mapping units; the mapping unit is the preset minimum management granularity of the logical address to physical address mapping table maintained by the flash translation layer of the solid-state drive; the granularity of the mapping unit is larger than the granularity of the storage page, so that the number of mapping entries required to record a single expert in the preset mapping table is less than the number of mapping entries required when managing at the storage page granularity.

5. The method according to claim 1, characterized in that, Determining the target physical address of any expert in the solid-state drive based on the expert's identifier and co-activation relationship includes: Based on the identifier of any expert and the co-activation relationship, the target partition corresponding to any expert is determined; wherein, the storage space of the solid-state drive is divided into multiple partitions, and at least two experts with the co-activation relationship are assigned to the same partition; The current write position within the target partition is determined as the target physical address of any expert in the solid-state drive; wherein, sequential writing is used within the target partition so that the weights of experts written to the target partition sequentially are adjacent in physical storage location.

6. The method according to claim 1, characterized in that, Determining the target physical address of any expert in the solid-state drive based on the expert's identifier and co-activation relationship includes: Based on the identifier of any expert and the co-activation relationship, the target recovery region identifier corresponding to any expert is determined; wherein, at least two experts with the co-activation relationship are assigned the same recovery region identifier. The write location within the recycling area corresponding to the target recycling area identifier is determined as the target physical address of any expert in the solid-state drive, so that the weights of at least two experts with the co-activation relationship are physically assigned to the same recycling area through the recycling area identifier.

7. The method according to claim 1, characterized in that, Determining the target physical address of any expert in the solid-state drive based on the expert's identifier and co-activation relationship includes: Based on the identifier of any expert, the co-activation relationship, and the alignment constraint corresponding to the target arrangement granularity, the target physical address of any expert in the solid-state drive is determined; wherein, when the target arrangement granularity is the storage page-level aligned arrangement, the starting physical address of the expert weight is aligned to the storage page boundary; when the target arrangement granularity is the storage block-level aligned arrangement, the starting physical address of the expert weight is aligned to the storage block boundary.

8. A data reading method, characterized in that, It is applied to a solid-state drive, which is used to store the weights of multiple experts in a hybrid expert model; The solid-state drive includes multiple storage dies that can be accessed in parallel; the method includes: Based on the obtained weight read request, determine the identifiers of multiple target experts whose weights are to be read; Based on a preset mapping table and the identifiers of the multiple target experts, multiple target physical addresses of the multiple target experts are obtained; wherein, the preset mapping table is used to record the correspondence between expert identifiers and physical addresses; the multiple target physical addresses are determined according to the method described in claim 1; According to the physical address order, the multiple target physical addresses are arranged, and adjacent or overlapping target physical addresses after arrangement are merged to obtain a reading plan; wherein, the merged target physical addresses store the weights of at least two experts with co-activation relationship; the co-activation relationship indicates that at least two experts among the multiple experts are simultaneously activated in the same inference iteration; Weights are read from the solid-state drive according to the read plan.

9. The method according to claim 8, characterized in that, The step of arranging the multiple target physical addresses according to their physical address order, and merging adjacent or overlapping target physical addresses after the arrangement to obtain a reading plan, includes: Based on the storage channels corresponding to the multiple target physical addresses, different parallel read tasks are allocated to the merged target physical addresses belonging to different storage channels to obtain the read plan.

10. The method according to claim 8, characterized in that, The step of reading weights from the solid-state drive according to the read plan includes: The Scattered Aggregation List (SGL) field in the NVMe read command, as specified in the standard non-volatile memory host controller interface specification, specifies multiple non-contiguous target memory address segments; each target memory address segment indicates the storage location of an expert weight to be read. Based on the read plan, the expert weights to be read are read from the solid-state drive, and according to the SGL field, the read expert weights are stored in the corresponding target memory address segments.

11. A data writing device, characterized in that, It is applied to a solid-state drive, which is used to store the weights of multiple experts in a hybrid expert model; The solid-state drive includes multiple storage dies that can be accessed in parallel, and the device includes: The processing module is used to obtain the identifier and weight of any expert in response to a weight write request for any expert. A determination module is used to determine the target physical address of any expert in the solid-state drive based on the identifier and co-activation relationship of any expert; wherein, the co-activation relationship is determined based on at least two experts that are simultaneously activated in the same inference iteration among the plurality of experts; the weights of the at least two experts with the co-activation relationship are stored in a region where the physical addresses are adjacent. The writing module is used to write the weight of any expert to the target physical address in parallel, starting from the same page in the plurality of storage dies; wherein, the identifier of any expert and the target physical address are used to update a preset mapping table; the preset mapping table is used to record the correspondence between expert identifiers and physical addresses; The determining module is used to: determine a target layout granularity from multiple layout granularities based on at least one of the following: the amount of weighted data, the access frequency, and whether there is a co-activation relationship among any expert; wherein, the multiple layout granularities include storage page-level aligned layout and storage block-level aligned layout; for expert combinations with small weighted data and co-activation relationships, storage page-level aligned layout is used; for experts with large weighted data but low access frequency, storage block-level aligned layout is used; for cold experts that are rarely activated, standard flash translation layer (FTL) storage is used.

12. A data reading device, characterized in that, It is applied to a solid-state drive, which is used to store the weights of multiple experts in a hybrid expert model; The solid-state drive includes multiple storage dies that can be accessed in parallel; the device includes: The analysis module is used to determine the identifiers of multiple target experts whose weights need to be read based on the obtained weight read requests; The acquisition module is used to acquire multiple target physical addresses of the multiple target experts according to a preset mapping table and the identifiers of the multiple target experts; wherein, the preset mapping table is used to record the correspondence between expert identifiers and physical addresses; the multiple target physical addresses are determined according to the method of claim 1; A generation module is used to arrange the plurality of target physical addresses according to the physical address order, and merge adjacent or overlapping target physical addresses after arrangement to obtain a reading plan; wherein, the merged target physical addresses store the weights of at least two experts with a co-activation relationship; the co-activation relationship indicates that at least two experts among the plurality of experts are simultaneously activated in the same inference iteration; A read module is used to read weights from the solid-state drive according to the read plan.

13. A storage controller, characterized in that, It includes at least a processor, firmware memory, host interface, and flash memory interface; The firmware memory stores a physical layout engine and a read execution engine. The physical layout engine determines the target physical address of expert weights in a non-volatile storage medium based on expert identifiers and co-activation relationships. The co-activation relationship is determined based on at least two experts who are simultaneously activated in the same inference iteration among multiple experts. The read execution engine maps discrete expert weight read requests to continuous physical read plans according to a preset mapping table. The preset mapping table records the correspondence between expert identifiers and physical addresses. The host interface is used to receive weight write requests and weight read requests; The flash memory interface is used to issue read and write commands to the non-volatile storage medium; The processor, the firmware memory, the host interface, and the flash memory interface are configured to collaboratively implement the steps of the method as described in any one of claims 1 to 7 or 8 to 10.

14. A storage device, characterized in that, Includes the storage controller as described in claim 13, and a non-volatile storage medium.

15. A computing device, characterized in that, It includes the storage device as described in claim 14, and a processor.

16. A computer-readable storage medium, characterized in that, The computer-readable storage medium is used to store computer-executable instructions that, when executed by a processor, perform the steps of the method as described in any one of claims 1 to 7 or 8 to 10.

Citation Information

Patent Citations

  • Data transmission method and device, medium, electronic equipment and program product

    CN120825446A

  • Solid state disk and data access acceleration method thereof

    CN120973698A

  • Large language model reasoning acceleration method and device

    CN122133713A

  • Data management method and device, electronic equipment and medium

    CN122240021A