Data access methods, apparatus, computer equipment, readable storage media and program products

Abstract

This application relates to a data access method, apparatus, computer device, readable storage medium, and program product. By dividing a data sequence to be processed stored in a first storage location in global memory into multiple data subsequences; reading each data subsequence from global memory sequentially; storing each data subsequence sequentially into a first memory; reading the target data corresponding to each expert model from each data subsequence in the first memory according to the index value of each expert model in the target index table; and storing the target data into a second memory. This improves the efficiency of data access and enhances the performance of the MoE architecture in distributed data processing.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a data access method, apparatus, computer equipment, readable storage medium, and program product. Background Technology

[0002] In high-performance computing and artificial intelligence applications, the hybrid expert model architecture serves as a key distributed data processing framework. It dynamically classifies input data sequences through routing mechanisms and distributes different data operators to multiple expert models for parallel processing.

[0003] However, due to the random and dynamic nature of data allocation, the data sequence to be processed is often stored non-contiguously in global memory. When the expert model performs data processing tasks, it must frequently access data blocks scattered across different locations in global memory, resulting in a large number of random memory access operations. This non-contiguous storage pattern not only significantly increases memory access latency but also consumes additional system bandwidth resources due to frequent data movement and copying operations.

[0004] In large-scale data processing scenarios, low data access efficiency restricts the throughput of hybrid expert model architectures, leading to a decline in overall processing performance and limiting the system's scalability in applications with high real-time requirements. Especially when processing long-sequence data, the fragmentation of data storage further exacerbates the memory access bottleneck, making it impossible to meet the demands of efficient distributed data processing. Summary of the Invention

[0005] Therefore, it is necessary to provide a data access method, apparatus, computer device, readable storage medium, and program product to address the aforementioned technical problems, which can improve the efficiency of data access and enhance the performance of the MoE architecture in distributed data processing.

[0006] Firstly, this application provides a data access method applied to a hybrid expert model architecture, which includes at least two distributed expert models, including:

[0007] The data sequence to be processed, stored in the first storage location in global memory, is divided into multiple data subsequences;

[0008] Read each data subsequence in global memory in a sequential manner;

[0009] Each data subsequence is stored sequentially in the first memory; wherein, the first memory is independent of the global memory and is used to temporarily store the data subsequences;

[0010] Based on the index value of each expert model in the target index table, the target data corresponding to each expert model is read from each data subsequence in the first memory; wherein, the target index table contains the index information of each expert model and the target data to be processed in the data subsequence;

[0011] The target data is stored in a second memory; the second memory corresponds to each expert model, and the target data in the second memory is used to retrieve the target data when each expert model processes the target data.

[0012] In one embodiment, after the step of storing the target data in the second memory according to the index value of each expert model in the target index table, the data access method includes:

[0013] In response to the start signal for data processing of the expert model, the target data corresponding to each expert model is retrieved from the second memory for data processing to obtain multiple corresponding result data; wherein, the multiple result data are stored consecutively in the second storage location of the global memory;

[0014] The result data of each expert model's data processing of the target data is read from the target location in global memory in a continuous reading manner.

[0015] Based on the index information of each expert model and the target data to be processed in the target index table, the result data corresponding to each expert model is written into the first memory.

[0016] Read the result subsequence corresponding to the result data of each expert model from the first memory;

[0017] Store the result subsequence corresponding to the result data of each expert model into the target area of ​​global memory.

[0018] In one embodiment, the step of dividing the data sequence to be processed, stored in a first storage location in global memory, to obtain multiple data sub-sequences includes:

[0019] Based on the hardware cache capacity, determine the granularity of the global memory partition; the granularity of the partition is proportional to the granularity of data access to the global memory, and the granularity of the partition is used to balance the speed and resources of data access in the global memory.

[0020] Using block granularity as the unit of division, the data sequence to be processed, stored in the first storage location in global memory, is divided into multiple data subsequences.

[0021] In one embodiment, after dividing the data sequence to be processed stored in a first storage location in global memory to obtain multiple data subsequences, the data memory access method further includes:

[0022] Obtain the starting memory address of the data subsequence and determine the granularity of data access;

[0023] Based on the starting memory address, determine whether the access data length and access granularity are aligned when accessing a data subsequence.

[0024] If the accessed data length and access granularity are not aligned, then calculate the alignment offset;

[0025] Based on the alignment offset, adjust the starting memory address of the data subsequence to determine the offset read address for reading the data subsequence.

[0026] In one embodiment, the step of adjusting the starting memory address of the data subsequence based on the alignment offset to determine the offset read address for reading the data subsequence includes:

[0027] Calculate the length of invalid data for the data subsequence based on the alignment offset; where the length of invalid data is the length of invalid data generated by offsetting the data subsequence.

[0028] If the sum of the alignment offset and the length of invalid data is less than the access granularity, then the first maximum offset value is determined based on the sum of the alignment offset and the length of invalid data.

[0029] If the length of the invalid data is less than the access granularity, then the second maximum offset value is determined based on the length of the invalid data.

[0030] The alignment offset is adjusted based on the safety offset value. The adjusted alignment offset is subtracted from the starting memory address of the data subsequence to determine the offset read address. The safety offset value includes either a first maximum offset value or a second maximum offset value.

[0031] In one embodiment, prior to the step of reading the target data corresponding to each expert model from each data subsequence in the first memory based on the index value of each expert model in the target index table, the data access method further includes:

[0032] Obtain the original index table generated by the routing decision; wherein the original index table includes at least two expert models, and the relative position index of the data to be processed in the original data sequence for each expert model;

[0033] The relative position index of each expert model in the original index table corresponding to the original data to be processed is converted into an absolute index value; where the absolute index value ranges from 0 to (N*M-1), where N is the number of data subsequences and M is the number of data elements contained in each data subsequence;

[0034] The absolute index values ​​are updated in the original index table to obtain the target index table.

[0035] Secondly, this application also provides a data access device, comprising:

[0036] The partitioning module is used to partition the data sequence to be processed, which is stored in the first storage location in global memory, to obtain multiple data subsequences;

[0037] The first access module is used to read each data subsequence in global memory in a sequential reading manner;

[0038] The first storage module is used to store each data subsequence sequentially into the first memory; wherein the first memory is independent of the global memory and is used to temporarily store the data subsequences.

[0039] The second access module is used to read the target data corresponding to each expert model from each data subsequence in the first memory according to the index value of each expert model in the target index table; wherein, the target index table contains the index information of each expert model and the target data to be processed in the data subsequence;

[0040] The second storage module is used to store the target data in the second memory; wherein the second memory corresponds to each expert model, and the target data in the second memory is used to call when each expert model processes the target data.

[0041] Thirdly, this application also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of the data access method of the first aspect.

[0042] Fourthly, this application also provides a readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of the data access method of the first aspect.

[0043] Fifthly, this application also provides a program product, including a computer program that, when executed by a processor, implements the steps of the data access method of the first aspect.

[0044] The aforementioned data access method, apparatus, computer device, readable storage medium, and program product are applied to a hybrid expert model architecture, which includes at least two distributed expert models. The process involves dividing a data sequence to be processed, stored in a first storage location in global memory, into multiple data sub-sequences. Each data sub-sequence in global memory is read sequentially. Each data sub-sequence is then stored sequentially in a first memory, which is independent of global memory and serves as a temporary storage location for the data sub-sequences. Based on the index value of each expert model in a target index table, the target data corresponding to each expert model is read from each data sub-sequence in the first memory. The target index table contains the index information of each expert model and its target data in the data sub-sequences. The target data is then stored in a second memory, which corresponds to each expert model and is used when each expert model processes the target data. By continuously reading data sequences, storing them in temporary storage, and accessing target data based on indexes, random memory access is reduced, improving the efficiency of data access to memory, reducing memory access latency, optimizing the utilization of system bandwidth resources, and enhancing the processing throughput of the hybrid expert model architecture. Attached Figure Description

[0045] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the drawings used in the description of the embodiments of this application or related technologies will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0046] Figure 1 This is a flowchart illustrating a data access method in one embodiment;

[0047] Figure 2 This is another flowchart illustrating the data access method in one embodiment;

[0048] Figure 3 This is a schematic diagram of a data access method in one embodiment;

[0049] Figure 4 This is a schematic diagram of another scenario for the data access method in one embodiment;

[0050] Figure 5 This is a detailed flowchart of step 11 of the data access method in one embodiment;

[0051] Figure 6 This is a flowchart illustrating an extension scheme for a data access method in one embodiment;

[0052] Figure 7 A flowchart illustrating a detailed scheme of a data access method in one embodiment;

[0053] Figure 8 This is a schematic diagram of another scenario of the data access method in one embodiment;

[0054] Figure 9 This is a flowchart illustrating another extension of the data access method in one embodiment;

[0055] Figure 10 This is a schematic diagram of the data access device in one embodiment;

[0056] Figure 11 This is a schematic diagram of the internal structure of a computer device in one embodiment. Detailed Implementation

[0057] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0058] It should be noted that the terms "first," "second," etc., used in this application can be used to describe various elements, but these elements are not limited by these terms. These terms are only used to distinguish the first element from the second element. The terms "comprising" and "having," and any variations thereof, used in this application, are intended to cover non-exclusive inclusion. The term "multiple" used in this application refers to two or more. The term "and / or" used in this application refers to one of the embodiments, or any combination of multiple embodiments.

[0059] In one embodiment, a data access method is provided. This embodiment illustrates the application of this method to a Mixture of Experts (MoE) architecture, which includes at least two expert models. It is understood that this data access method can also be applied to other distributed data processing architectures, and the controller can be used to enable each distributed endpoint in the distributed architecture to access data in global memory.

[0060] In a hybrid expert model architecture, the non-contiguous storage of the data sequence in global memory forces the expert model to perform numerous random access operations during data access. This non-contiguous storage inefficiency leads to increased memory access latency and decreased bandwidth utilization, thus impacting the overall performance of distributed data processing.

[0061] It should be noted that the MoE architecture is a distributed data processing architecture characterized by breaking down complex tasks into multiple sub-tasks, each handled by a different expert model. Each expert model focuses on processing a specific type of data or task, thereby improving overall processing efficiency.

[0062] Global memory refers to a shared storage area in the storage system corresponding to the MoE architecture that can be directly accessed by all processors or computing units. Global memory usually has a large capacity, but the access latency may be relatively high.

[0063] Based on this, in this embodiment, a data access method is proposed, such as... Figure 1 As shown, the method includes steps 11 to 15, wherein:

[0064] Step 11: Divide the data sequence to be processed stored in the first storage location in global memory to obtain multiple data subsequences.

[0065] The first storage location can be a specific area in global memory, used to store the original data sequence to be processed. The data sequence is the original set of data to be processed, which is usually stored in memory in a continuous or non-contiguous manner. The data subsequence is a smaller data block obtained by dividing the original data sequence according to specific rules.

[0066] One implementation approach is to pre-define a fixed data block size, for example, dividing the original data sequence into subsequences of 1KB or 4KB each.

[0067] As another implementation, the length of each subsequence can be evenly distributed based on the total length of the data sequence and the desired number of subsequences. For example, if a data sequence has 1000 elements and needs to be divided into 10 data subsequences, then each data subsequence contains 100 elements.

[0068] Step 12: Read each data subsequence in global memory in a sequential reading manner.

[0069] Here, sequential read is an access mode for reading data in global memory. It can read data sequentially according to the memory address order in global memory. It can usually utilize hardware caching mechanisms to improve data reading efficiency.

[0070] It should be noted that since each data subsequence in global memory can be a contiguous data block on multiple logical blocks in global memory, when moving multiple data subsequences from the slow but large-capacity global memory to the high-speed but limited-capacity first memory, we can start from a starting address in global memory and read the corresponding multiple data subsequences stored in a contiguous physical address in global memory in one go, without interruption, according to the order of the storage addresses.

[0071] Step 13: Store each data subsequence sequentially into the first memory.

[0072] The first memory is independent of the global memory and is used to temporarily store data subsequences.

[0073] Here, the first memory can be used to provide an intermediate cache layer between the global memory and the second memory corresponding to the expert model, and can be used to optimize data transmission and improve data access efficiency. For example, the first memory can be on-chip memory independent of the global memory.

[0074] After reading multiple consecutive data subsequences from a contiguous physical address in global memory, each data subsequence can be temporarily written into the first memory.

[0075] It should be noted that the order in which multiple data subsequences are written to the first memory is the same as the order in which multiple data subsequences are read from the global memory.

[0076] As a storage method, the collected data subsequences can be treated as a whole and directly written into the reserved contiguous storage area in the first memory.

[0077] As an alternative storage method, space in the first memory can be dynamically allocated based on the size of the data subsequence, and the data can be written to this dynamically allocated area. This first memory can be a storage area with access speed faster than global memory.

[0078] Step 14: Based on the index value of each expert model in the target index table, read the target data corresponding to each expert model from each data subsequence in the first memory.

[0079] The target index table contains the index information of each expert model and the target data it needs to process in the data subsequence.

[0080] Here, the target index table includes the index value corresponding to each expert model, as well as the index value of the target data to be processed by each expert model in the data subsequence. The index value can be an identifier used to locate data or expert models in the target index table. Specifically, during the operation of the MoE architecture, an index table generated based on the routing decision of the MoE architecture can be maintained for each expert model and its corresponding target data to be processed. This index table can indicate the target data to be processed by each expert model, as well as the address information of the target data in global memory.

[0081] After each data subsequence is written into the first memory, the target data required for each expert model in each data subsequence can be determined according to the mapping relationship between the expert models and their corresponding target data in the target index table.

[0082] It should be noted that reading the target data corresponding to each expert model from each data subsequence in the first memory in a non-continuous manner can be done by collecting the target data corresponding to the expert model from each data subsequence in the first memory based on the index information between the expert model and the target data in the target index table.

[0083] As an example of a reading method, the target index table can be scanned sequentially. For each expert model recorded in the table and its corresponding target data index value, the corresponding target data to be processed can be directly located and read from the data subsequence in the first memory. For example, the index value can be an offset based on the initial address or a memory address in the first memory.

[0084] As another exemplary reading method, multiple expert model indexes in the target index table can be processed in parallel, and multiple read operations can be initiated simultaneously to obtain the target data corresponding to each expert model to be processed from the data subsequence stored in the first memory.

[0085] Step 15: Store the target data in the second memory.

[0086] The second memory corresponds to each expert model, and the target data in the second memory is used when each expert model processes the target data.

[0087] Here, the second memory corresponds to each expert model and is used to store the target data that the expert model is about to process. The second memory is usually located near the expert model and serves as the local thread processor for each expert model, providing fast data access for the expert model to process the target data. For example, the local thread register corresponding to the expert model.

[0088] As an example of storage methods, target data belonging to a specific expert model, read from a first memory, can be directly transferred and written to a second memory closely associated with that expert model. This second memory can be a local cache or register file dedicated to the expert model.

[0089] As an alternative storage method, a fixed-size secondary memory region can be pre-allocated for each expert model, and the corresponding data can be written into it. When the expert model needs to process data, it can quickly retrieve it directly from its corresponding secondary memory.

[0090] As an example, the hybrid expert model architecture can process the original video data sequence {1,2,3...30} stored in the first storage location in global memory. Before processing, the hybrid expert model architecture can classify the processing data in the original video data sequence based on routing decisions, and determine the corresponding mapping relationship between different types of processing data in the original video data and the expert models, that is, the index relationship between the expert models and different processing data. Specifically, the original video data includes the audio data sequence {1,11,21} and the image data sequence {2,12,22}. Based on routing decisions, it can be determined that the audio data sequence and the image data sequence can be assigned to the audio expert model and the image expert model respectively for processing. Thus, an index table is generated that maps the audio data sequence {1,11,21} to the audio expert model and the image data sequence {2,12,22} to the image expert model. That is, the audio expert model maps the processing data at address {1,11,21}, and the image expert model maps the processing data at address {2,12,22}. After determining the processing location of the raw video data sequence in the first storage location in global memory, access to the raw video data sequence in global memory can begin.

[0091] To improve the access efficiency of accessing the original video data sequence, this embodiment can divide the original video data sequence {1,2,3...30} into multiple data subsequences {1-10,11-20,21-30} according to the preset sequence length of 10. Each data subsequence (1-10), (11-20), and (21-30) in the global memory is read in a continuous reading manner.

[0092] After storing each data subsequence sequentially into the first memory, the target data corresponding to each expert model can be read from each data subsequence in the first memory according to the pre-generated target index table, and after reading the target data, the target data is stored into the second memory corresponding to each expert model.

[0093] For example, the target index table records in detail the specific location of each expert model and the target data it needs to process in the first memory. Based on the mapping relationship between expert models and processing data in the target index table, target data belonging to a specific expert model, read from the first memory, is stored in the second memory corresponding to that expert model. For instance, based on the mapping relationship between the audio expert model's processing data at address {1,11,21} and the image expert model's processing data at address {2,12,22}, the {1,11,21} data to be processed by the audio expert model is stored in its dedicated local thread register, and the {2,12,22} data to be processed by the image expert model is also stored in its dedicated local thread register. When the audio expert model and the image expert model start data processing tasks, they can directly and quickly call these target data from their corresponding local thread processors, thereby avoiding frequent access to global memory and significantly improving data access efficiency.

[0094] It should be noted that after the target data is stored in the second memory, each expert model can retrieve the target data stored in its corresponding second memory, perform calculations on the target data in global memory, and obtain the corresponding result data. Here, the result data is the processing result data of each expert model after processing the target data, and each processing result data corresponds to the location in global memory where each expert model performed its processing.

[0095] Furthermore, it should be explained in detail that the audio expert model and image expert model mentioned in the above embodiments are for better illustrating the data processing rearrangement process. In the actual processing process, each data can select multiple suitable expert models to process the data. That is, each data corresponds to TopK expert models, where TopK is greater than or equal to 1.

[0096] In the aforementioned data access method, this embodiment reads and writes multiple data subsequences from global memory sequentially into a high-speed first memory for temporary storage. Then, based on the target index table, it collects the target data corresponding to each expert model from the first memory and stores it in a dedicated second memory for the expert model. This transforms the data originally scattered in global memory into local data that the expert model can access efficiently. Thus, a hierarchical, preprocessing, and localized access strategy is implemented, effectively solving the problem of low data access efficiency caused by the discontinuous data storage in hybrid expert model architectures.

[0097] In one exemplary embodiment, such as Figure 2 As shown, after step 15, the data access method may further include a scheme for reversing the processed result data, specifically including steps 16 to 20, wherein:

[0098] Step 16: In response to the start signal of data processing of the expert model, the target data corresponding to each expert model is retrieved from the second memory for data processing to obtain multiple corresponding result data.

[0099] Multiple result data are stored consecutively in the second storage location of the global memory.

[0100] When the expert model is ready to perform calculations, it can send a start signal. In response, the system loads the target data previously prepared for the expert model and stored in the second memory into the second storage location of global memory for the expert model to perform calculations. After processing, the expert model can output the corresponding result data.

[0101] Here, the start signal can be a hardware interrupt signal, triggered by the expert model after completing preliminary preparations. Specifically, a call signal can be obtained through the Direct Memory Access (DMA) controller. In response to this call signal, data blocks in the secondary memory are efficiently transferred to a designated area of ​​global memory for data processing. Furthermore, data calls can be performed by directly reading from the secondary memory and writing to global memory via processor instructions, or through memory-mapped I / O. The expert model can be a software module running on a specific computing core, and its data processing involves executing predefined algorithms.

[0102] Step 17: Read the result data of each expert model's data processing of the target data from the target location in global memory in a continuous reading manner.

[0103] Since expert models may process data in parallel or asynchronously, their results are scattered across different "secondary storage locations" in global memory. By using sequential reading, memory access efficiency can be optimized, latency caused by random access can be reduced, and the collection of results data can be accelerated.

[0104] Specifically, when the result data is stored in contiguous or predictably contiguous blocks in global memory, the DMA controller can be configured to read the result data of each expert model from global memory in batches and continuously according to a preset address range and step size. Alternatively, when the result data is stored in global memory with some intervals but is generally ordered, the data collection unit or processor core can sequentially traverse the areas in global memory where the result data is stored according to a pre-planned memory layout and read the data in a streaming manner.

[0105] Step 18: Based on the index information of each expert model in the target index table and the target data to be processed, write the result data corresponding to each expert model into the first memory.

[0106] Here, the target index table includes the mapping relationship between expert models and result data, and the first memory serves as a temporary storage area, which can improve the efficiency of accessing result data in global memory.

[0107] It should be noted that writing the result data corresponding to each expert model into the first memory can be done by distributing (Scattering) the result data corresponding to each expert model to the first memory in a non-contiguous manner, based on the index information between the expert model and the target data in the target index table.

[0108] In addition, when the expert model is processing data, a result data buffer can be maintained. When the result data of an expert model is read from global memory, the target write address in the first memory is calculated according to the index information of the expert model in the target index table. Then the result data is written to the target address in the result data buffer. In this way, the address of receiving result data from global memory can be obtained. Furthermore, the result data of different expert models can be written to the preset storage area of ​​the first memory in a predetermined order or grouping manner, in combination with the expert model ID and result data length in the target index table.

[0109] Step 19: Read the result subsequence corresponding to the result data of each expert model from the first memory.

[0110] Step 20: Store the result subsequence corresponding to the result data of each expert model into the target area of ​​global memory.

[0111] After the result data obtained after expert model processing is written non-continuously to the first memory, the result data can be retrieved from the first memory and connected to the target area corresponding to the global memory. Therefore, it is necessary to read the result subsequence corresponding to the result data of each expert model from the first memory in order to feed the result data processed by the expert model back to the main data stream of the global memory.

[0112] Specifically, the DMA controller can be used to transfer the integrated result data blocks in the first memory to a specified target area in the first storage location in the global memory in one go or in batches. In this way, the result data can be accurately written back to the target area in the original first storage location in the global memory according to the layout of the original data.

[0113] It should be noted that the above memory access method optimizes the collection and write-back mechanism of expert model processing results. The target data corresponding to the expert model in the second memory is retrieved to the second storage location in global memory for actual data processing, thus obtaining the result data. This data is then loaded from the second memory into global memory for processing, enabling the expert model to utilize the high-speed access capabilities of global memory for efficient computation. Subsequently, the result data of each expert model is read sequentially from the designated target location in global memory. This sequential reading strategy effectively reduces the randomness of memory access and improves the efficiency of data collection. The first memory, serving as an intermediate buffer, can first integrate the result data before reading this integrated result data from the first memory and writing it back to the target area in the first storage location of global memory. This achieves effective distribution and processing of expert model data, avoiding the low access efficiency caused by scattered result data, ensuring a smooth data processing flow and the integrity of the result data, thereby improving the overall operating efficiency of the hybrid expert model architecture.

[0114] As an example, in the hybrid expert model architecture, after the audio expert model and the image expert model have completed processing their respective target data, the audio expert model generates the processed result data {1', 11', 21'}, and the image expert model generates the processed result data {2', 12', 22'}. These result data are stored consecutively in the audio expert result region and the image expert result region in the second storage location of global memory, according to expert category. The result data is then de-sorted, with the goal of restoring the result data to the corresponding positions of the original sequence {1,2,3...30}.

[0115] First, taking the first data subsequence (1-10) as an example, the result data {1'} belonging to the data subsequence (1-10) can be read continuously from the audio expert result area of ​​the global memory, and the result data {2'} of the data subsequence (1-10) can be read from the image expert result area. These data are then directly loaded into the second memory, namely the thread local register corresponding to each expert model.

[0116] Based on the target index table {Audio: 1, 11, 21; Image: 2, 12, 22}, the result data temporarily stored in the second memory (thread-local register) can be distributed and stored in the corresponding locations of the first memory (high-speed on-chip cache) in a scattered manner. Specifically, the audio result 1' in the register is written to the storage unit in the on-chip cache mapped to the final sequence position 1, and the image result 2' is written to the storage unit mapped to position 2.

[0117] After all expert results in the first data subsequence (1-10) have been written from the second memory to the first memory, the complete subsequence data block {1', 2', …} in the first memory (on-chip cache), which now contains data in the correct order, can be written continuously and output to the area (1-10) of the final result sequence in the global memory at once.

[0118] Subsequences (11-20) and (21-30) perform the same de-rearrangement steps: first, the expert results are read continuously from global memory into registers, then written from registers to the corresponding positions in the on-chip cache, and finally the complete block of the on-chip cache is written continuously back to global memory.

[0119] In this way, all expert model results are efficiently reorganized and written back to their correct sequential positions in global memory, forming a complete final result sequence. Throughout the de-reordering process, read operations on the expert result region in global memory are continuous, as are write operations on the final result region; non-contiguous, scattered write operations are only performed within the fastest on-chip cache. This reduces non-contiguous access to slow global memory, shifting time-consuming data movement and reorganization overhead to the high-speed on-chip storage hierarchy, thus significantly improving the overall efficiency of data access and reorganization.

[0120] As a concrete example of rearrangement, such as Figure 3 As shown, Figure 3 This diagram illustrates a scenario where permute is rearranged for memory access methods. Specifically, assume a hybrid expert model architecture includes four expert models: Expert 0, Expert 1, Expert 2, and Expert 3. During the routing computation phase of the MoE architecture, the processing data 0 to 3, and tokens 0, 1, 2, and 3 corresponding to each expert model can be determined. At this point, the mapping relationship between each expert model and the processing data tokens can be established.

[0121] Here, each processed data can correspond to two expert models, i.e., TopK=2, where token0 corresponds to expert model numbers 1 and 3; token1 corresponds to expert model numbers 0 and 3; token2 corresponds to expert model numbers 0 and 1; and token3 corresponds to expert model numbers 1 and 3. Furthermore, the hidden size in each processed data is not fixed.

[0122] During the data access phase, a permute step can be performed to rearrange the data from global memory to the local thread registers corresponding to each expert model. Specifically, after reading and inputting the processing data token0, token1, token2, and token3 from global memory into the first memory, the data stored in the first memory is written to the second memory corresponding to each expert model (Expert 0, Expert 1, Expert 2, Expert 3). Finally, based on the processing data corresponding to each expert model on the second controller, the output value is processed according to the algorithm on each expert model at a preset location in global memory. This achieves the rearrangement of the processed data.

[0123] As a concrete example of unrearrangement, such as Figure 4 As shown, Figure 4 This diagram illustrates a scenario for unpermute in memory access methods. Specifically, Expert 0, Expert 1, and Expert 3 (i.e., expert models 0, 1, and 3) complete data processing at their respective locations in global memory, obtaining the corresponding processing results, which are then used as output values. Here, the obtained probability values ​​(probs) are used as the output values, input from global memory to the first memory, and then transferred to the second memory. Finally, the second memory outputs the output to the corresponding location in global memory, thus achieving unpermute of the processing results.

[0124] Therefore, in an expert model architecture, the processed results are often scattered across different areas of global memory. This embodiment loads data from the second memory into global memory for processing in response to the expert model's start signal, and efficiently collects these scattered results using continuous reads. Subsequently, the first memory is used as an intermediate buffer, and combined with the target index table, the results are centrally integrated, and finally written back to a designated area of ​​global memory. This mechanism effectively solves the problems of scattered results and low write-back efficiency, avoids frequent random memory accesses, significantly improves data flow efficiency and overall system throughput, and ensures that the hybrid expert model architecture can more efficiently complete complex computational tasks.

[0125] In some of the embodiments described above in this application, a continuous data sequence can be divided into multiple data subsequences and accessed in a continuous reading manner. However, in actual access, if the data sequence is divided in an unreasonable way, for example, if the memory access characteristics of the underlying hardware are not fully considered, it may lead to low data access efficiency, increased memory bandwidth consumption, and thus affect the data processing performance of the entire hybrid expert model architecture.

[0126] Based on this, in an exemplary embodiment, such as Figure 5 As shown, step 11 involves dividing the data sequence to be processed, stored in the first storage location in global memory, into multiple data sub-sequences, including steps 111 to 112, where:

[0127] Step 111: Determine the granularity of the global memory partitioning based on the hardware cache capacity.

[0128] The block granularity is proportional to the access granularity of data access to global memory. The block granularity is used to balance the speed and resources of accessing data in global memory.

[0129] Before partitioning the data, since reading data from the cache is usually much faster than reading from global memory, the underlying hardware caching mechanism can be identified and utilized to determine the granularity of access that the first memory must use to access data from global memory to achieve optimal performance. This determines the granularity of partitioning the global memory.

[0130] Here, the granularity of the partitioning can be proportional to the granularity of the access, which is the smallest unit or boundary that data follows when stored and accessed in memory.

[0131] It should be noted that the access granularity is the smallest data block size that the hardware reads or writes from memory at a time. For example, the block granularity can be an integer multiple of the access granularity. This allows full utilization of the hardware's parallelism and block transfer capabilities, thereby improving data access speed while avoiding excessive consumption of memory bandwidth or computing resources.

[0132] For example, the current system cache line size can be obtained by querying system hardware information, including CPU registers, BIOS / UEFI information, or operating system APIs, and used as a benchmark for the block granularity. For example, 64 bytes or 128 bytes. Alternatively, performance testing and tuning can be performed by running benchmarks under different hardware configurations to analyze memory access performance and resource consumption under different block granularities, thereby selecting an optimal block granularity.

[0133] Step 112: Divide the data sequence to be processed, which is stored in the first storage location in the global memory, into multiple data subsequences, using block granularity as the unit of division.

[0134] When dividing a large, original sequence of data to be processed into smaller subsequences, the starting address and / or length of each subsequence should be an integer multiple of a predetermined block granularity. This ensures that each subsequence is aligned in memory, enabling the hardware to efficiently transfer blocks when reading these subsequences and avoiding the performance penalty caused by unaligned access.

[0135] For example, the total length of the original data sequence can be determined, and then the number of data subsequences that can be divided can be calculated based on the determined block granularity. The length of each data subsequence can be fixed at the block granularity or an integer multiple of the block granularity.

[0136] Another approach is to ensure that the starting address of each data subsequence is an integer multiple of the block granularity relative to the base address of global memory during partitioning, or, depending on the actual data volume and the needs of the expert model, round up the starting address of each subsequence to the nearest block granularity boundary.

[0137] Therefore, when partitioning the data sequence to be processed in global memory, the memory access characteristics of the underlying hardware are fully considered. By determining the block granularity based on the hardware cache capacity and then partitioning the data according to this granularity, it is ensured that the generated data subsequences can efficiently match the hardware access patterns. This significantly improves the efficiency of reading data from global memory, reduces performance losses caused by unaligned accesses or cache misses, thereby optimizing memory bandwidth utilization and ultimately improving the overall speed and resource utilization of data access in the hybrid expert model architecture.

[0138] In the above implementation, a method is proposed to determine the block granularity based on the hardware cache capacity and to divide the data subsequences using this block granularity. However, in the actual data access process, the starting memory address and access data length of the data subsequence may not completely match the preset block granularity, which may lead to reduced memory access efficiency or unnecessary boundary data reads, thereby affecting the overall memory access performance.

[0139] Based on this, in an exemplary embodiment, such as Figure 6 As shown, in step 11, after dividing the data sequence to be processed stored in the first storage location in global memory into multiple data sub-sequences, the data access method may further include a scheme for determining whether the multiple data sub-sequences after division are aligned with a preset block granularity, including steps 113 to 116, wherein:

[0140] Step 113: Obtain the starting memory address of the data subsequence and determine the access granularity for data access.

[0141] Here, the memory address can be queried through the memory management unit or read directly from the pre-stored address information in the data structure, or it can be automatically allocated by the system and returned to its starting position in memory after the data subsequence is divided.

[0142] In addition, the granularity of data access can be determined. Here, the granularity of access is the smallest data block size that the hardware reads or writes data from memory at one time. This granularity is usually related to the hardware bus width, cache line size, or page size, and is the smallest effective unit of memory access. It can be an access range determined after fully considering the hardware's parallelism and block transfer capabilities.

[0143] As one implementation method, the access granularity can be an access range that is determined in real time based on the system's memory access capabilities.

[0144] Step 114: Based on the starting memory address, determine whether the access data length and access granularity for accessing the data subsequence are aligned.

[0145] Determining whether the access data length and access granularity are aligned can help determine whether the actual amount of data to be read from memory is an integer multiple of the preset block granularity.

[0146] Step 115: If the accessed data length and access granularity are not aligned, calculate the alignment offset.

[0147] The alignment offset is the correction value needed to adjust the length or starting address of unaligned accessed data to the alignment boundary.

[0148] Step 116: Adjust the starting memory address of the data subsequence according to the alignment offset to determine the offset read address for reading the data subsequence.

[0149] Adjusting the starting memory address ensures that actual memory read operations begin from an aligned address, even if the logical starting address of the original data subsequence is not aligned. This adjusted address better matches the hardware's memory access patterns, improving access efficiency.

[0150] For example, the calculated alignment offset can be subtracted from the original starting memory address to obtain an earlier, aligned read start point, or the original starting memory address can be adjusted up or down to the nearest alignment boundary.

[0151] Therefore, by performing alignment checks and address adjustments on the actual access of each data subsequence after data subsequence partitioning, even if the logical starting address or access length of the original data subsequence is not aligned, the actual memory read operation can start from an aligned address and potentially read a data block of aligned length. This allows the memory controller to perform data transfer using its optimal access mode, avoiding the additional overhead caused by unaligned access and significantly improving the efficiency and performance of data memory access.

[0152] In some of the above implementations, a scheme was proposed to adjust the starting memory address of the data subsequence to determine the offset read address. However, in actual memory access, simply adjusting the starting memory address may result in reading invalid data or accessing out of bounds, thereby affecting the accuracy of data access and the stability of the system.

[0153] Based on this, in one implementation, such as Figure 7 As shown, step 116, adjusting the starting memory address of the data subsequence according to the alignment offset to determine the offset read address for reading the data subsequence, includes steps 116A to 116D, wherein:

[0154] Step 116A: Calculate the length of invalid data for the data subsequence based on the alignment offset.

[0155] The invalid data length is the length of the invalid data generated by offsetting the data subsequence.

[0156] It should be noted that when reading a data subsequence, if the actual reading range exceeds the valid data range due to the offset operation, or if additional data that does not belong to the current data subsequence is read, corresponding invalid data will be generated.

[0157] The length of invalid data can be determined by comparing the boundaries of the original data subsequence with the read range after the offset. For example, if the original data subsequence starts at address A and has a length of L, and after the offset, it starts reading from address A', then the portion between A' and A (if A' < A) or the portion between A+L and A'+L' (if A'+L' > A+L) may contain invalid data. Alternatively, when performing the offset operation, the system can pre-calculate the amount of additional data that may be introduced due to the offset; this additional data is useless in the current processing context and is therefore invalid.

[0158] Step 116B: If the sum of the alignment offset and the length of the invalid data is less than the access granularity, then determine the first maximum offset value based on the sum of the alignment offset and the length of the invalid data.

[0159] When adjusting the offset, to prevent excessively large offsets from reducing read efficiency or introducing too much invalid data, a threshold can be set. For example, the sum of the access granularity and the invalid data can be set as the offset threshold. When the calculated total offset (the sum of the alignment offset and the length of invalid data) is less than this threshold, this sum is used as the first maximum offset value for subsequent safety adjustments. Alternatively, hardware logic or software algorithms can be used to compare the sum of the alignment offset and the length of invalid data with a preset access granularity in real time. If the condition is met, this sum is used as the upper limit for restricting the offset.

[0160] Step 116C: If the length of the invalid data is less than the alignment offset, then determine the second maximum offset value based on the length of the invalid data.

[0161] As another boundary case, when the length of invalid data is relatively small and smaller than the access granularity, the length of invalid data is used as a limit to avoid excessive offset. After calculating the length of invalid data and the alignment offset, a conditional judgment is made. If the length of invalid data is small, it is used as the second maximum offset value. Alternatively, through logic gates or conditional branches, the length of invalid data can be selected as a reference upper limit for adjusting the offset under specific conditions.

[0162] Step 116D: Adjust the alignment offset based on the safety offset value by subtracting the adjusted alignment offset from the starting memory address of the data subsequence to determine the offset read address.

[0163] The safety offset value includes either the first maximum offset value or the second maximum offset value.

[0164] Based on the results of the aforementioned conditional judgment, either the first maximum offset value or the second maximum offset value is selected as the safe offset value. Then, the original alignment offset is compared with the safe offset value, and the smaller of the two values ​​is taken as the final adjustment amount, which is subtracted from the starting memory address.

[0165] Alternatively, an offset adjustment module can be designed. This module takes the original alignment offset, invalid data length, and access granularity as input, generates a safe offset value, uses the safe offset value to correct the original alignment offset, and finally outputs the adjusted offset for address calculation.

[0166] As a concrete example of aligned access, such as Figure 8 As shown, Figure 8A schematic diagram of a scenario for aligned access in a memory access method. Specifically, this scenario involves block (seq_tile) access to serialized data (such as permutated_hidden_states) during data rearrangement or anti-rearrangement. Assume that each seq_tile contains token data distributed to different experts (for example, tile0_token_num_expert_0 represents the number of tokens of expert model 0 in the 0th seq_tile, and tile1_token_num_expert_0 represents the number of tokens of expert model 0 in the 1st seq_tile), and the access needs to meet the memory alignment requirement (the alignment granularity is align_granu). If the original starting address is not aligned, it needs to be offset forward by align_offset to obtain the aligned address align_addr.

[0167] However, since in each seq_tile according to the corresponding target index table indices_trans, there may be indices_padding invalid addresses (invalid_addr), that is, when the data length value in each seq_tile exceeds the total sequence range, directly offsetting the address may cause the following problems:

[0168] If align_offset + indices_padding < align_granu, that is, if the sum of the forward offset align_offset and the invalid address is less than the access granularity, then when processing the current seq_tile, the aligned read operation prematurely touches the data area of the next seq_tile, causing unexpected data overwriting.

[0169] If indices_padding < align_offset, that is, if the length of the invalid data length is less than the forward alignment offset align_offset, then the alignment offset may cause the read range to read the processed area of one more seq_tile forward, resulting in data being repeatedly accessed and processed.

[0170] Therefore, in this scenario, the invalid data length related to indices_padding can be dynamically calculated, and the safe offset value can be determined accordingly. For example, when facing the risk of forward overwriting, the system can select the first maximum offset value to limit the offset amount; when facing the risk of backward repetition, the second maximum offset value is selected for constraint. Finally, the adjusted offset read address can ensure that the access to each seq_tile is strictly limited within its valid data boundary, while meeting the alignment requirement, thus avoiding cross-seq_tile data overwriting problems and ensuring the correctness and system stability of the data rearrangement / anti-rearrangement process.

[0171] This allows for precise control over the offset read address of data subsequences, avoiding issues such as reading invalid data or accessing out-of-bounds errors that might result from simple offsets. This significantly improves the accuracy of data memory access and system stability, ensuring the reliability of data processing in the hybrid expert model architecture. Simultaneously, intelligent adjustment of the offset effectively reduces unnecessary memory accesses and data transfers, thereby optimizing data memory access efficiency and reducing system resource consumption.

[0172] In one exemplary embodiment, such as Figure 9 As shown, in step 14, before reading the target data corresponding to each expert model from each data subsequence in the first memory according to the index value of each expert model in the target index table, the data access method may further include a scheme for obtaining the target index, including steps 141 to 143. Wherein:

[0173] Step 141: Obtain the original index table generated by the routing decision.

[0174] The original index table includes at least two expert models, and the relative position index of the data to be processed in the original data sequence for each expert model.

[0175] Through a specific routing mechanism, based on the characteristics of the data to be processed or the load on the expert model, the mapping relationship between data segments and the expert model for processing is determined, and these decision results are output in the form of an index table. This original index table typically contains the identifier of the expert model and the relative position information of its required data in the original data sequence.

[0176] For example, routing decisions can be generated by a gating network that assigns a weight or probability to each expert model based on input features and determines the data routing path accordingly. Alternatively, routing decisions can be implemented using pre-defined rules or lookup tables that directly map certain attributes of the data to the corresponding expert models.

[0177] The original index table comprises at least two expert models and the relative position index of the data to be processed within the original data sequence for each expert model, thus defining the structure of the original index table. The relative position index represents the offset of data within a local range, rather than the actual physical address in global memory. For example, the original index table could be a data structure where each entry contains an expert model ID and one or more relative index values. These relative index values ​​might indicate the offset of data within a data block or relative to a logical starting point.

[0178] Step 142: Convert the relative position index of each expert model in the original index table corresponding to the original data to be processed into an absolute index value.

[0179] The absolute index value ranges from 0 to (N*M-1), where N is the number of data subsequences and M is the number of data elements contained in each data subsequence.

[0180] Because data may be partitioned and rearranged in global memory, the original relative indexes cannot be directly used to read data from the first memory. Therefore, these relative indexes need to be converted into global absolute indexes that are available in the first memory. The conversion process may involve adding a base address or offset to the relative index, which represents the starting position of the corresponding data subsequence in the first memory.

[0181] For example, if the original data sequence is divided into multiple data subsequences and these subsequences are stored contiguously in the first memory, then a relative index needs to be added to the starting offset of its corresponding data subsequence in the first memory in order to obtain a global absolute index.

[0182] The absolute index value ranges from 0 to (N*M-1) based on the system's division and storage of data sequences, ensuring that the absolute index value can cover the entire logical space from the first data element to the last data element.

[0183] For example, if the original data sequence is uniformly divided into N subsequences of size M, and these subsequences are logically and continuously organized in the first memory, then the absolute index value will naturally fall within the range of 0 to N*M-1.

[0184] Step 143: Update the absolute index values ​​in the original index table to obtain the target index table.

[0185] After converting relative indexes to absolute indexes, the relative index information in the original index table can be replaced with the calculated absolute index values, thus forming the final target index table. This target index table can then be used directly for efficient data retrieval from the first memory.

[0186] For example, you can directly modify the data structure of the original index table, updating the field storing the relative index to the field storing the absolute index.

[0187] This ensures that the relative indexes generated by routing decisions can be accurately translated into absolute indexes available in the first memory, thus providing each expert model with a precise access path to the data it needs. This avoids data access errors caused by index mismatches, improves the accuracy and efficiency of data access, and ultimately optimizes the overall performance of the hybrid expert model architecture.

[0188] In the above implementation, by dividing the data sequence in global memory and acquiring the target data in a continuous reading manner, combined with the temporary storage of data subsequences in the first memory, and reading the target data from the first memory to the second memory corresponding to the expert model according to the target index table, the problem of low memory access efficiency caused by the non-continuous data storage is solved. Specifically, the partitioning module and the first access module acquire the target data from global memory in a continuous reading manner, avoiding the inefficiency of expert models directly performing non-continuous access in traditional solutions; the first storage module temporarily stores the data subsequences in an independent first memory as a high-speed cache layer, significantly reducing the latency of subsequent data access; the second access module and the second storage module accurately distribute the data to the second memory corresponding to each expert model according to the target index table, providing the expert model with a near-source, high-speed data access channel. Through the above technical solution, the non-continuous data originally scattered in global memory is transformed into local data that can be efficiently accessed by the expert model, effectively improving the data access efficiency and overall processing performance of the hybrid expert model architecture.

[0189] It should be understood that although the steps in the flowcharts of the above embodiments are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the above embodiments may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages in other steps. It is understood that the steps in different embodiments can be freely combined as needed, and all non-contradictory solutions formed by such combinations are within the scope of protection of this application.

[0190] Based on the same inventive concept, this application also provides a data access device for implementing the data access method described above. The solution provided by this device is similar to the implementation described in the above method; therefore, the specific limitations in one or more data access device embodiments provided below can be found in the limitations of the data access method described above, and will not be repeated here.

[0191] In one exemplary embodiment, such as Figure 10 As shown, a data access and storage device is provided, comprising: a partitioning module, a first access module, a first storage module, a second access module, and a second storage module, wherein:

[0192] The partitioning module is used to partition the data sequence to be processed, which is stored in the first storage location in global memory, to obtain multiple data subsequences;

[0193] The first access module is used to read each data subsequence in global memory in a sequential reading manner;

[0194] The first storage module is used to store each data subsequence sequentially into the first memory; wherein the first memory is independent of the global memory and is used to temporarily store the data subsequences.

[0195] The second access module is used to read the target data corresponding to each expert model from each data subsequence in the first memory according to the index value of each expert model in the target index table; wherein, the target index table contains the index information of each expert model and the target data to be processed in the data subsequence;

[0196] The second storage module is used to store the target data in the second memory; wherein the second memory corresponds to each expert model, and the target data in the second memory is used to call when each expert model processes the target data.

[0197] In one embodiment, the data access device further includes: a partitioning module, a first access module, a first storage module, a second access module, and a second storage module, wherein:

[0198] The processing module is used to respond to the start signal of data processing by the expert model, retrieve the target data corresponding to each expert model from the second memory for data processing, and obtain multiple result data; wherein, the multiple result data are stored consecutively in the second storage location of the global memory.

[0199] The third access module is used to read the result data of each expert model's data processing of the target data from the target location in global memory in a continuous reading manner;

[0200] The third storage module is used to write the result data corresponding to each expert model into the first storage based on the index information of each expert model in the target index table and the target data to be processed.

[0201] The fourth access module is used to read the result subsequence corresponding to the result data of each expert model from the first memory;

[0202] The fourth storage module is used to store the result subsequences corresponding to the result data of each expert model into the target area of ​​global memory.

[0203] In one embodiment, the partitioning module includes a granularity confirmation unit and a partitioning execution unit, wherein:

[0204] The granularity confirmation unit is used to determine the block granularity of the global memory partition based on the hardware cache capacity. The block granularity is proportional to the access granularity of data access to the global memory, and the block granularity is used to balance the speed and resources of data access in the global memory.

[0205] The execution partitioning unit is used to divide the data sequence to be processed, which is stored in the first storage location in global memory, into multiple data subsequences, with the partitioning granularity as the unit.

[0206] In one embodiment, the data access device further includes an alignment module, which comprises an address acquisition unit, an alignment determination unit, a first branch unit, and an adjustment unit, wherein:

[0207] The address acquisition unit is used to acquire the starting memory address of the data subsequence and determine the access granularity for data access.

[0208] The alignment determination unit is used to determine whether the access data length and access granularity are aligned when accessing a data subsequence based on the starting memory address.

[0209] The first branch unit is used to calculate the alignment offset if the accessed data length and access granularity are not aligned.

[0210] The adjustment unit is used to adjust the starting memory address of the data subsequence according to the alignment offset, so as to determine the offset read address for reading the data subsequence.

[0211] In one embodiment, the adjustment unit includes a calculation subunit, a second branch subunit, a third branch subunit, and an offset adjustment subunit, wherein:

[0212] The calculation subunit is used to calculate the length of invalid data for the data subsequence based on the alignment offset; where the length of invalid data is the length of invalid data generated by offsetting the data subsequence.

[0213] The second branch subunit is used to determine the first maximum offset value based on the sum of the alignment offset and the invalid data length if the sum of the alignment offset and the invalid data length is less than the access granularity.

[0214] The third branch subunit is used to determine the second maximum offset value based on the length of the invalid data if the length of the invalid data is less than the access granularity.

[0215] The offset adjustment subunit is used to adjust the alignment offset according to the safety offset value. The adjusted alignment offset is subtracted from the starting memory address of the data subsequence to determine the offset read address. The safety offset value includes a first maximum offset value or a second maximum offset value.

[0216] In one embodiment, the data access device further includes an index adjustment module, which comprises: an acquisition unit, an index conversion unit, and an index update unit, wherein:

[0217] The acquisition unit is used to acquire the original index table generated by the routing decision; wherein, the original index table includes at least two expert models, and the relative position index of the data to be processed in the original data sequence for each expert model;

[0218] The index transformation unit is used to convert the relative position index of each expert model in the original index table corresponding to the original data to be processed into an absolute index value; where the absolute index value ranges from 0 to (N*M-1), where N is the number of data subsequences and M is the number of data elements contained in each data subsequence;

[0219] The index update unit is used to update the absolute index value in the original index table to obtain the target index table.

[0220] Each module in the aforementioned data access device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in the processor of a computer device in hardware form or independent of it, or stored in the memory of the computer device in software form, so that the processor can call and execute the operations corresponding to each module.

[0221] In one exemplary embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 11 As shown, this computer device includes a processor, memory, input / output (I / O) interfaces, and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides the environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The database stores matrix data. The I / O interfaces are used for exchanging information between the processor and external devices. The communication interface is used for communication with external terminals via a network connection. When the computer program is executed by the processor, it implements a data access method.

[0222] Those skilled in the art will understand that Figure 11The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0223] In one exemplary embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to perform the steps of the data memory access method described above.

[0224] In one embodiment, a readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, performs the steps of the data access method described above.

[0225] In one embodiment, a program product is provided, including a computer program that, when executed by a processor, implements the steps in the above method embodiments.

[0226] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, artificial intelligence (AI) processors, etc., and are not limited to these.

[0227] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0228] The above embodiments are merely illustrative of several implementation methods of this application, and their descriptions are relatively specific and detailed. However, they should not be construed as limiting the scope of this application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A data access method, comprising: The data access method is applied to a hybrid expert model architecture including at least two distributed expert models, and the method includes: dividing a data sequence to be processed stored in a first storage location in a global memory to obtain a plurality of data subsequences; reading each of the data subsequences in the global memory in a continuous reading manner; storing each of the data subsequences to a first memory one by one; wherein the first memory is a memory independent of the global memory and temporarily storing the data subsequences; reading target data required by each expert model from each of the data subsequences in the first memory according to an index value of each expert model in a target index table; wherein the target index table contains index information of each expert model and target data required to be processed by the expert model in the data subsequence; storing the target data to a second memory; wherein the second memory corresponds to each of the expert models, and the target data in the second memory is used to call the target data for data processing by each expert model.

2. The data access method of claim 1, wherein, After the step of storing the target data to the second memory, the data access method includes: in response to a start signal of data processing by an expert model, calling target data corresponding to each expert model from the second memory for data processing to obtain a plurality of corresponding result data; wherein the plurality of result data is continuously stored in a second storage location of the global memory; reading the result data corresponding to each expert model from the second storage location of the global memory in a continuous reading manner; writing the result data corresponding to each of the expert models to the first memory according to the index information of each expert model and target data required to be processed by the expert model in the target index table; reading a result subsequence corresponding to the result data of each expert model from the first memory; storing the result subsequence corresponding to the result data of each expert model to a target area of the global memory.

3. The data access method of claim 1, wherein, The step of dividing the data sequence to be processed stored in the first storage location in the global memory to obtain a plurality of data subsequences includes: determining a block granularity for dividing the global memory according to a hardware cache capacity; wherein the block granularity is proportional to an access granularity for accessing data in the global memory, and the block granularity is used to balance the speed and resources for accessing data in the global memory; dividing the data sequence to be processed stored in the first storage location in the global memory to obtain a plurality of data subsequences in the block granularity as a division unit.

4. The data access method of claim 3, wherein, After the step of dividing the data sequence to be processed stored in the first storage location in the global memory to obtain a plurality of data subsequences, the data access method further includes: obtaining a starting memory address of the data subsequence and determining the access granularity for data access; determining whether an access data length for accessing the data subsequence and the access granularity are aligned according to the starting memory address; calculating an alignment offset if the access data length and the access granularity are not aligned; adjusting a start memory address of the data subsequence according to the alignment offset to determine an offset read address for reading the data subsequence.

5. The data access method of claim 4, wherein, The step of adjusting the start memory address of the data subsequence according to the alignment offset to determine the offset read address for reading the data subsequence includes: calculating an invalid data length of the data subsequence according to the alignment offset, wherein the invalid data length is a length of invalid data generated by offsetting the data subsequence; if the sum of the alignment offset and the invalid data length is less than the access granularity, determining a first maximum offset value according to the sum of the alignment offset and the invalid data length; if the invalid data length is less than the access granularity, determining a second maximum offset value according to the invalid data length; adjusting the alignment offset according to a security offset value, and subtracting the adjusted alignment offset from the start memory address of the data subsequence to determine the offset read address, wherein the security offset value includes the first maximum offset value or the second maximum offset value.

6. The data access method according to any one of claims 1 to 5, characterized in that, Before the step of reading target data corresponding to each expert model from each data subsequence in the first memory according to an index value of each expert model in a target index table, the data access method further includes: obtaining an original index table generated by a routing decision, wherein the original index table includes at least two expert models, and a relative position index of the original data corresponding to each expert model in the original data sequence; converting the relative position index of the original data corresponding to each expert model in the original index table into an absolute index value, wherein the range of the absolute index value is 0 to (N*M-1), N is the number of data subsequences, and M is the number of data elements included in each data subsequence; updating the absolute index value to the original index table to obtain the target index table.

7. A data access device, comprising: The data access device is applied to a hybrid expert model architecture including at least two distributed expert models, and includes: a division module configured to divide a data sequence to be processed stored at a first storage position in a global memory to obtain a plurality of data subsequences; a first access module configured to read each data subsequence in the global memory in a continuous reading manner; a first storage module configured to sequentially store each data subsequence to a first memory, wherein the first memory is a memory independent of the global memory and temporarily stores the data subsequences; a second access module configured to read target data corresponding to each expert model from each data subsequence in the first memory according to an index value of each expert model in a target index table, wherein the target index table includes index information of each expert model and target data to be processed by the expert model in the data subsequence. A second storage module is configured to store the target data into a second storage, wherein the second storage corresponds to each of the expert models, and the target data in the second storage is used to call each expert model to process the target data.

8. A computer device comprising a memory and a processor, the memory storing a computer program, characterized in that, The computer program is executed by the processor to implement the steps of the method in any one of claims 1 to 6.

9. A readable storage medium, having stored thereon a computer program, characterized in that, The computer program is executed by the processor to implement the steps of the method in any one of claims 1 to 6.

10. A program product comprising a computer program, characterized in that The computer program is executed by the processor to implement the steps of the method in any one of claims 1 to 6. The computer program is executed by the processor to implement the steps of the method in any one of claims 1 to 6.

Images