Hybrid expert model-based ensemble communication method, system, and device

Abstract

The disclosure provides a set communication method, system and device based on a mixed expert model, relates to the technical field of communication, and in particular to the technical fields of chip processors, computing power clusters, set communication operations, large models and the like. The set communication method is applied to a communication receiving end and includes the following steps: receiving a data write instruction, wherein the data write instruction includes to-be-processed data and first address information; accessing a virtual expert address in a pre-created virtual address space according to the first address information; applying a corresponding actual physical space to the virtual expert address based on the size of the to-be-processed data; and writing the to-be-processed data into the actual physical space. The set communication method provided by the disclosure can realize set communication without memory information synchronization.

Description

Technical Field

[0001] This disclosure relates to the field of communication technology, specifically to the fields of chip processors, computing clusters, collective communication operations, and large models, and particularly to a collective communication method, system, device, electronic device, computer-readable storage medium, and computer program product based on a hybrid expert model. Background Technology

[0002] In large-scale distributed training, Mixture-of-Experts (MoE) models are a model architecture that uses multiple "expert" networks to handle different tasks. In the MoE scenario, multiple processors participating in training require many-to-many variable-count communication (all-to-allv). Summary of the Invention

[0003] This disclosure presents a collection communication method, system, electronic device, computer-readable storage medium, and computer program product based on a hybrid expert model.

[0004] In a first aspect, embodiments of this disclosure propose a collective communication method based on a hybrid expert model, comprising: receiving a data writing instruction, wherein the data writing instruction includes data to be processed and first address information; accessing a virtual expert address in a pre-created virtual address space according to the first address information; requesting a corresponding actual physical space for the virtual expert address based on the size of the data to be processed; and writing the data to be processed into the actual physical space.

[0005] Secondly, embodiments of this disclosure propose a collective communication system based on a hybrid expert model, comprising: multiple processors and a switch, wherein the switch is configured to receive instructions, the instructions including data to be processed, address information, and data routing information; and multicast the instructions to the corresponding processors based on the data routing information; the processors are configured to respond to the received instructions as data write instructions, access virtual expert addresses in a pre-created virtual address space according to the address information; and apply for corresponding actual physical space for the virtual expert addresses based on the size of the data to be processed; and write the data to be processed into the actual physical space.

[0006] Thirdly, embodiments of this disclosure propose a collective communication device based on a hybrid expert model, comprising: a receiving unit configured to receive a data writing instruction, wherein the data writing instruction includes data to be processed and first address information; an access unit configured to access a virtual expert address in a pre-created virtual address space according to the first address information; an application unit configured to apply for corresponding actual physical space for the virtual expert address based on the size of the data to be processed; and a writing unit configured to write the data to be processed into the actual physical space.

[0007] Fourthly, embodiments of this disclosure provide an electronic device comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to implement the collection communication method as described in any implementation of the first aspect.

[0008] Fifthly, embodiments of this disclosure provide a non-transitory computer-readable storage medium storing computer instructions that enable a computer to implement the collection communication method as described in any implementation of the first aspect.

[0009] In a sixth aspect, embodiments of this disclosure provide a computer program product including a computer program that, when executed by a processor, can implement the set communication method as described in any implementation of the first aspect. Attached Figure Description

[0010] Other features, objects, and advantages of this disclosure will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings:

[0011] Figure 1 An exemplary system architecture is shown that embodiments of the collection of communication methods, systems, electronic devices, and computer-readable storage media of this disclosure can be applied;

[0012] Figure 2 A flowchart illustrating a ensemble communication method based on a hybrid expert model, provided as an embodiment of this disclosure;

[0013] Figure 3 A schematic diagram of virtual number address space and virtual expert address space in a collection communication based on a hybrid expert model provided in this disclosure embodiment;

[0014] Figure 4 A flowchart illustrating yet another ensemble communication method based on a hybrid expert model provided in this disclosure;

[0015] Figure 5 This is a schematic diagram illustrating data multicasting during the Dispatch phase, as provided in an embodiment of this disclosure.

[0016] Figure 6 To and Figure 5 A schematic diagram of the first routing information carried by the relevant data writing command;

[0017] Figure 7 This is a schematic diagram illustrating data reduction during the Combine stage, as provided in an embodiment of this disclosure.

[0018] Figure 8 To and Figure 7 A diagram illustrating the second routing information carried by the relevant data read command;

[0019] Figure 9 This is a schematic diagram of a collective communication system based on a hybrid expert model provided in this disclosure;

[0020] Figure 10 A structural diagram of a physical memory allocation device provided in an embodiment of this disclosure;

[0021] Figure 11 A structural diagram of yet another physical memory allocation device provided in the embodiments of this disclosure;

[0022] Figure 12 This is a structural block diagram of a collective communication device based on a hybrid expert model provided in this disclosure;

[0023] Figure 13 A schematic block diagram of an electronic device provided for embodiments of this disclosure. Detailed Implementation

[0024] The exemplary embodiments of this disclosure are described below with reference to the accompanying drawings, including various details of the embodiments to aid understanding; these should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this disclosure. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description. It should be noted that, unless otherwise specified, the embodiments and features described in this disclosure can be combined with each other.

[0025] The collection, storage, use, processing, transmission, provision, and disclosure of user personal information involved in the technical solution disclosed herein comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0026] This disclosure provides a processing procedure for all-to-allv communication in a MoE scenario. It assumes communication between four GPUs, where Rank is a logical number used to identify different GPUs; for example, Rank 0 identifies GPU number 0, Rank 1 identifies GPU number 1, and so on. The explanation assumes that each Rank contains only one expert model, and the communication initiator communicates with two communication receivers at a time. The MoE all-to-allv process consists of two phases: a dispatch phase and a combine phase.

[0027] Specifically, in the Dispatch phase, the input data (Token) is first calculated based on attention computation, addition operations, and normalization operations. The core of attention computation is calculating the attention weights of the input data. Common attention mechanisms include dot product attention and multi-head attention; addition operations are used to add the output of the attention layer to the input data; and normalization operations are used to normalize the result of the addition operations.

[0028] The token is then used as input and sent to the corresponding expert model according to the routing graph output by the gating network. Different expert models then perform feedforward neural network computations. The routing graph identifies the target expert model to which the data is sent; it can be represented as a list, with rows representing expert model identifiers and columns representing token identifiers. From Rank0's perspective, Rank0 needs to send the token to expert models at other ranks. For all Rank nodes, the token must be sent to other Ranks according to the routing graph.

[0029] In the Combine phase, the Combine process is essentially the reverse of the Dispatch process. After all expert models have completed the feedforward neural network calculations, the tokens need to be returned to the original Rank, and the weights of all tokens are calculated based on the pre-defined weight values ​​of each token. From Rank0's perspective, Rank0 needs to collect all the calculated tokens from the expert models of other Ranks and calculate the token weights based on the weight values.

[0030] In the alltoallv data transmission process of the MoE described above, the process of sending data requires sensing the specific location of the data at the remote end, inevitably necessitating the synchronization of some memory information. Moreover, this process occurs during inference or training, making it very time-consuming. Simultaneously, the sending end needs to inform the receiving end of the required memory amount, and the receiving end must allocate the memory space and synchronize the memory information with the sending end. Afterward, the sending end sends the data to the remote end based on the remote memory address information.

[0031] Based on this, this disclosure further provides a collection communication method, system, electronic device, computer-readable storage medium, and computer program product based on a hybrid expert model that does not require synchronization of address information.

[0032] Figure 1 An exemplary system architecture 100 is shown, in which embodiments of the collection of communication methods, systems, electronic devices, and computer-readable storage media of this disclosure can be applied.

[0033] like Figure 1As shown, the system architecture 100 may include a server 110, a network 120, and a computing cluster 130. The computing cluster 130 includes multiple graphics processors, such as graphics cards 131, 132, 133, 134, and 135 (not just five). Figure 1 (Only a few examples are used here), and graphics cards 131, 132, 133, 134, and 135 contain multiple graphics processing chips (i.e., GPUs), and these GPUs can communicate with each other in the MOE scenario. Network 120 is used as a medium to provide a communication link between server 110 and computing cluster 130. Network 120 can include various connection types, such as wired, wireless communication links, or fiber optic cables, etc.

[0034] Users can use terminal devices to interact with server 110 via network 120 to receive or send messages, etc. Various applications can be installed on server 110, computing cluster 130, and terminal devices to enable information communication between them, thereby enabling communication between multiple image processing chips.

[0035] The terminal device and server 110 can be hardware or software. When the terminal device is hardware, it can be various electronic devices with a display screen, including but not limited to smartphones, tablets, laptops, and desktop computers. When the terminal device is software, it can be installed in the aforementioned electronic devices and can be implemented as multiple software programs or software modules, or as a single software program or software module; no specific limitation is made here. When the server 110 is hardware, it can be implemented as a distributed server cluster composed of multiple servers, or as a single server. When the server 110 is software, it can be implemented as multiple software programs or software modules, or as a single software program or software module; no specific limitation is made here. The graphics processor constituting the computing power cluster 130 is usually hardware, but in special scenarios (such as simulation scenarios), it can also be software or software execution products; no specific limitation is made here.

[0036] Server 110 can provide various services through its built-in applications, such as sending communication commands to enable multiple GPUs in computing cluster 130 to communicate with each other.

[0037] Furthermore, after the multiple GPUs in the computing cluster 130 complete communication, they can also send feedback to the server 110.

[0038] It should be understood that Figure 1 The number of servers, networks, computing clusters, and graphics cards shown is merely illustrative. Depending on implementation needs, any number of servers, networks, computing clusters, and graphics cards can be included.

[0039] Please refer to Figure 2 , Figure 2 A flowchart of a collective communication method based on a hybrid expert model provided in this disclosure embodiment, the method being applied at a communication receiving end, wherein process 200 includes the following steps:

[0040] Step 201: Receive data write command.

[0041] In this collective communication approach, each communicating entity can either initiate a communication request as a sender or process the received request as a receiver. This step aims to have the executing entity of the collective communication method based on a hybrid expert model act as the receiving end for processing. This executing entity can be... Figure 2 One or more GPUs in the computing cluster 130 shown may also be other electronic devices capable of receiving and processing data, and this disclosure does not limit them.

[0042] Specifically, when a communication requester needs to communicate with other execution entities, the communication requester invokes a communication instruction and sends the communication instruction to the other entity. This communication instruction can be a data write instruction, a data read instruction, or other communication instructions; this disclosure does not limit the specific type of instruction.

[0043] The instruction form for writing data can be inc_v.store, and the instruction form for reading data can be inc_v.load_reduce.

[0044] In `inc_v.store(input_buffer, inc_group_buffer_va, length, rank_info, expert_info)`, `input_buffer` represents the source address of the data to be processed, `inc_group_buffer_va` represents the destination address of the data to be processed, `length` represents the length of the data to be processed, `rank_info` represents the rank ID of the `inc_v` store operation within the `inc` group, and `expert_info` represents the Expert ID that the token input data needs to reach. In other words, `inc_v.store(input_buffer, inc_group_buffer_va, length, rank_info, expert_info)` reads data of size `length` from the local input buffer and writes this data to the corresponding VA of the `INCgroup` based on `rank_info` and `expert_info`.

[0045] In `inc_v.load_reduce(inc_group_buffer_va, output_buffer, length, rank_info, expert_info, weight_info)`, `inc_group_buffer_va` represents the source address of the data to be processed, `output_buffer` represents the destination address of the data to be processed, `length` represents the length of the data to be processed, and `weight_info` represents the weight information of the token corresponding to the `expert`. In other words, `inc_v.load_reduce(inc_group_buffer_va, output_buffer, length)` reads `length` size of data from the `VA` corresponding to the `INC` group and writes it to the local output buffer based on `rank_info`, `expert_info`, and `weight_info`.

[0046] It should be noted that rank_info and expert_info are just one representation, and other methods can be used to replace them, such as bitmap, etc.

[0047] In some optional implementations of this embodiment, the data writing instruction includes data to be processed and first address information. The data to be processed refers to the data to be written in the data writing instruction, i.e., the input data (Token) of the communication request end. This data to be processed can be parameters, feature vectors, or other data that needs to be transmitted during the aggregate communication process in a hybrid expert model. The first address information refers to the address information corresponding to the target address where the data to be processed is written.

[0048] In some optional implementations of this embodiment, the data writing instruction may further include first routing information.

[0049] Specifically, the first routing information is used to identify the communication receiving end that sends the data to be processed. This first routing information is obtained by the gating network in the hybrid expert model based on the data to be processed. The requesting end invokes a data write command and sends it to the switch. The switch then sends the data write command to the communication receiving end based on the first routing information in the data write command. The communication receiving end can be one or more, depending on the first routing information; this disclosure does not impose any restrictions on this.

[0050] It should be noted that after the communication command is transmitted to the C2C engine via the System-on-Chip (NOC), the C2C engine converts the command into a data message or data packet for transmission within the aggregated communication network. In all the embodiments described below, the communication commands (data write commands or data read commands) refer to the data messages corresponding to the communication commands.

[0051] Step 202: Access the virtual expert address in the pre-created virtual address space based on the first address information.

[0052] Based on step 201, this step aims to enable the communication receiving end to perform a data writing operation according to the received data writing instruction.

[0053] In this embodiment of the disclosure, each execution entity in the communication set pre-creates its own virtual address space, which is used to store data of various expert models in the hybrid expert model. Each execution entity in the communication set contains one or more expert models. The virtual expert address in the virtual address space corresponds to an expert model in the execution entity; the virtual expert address is a block of addresses within the execution entity's virtual address space, and is an address segment allocated to a specific expert model within the virtual address space.

[0054] When the execution entity receives the data write instruction as the communication receiver, it first accesses the pre-created virtual address space and locates the virtual expert address in the virtual space according to the first address information carried by the data write instruction, so as to write the data to be processed in the data write instruction into the expert model corresponding to the communication receiver.

[0055] Step 203: Based on the size of the data to be processed, apply for the corresponding actual physical space for the virtual expert address.

[0056] Building on step 202, this step aims to allocate actual physical space for the data to be processed by the communication receiver.

[0057] The virtual expert address corresponds to the address pointed to by the first address information. Only when a data write instruction arrives and accesses this virtual address is the corresponding physical space requested. The actual physical space refers to the physical memory space in the computing device, used for actual data storage. The communication receiving end dynamically allocates physical memory based on the size of the data to be processed to ensure sufficient space for storage. The size of this actual physical space is consistent with the size of the data to be written. Of course, the size of this actual physical space can also be slightly larger or smaller than the size of the data to be processed; the specific size can be determined according to the actual situation, and this disclosure does not impose any limitations on this.

[0058] Step 204: Write the data to be processed into the actual physical space.

[0059] Building upon step 203, this step aims to write the data to be processed to the target address according to the data writing instruction to complete this communication.

[0060] The communication receiver writes the data to be processed into the actual physical space requested in step 203, completing the data storage process. By writing the data into the actual physical space, the hybrid expert model can access and use this data in subsequent computation processes.

[0061] This disclosure's embodiments pre-create a virtual address space in each execution entity, rather than requesting actual physical memory. When the execution entity at the communication receiving end receives a data write instruction (i.e., a data packet) and accesses the virtual expert address of the virtual address space according to the first address information in the data write instruction, it allocates actual physical space as needed in the address pointed to by the first address information, based on the size of the data to be processed in the data write instruction. The aggregate communication method provided in this disclosure does not require address information synchronization during communication, saving time and improving processing performance. Furthermore, the communication method provided in this disclosure requests the corresponding actual physical space based on the size of the data to be processed only when the data packet arrives and accesses the virtual expert address, preventing excessive memory usage and resource explosion, further improving processing performance.

[0062] In some optional implementations of this embodiment, the virtual address space is created based on the total size of all data to be processed in the collection communication.

[0063] Specifically, the virtual address space is pre-created internally by the executing entity. Before initiating aggregated communication, the receiving end estimates the total size of all data to be processed that needs to be transmitted during the process. Based on this total size, the receiving end creates a virtual address space of corresponding size to ensure that there is enough virtual address space to map all the data to be processed.

[0064] Since the executing entity does not know how much data it will receive during training or inference, the virtual address space is created according to the total size of all pending data of all executing entities in the collective communication. This ensures that during communication, the virtual expert address corresponding to the pending data can be found in the virtual address space based on the first address information. Thus, the actual physical space can be requested based on the virtual expert address, thereby enabling accurate data communication.

[0065] For example, if the total size of all the data to be processed during a collection communication process is 1GB, the receiving end will create a virtual address space of at least 1GB. This ensures that all the data to be processed can be located in the virtual address space.

[0066] The total size of all unprocessed data in the ensemble communication can be obtained by summing the sizes of all unprocessed data. Optionally, the total size of all unprocessed data in the ensemble communication can also be determined based on a pre-stored log file or a dataset pre-set for training or inference, etc., and this disclosure does not limit it in this way.

[0067] In some optional implementations of this embodiment, the size of the virtual address space is determined by the total number of data to be processed in the aggregate communication and the size of the data to be processed.

[0068] Specifically, each executing entity can process one or more pieces of data to be processed, and the size of the data to be processed is equal. Therefore, the total size of all data to be processed in the collective communication can be obtained by multiplying the total number of data to be processed by the size of each data piece. That is, the total number of tokens * hidden Size, where hidden Size is equal to the token size, and token size is a fixed value. Determining the size of the virtual address space in this way ensures that the virtual address space is neither too large, wasting resources, nor too small, leading to insufficient space.

[0069] In some optional implementations of this embodiment, the virtual address space includes a virtual numbered address space and a virtual expert address space. The virtual numbered address space stores the numbering information of each participant (i.e., each execution entity, including the execution entity corresponding to the communication requesting end and the execution entity corresponding to the communication receiving end) in the collective communication. The virtual expert address space stores the data of each expert model, with each virtual expert address space corresponding to a data storage area for one expert model. When the execution entity of the communication receiving end accesses the virtual numbered address space, a route is routed to the corresponding virtual expert address space.

[0070] The virtual numbered address space can be created by any execution entity in the collective communication process, and then broadcast to the other execution entities. The virtual expert address space is created by each execution entity individually. Both the virtual numbered address space and the virtual expert address space are the same size, created according to the total size of all data to be processed during the collective communication process.

[0071] It should be noted that this virtual address space is created for interaction with the switch during aggregated communication. After receiving a data write command from the requesting end, the switch first multicasts the data write command to the virtual address space corresponding to the receiving end; it does not actually store data. Therefore, this virtual address space may not be a separate virtual address space, but simply a handle to one. Only the virtual expert address space will request actual memory, and the virtual expert address space will be bound and mapped to the virtual address space.

[0072] Figure 3 This is a schematic diagram of the virtual number address space and the virtual expert address space in a collection communication based on a hybrid expert model, provided as an embodiment of this disclosure.

[0073] like Figure 3 As shown, assume the execution entity is a processor. The example illustrates communication between four processors: Rank0, Rank1, Rank2, and Rank3. Each processor's input includes four input tokens, and each processor includes two expert models. Therefore, the size of the virtual address space created by the processors is equal to the total size of the four processors' input tokens, which is the space occupied by 16 input tokens. The mapping relationship between the virtual address space and the virtual expert address space is a one-to-two mapping.

[0074] See you again Figure 3 The system issues a data write command, where the first address information in the data write command points to a virtual address in the virtual number address space. The execution entity of the communication requesting end sends the data write command (message) to the switch. The switch copies the data write command multiple times, performs a multicast operation on the data write command, and sends it to the virtual number address space of the communication receiving end. Based on the mapping relationship between the virtual number address space and the virtual expert address space, the switch accesses the virtual expert address in the virtual expert address space.

[0075] Figure 4 A flowchart illustrating another ensemble communication method based on a hybrid expert model provided in this disclosure is shown below. Figure 4 As shown, the method 400 includes the following steps:

[0076] Step 401: Receive data write command.

[0077] The specific operation of this step is the same as step 201 above, and will not be repeated here.

[0078] Step 402: Access the virtual number address of the pre-created virtual number address space based on the first address information.

[0079] Specifically, the receiving end locates the corresponding virtual number address in the virtual number address space based on the first address information. The virtual number address contains information related to a specific number, such as the number value and the expert model corresponding to the number.

[0080] Step 403: Access the virtual expert address in the virtual expert address space by mapping the virtual number address space to the virtual expert address space.

[0081] Specifically, there is a mapping relationship between the virtual number address space and the virtual expert address space. The communication receiving end uses this mapping relationship to find the corresponding virtual expert address in the virtual expert address space from the virtual number address.

[0082] Each implementing entity may include one expert model or multiple expert models.

[0083] When the execution entity of the communication receiving end includes only one expert model, the virtual number address space and the virtual expert address space are linearly mapped. In this case, accessing the virtual number address space is equivalent to accessing the virtual expert address space. After accessing the virtual number address of the virtual number address space according to the first address information, the virtual expert address space corresponding to the virtual number address space can be directly routed.

[0084] When the execution entity of the communication receiver includes at least two expert models, the virtual number address space and the virtual expert address space have a one-to-many mapping. In this case, accessing the virtual number address space is equivalent to accessing multiple virtual expert address spaces, requiring multiple memory accesses within the execution entity. The data write instruction received by the communication receiver may only need to be written to a portion of the expert models of that communication receiver. Therefore, after accessing the virtual number address of the virtual number address space based on the first address information, the corresponding virtual expert address space can be accessed according to the expert model to which the data to be processed is to be written.

[0085] This embodiment of the disclosure, through its two-level address space design, enables flexible mapping between numbering and expert models, facilitating rapid location of the corresponding expert model data based on the numbering information during aggregate communication.

[0086] Step 404: Based on the size of the data to be processed, request the corresponding physical space for the virtual expert address.

[0087] The specific operation of this step is the same as step 203 above, and will not be repeated here.

[0088] Step 405: Write the data to be processed into the actual physical space.

[0089] The specific operation of this step is the same as step 204 above, and will not be repeated here.

[0090] In some optional implementations of this embodiment, the data writing instruction further includes first routing information. Step 403: Accessing the virtual expert address of the virtual expert address space through the mapping between the virtual number address space and the virtual expert address space specifically includes: in response to the virtual number address space corresponding to at least two virtual expert address spaces, accessing the virtual expert address of the virtual expert address space through the mapping between the virtual number address space and the virtual expert address space according to the first routing information.

[0091] Specifically, a virtual address may correspond to multiple virtual expert address spaces, meaning that an executing entity may be associated with multiple expert models. In this case, the receiving end needs to determine the virtual expert address space to be accessed based on the first data routing information.

[0092] The first data routing information can be in the form of an expert bitmap or other data structure information. For example, the first data routing information can be an indicator or identifier used to indicate which virtual expert address space to select from multiple possible virtual expert address spaces. The communication receiving end selects one from the multiple virtual expert address spaces corresponding to the virtual number address based on the first data routing information, and then accesses the virtual expert address in that virtual expert address space.

[0093] In this way, the present disclosure can achieve a flexible mapping of one number to multiple expert models, enhancing the scalability and flexibility of the system.

[0094] In some optional implementations of this embodiment, step 404: Based on the size of the data to be processed, requesting the corresponding actual physical space for the virtual expert address, specifically includes the following steps:

[0095] In response to writing the data to be processed to the virtual expert address, a page fault interrupt is triggered. Based on the page fault interrupt, actual physical space is allocated to the virtual expert address, and the mapping relationship between the virtual expert address and the actual physical space is written into the page table memory. The actual physical space corresponding to the virtual expert address is obtained based on the page table memory.

[0096] Specifically, when the receiving end accesses the virtual expert address in the pre-created virtual address space based on the first address information, it attempts to write the data to be processed to the virtual expert address. At this time, since the virtual expert address has not yet been mapped to actual physical memory, the system triggers a page fault interrupt. A page fault interrupt is an exception handling mechanism in the operating system used to handle situations where the mapping from virtual address to physical address is missing.

[0097] In response to a page fault interrupt, the system allocates actual physical memory space for the virtual expert address. After allocation, the system writes the mapping between the virtual expert address and the actual physical memory space into the page table memory. The page table memory is a data structure used by the operating system to store the mapping relationship between virtual addresses and physical addresses.

[0098] The receiving end retrieves the actual physical address corresponding to the virtual expert address by querying the page table memory. Based on this actual physical address, the receiving end can write the data to be processed into the actual physical memory.

[0099] This embodiment of the disclosure triggers a page terminal error to request the corresponding actual physical memory for the virtual expert address only when the data write instruction reaches the communication receiving end and accesses the virtual expert address of the communication receiving end. This eliminates the need for memory information synchronization during communication, reducing time consumption and improving performance. Furthermore, this on-demand allocation of physical memory improves memory utilization efficiency and avoids pre-allocating large amounts of physical memory that may not be used.

[0100] In some optional implementations of this embodiment, allocating actual physical space for the virtual expert address in step 404 specifically includes: allocating actual physical space for the virtual expert address through the memory processing module within the processor.

[0101] Specifically, the processor integrates a dedicated memory processing module to handle the mapping of virtual addresses to physical addresses and the allocation of physical memory. When it is necessary to allocate actual physical space for virtual addresses, the system calls this memory processing module to complete the allocation task.

[0102] The memory processing module allocates an appropriate size physical memory block from the available physical memory pool based on the size of the data to be processed, and then establishes a mapping relationship between the virtual expert address and this physical memory block.

[0103] By using a dedicated memory processing module within the processor, the efficiency of memory allocation and mapping can be improved, reducing the intervention of the operating system and thus enhancing the overall performance of the system.

[0104] In some optional implementations of this embodiment, the memory processing module is a page manager core or a hardware page management engine.

[0105] Specifically, the PageManager Core is a software-implemented memory management module that runs as part of the operating system kernel. The PageManager Core is responsible for managing the mapping between virtual and physical memory, handling page faults, and allocating and releasing physical memory pages.

[0106] A hardware page manager engine (HW PageManager Engine) is a hardware-implemented memory management module integrated into the processor or memory controller. It directly handles the translation from virtual addresses to physical addresses without software intervention, thereby improving memory access efficiency.

[0107] The memory processing module in this embodiment can use a software-implemented page manager core or a hardware-implemented page management engine. Both can effectively allocate actual physical space for virtual expert addresses and establish a mapping relationship between virtual addresses and physical addresses. The specific configuration can be flexibly set according to the actual application scenario.

[0108] Figure 5 This is a schematic diagram illustrating data multicasting during the Dispatch phase, as provided in an embodiment of this disclosure. Figure 6 To and Figure 5 A diagram illustrating the first routing information carried by the relevant data write command.

[0109] like Figure 5-6 As shown, the example illustrates a communication mechanism that includes four processors (Rank), each Rank receiving five input tokens, and each Rank corresponding to two expert models.

[0110] Each Rank is based on each Input Token and carries the first routing information corresponding to that Token. This first routing information includes a number bitmap and an expert bitmap.

[0111] Taking Token2 as an example, Token2 from Rank0 needs to be sent to Rank1 and Rank3 according to the number bitmap. The expert bitmap is used by the Rank GPU at the communication receiving end. When a processor includes only one expert model, the number bitmap and expert bitmap can be unified. It should be noted that the number bitmap and expert bitmap are only a schematic representation of the first routing information; other representation methods can also be used, such as data structure information corresponding to input data and experts.

[0112] like Figure 5 As shown, Rank0 invokes the data write instruction `inc_v.store`. In Rank0, this data write instruction is converted into a data packet. This data packet includes a data write request, first routing information (including a number bitmap and an expert bitmap) corresponding to the data to be processed (Token2), first address information, and the data to be processed (Token2). Rank0 then sends the data packet to the switch.

[0113] When the switch receives a data packet sent by Rank0, it can multicast the data packet to the corresponding communication receivers Rank1 and Rank3 based on the number bitmap, thus completing the multicast transmission of the data packet.

[0114] For the receiving ends Rank1 and Rank3, based on the expert bitmap, experts 2 and 3 within Rank1 and experts 6 and 7 within Rank3 write data into multiple actual physical spaces corresponding to the virtual expert addresses pointed to by the first address information. After the data writing operation is completed, Rank1 and Rank3 then send the write response back to the switch, which in turn sends the write response back to the communication requesting end processor Rank0.

[0115] In some optional implementations of this embodiment, such as Figure 4 As shown, the ensemble communication method based on the hybrid expert model includes not only steps 401 to 405 as described above, but also the following steps:

[0116] Step 406: Receive data read instruction, which includes second address information and weight allocation information.

[0117] Specifically, the communication receiver receives a data read instruction, which is used to read specific data from stored data and perform calculations. The data read instruction includes second address information and weight allocation information. The second address information indicates the storage location of the data to be read, and the weight allocation information indicates how to weight the data from multiple expert models at the communication receiver for reading.

[0118] Step 407: Based on the second address information, read the data to be processed from the actual physical address of the virtual expert address in the virtual address space.

[0119] Specifically, the communication receiving end locates the virtual expert address in the virtual address space based on the second address information, then finds the corresponding actual physical address through the virtual expert address, and reads the data to be processed from the actual physical address.

[0120] Step 408: Calculate the response data based on the weight allocation information.

[0121] Specifically, the receiving end performs weighted calculations on the data to be processed read from the expert model corresponding to the receiving end, based on the weight allocation information, to obtain the response data. The weight allocation information can be a set of weight values ​​used to indicate the importance or contribution of different data in the calculation.

[0122] It should be noted that the process of calculating response data based on weight allocation information requires hardware to complete the calculation. If the hardware does not support it, it can also be implemented in software. In this case, it is necessary to maintain the real physical memory corresponding to the virtual expert address space, complete the above response data calculation based on the virtual expert address space in the expert model, and write it into the actual physical space pointed to by the virtual expert address.

[0123] This disclosure enables the weighted fusion of the outputs of different expert models in a hybrid expert model to obtain the final model output.

[0124] In some optional implementations of this embodiment, the data read instruction further includes second routing information, and the virtual address space includes a virtual number address space and a virtual expert address space.

[0125] Step 407: Based on the second address information, read the data to be processed from the actual physical address of the virtual expert address in the virtual address space. This includes the following steps:

[0126] Based on the second address information, determine the virtual number address of the virtual number address space; based on the second routing information, determine the virtual expert address from the virtual expert address space through the mapping between the virtual number address space and the virtual expert address space; read the data to be processed from the actual physical address of the virtual expert address.

[0127] Specifically, the receiving end locates the corresponding virtual number address in the virtual number address space based on the second address information. When a virtual number address corresponds to multiple virtual expert address spaces, the receiving end needs to determine which virtual expert address space to access based on the second routing information. The receiving end selects one of the multiple virtual expert address spaces corresponding to the virtual number address space based on the second routing information, and then determines the virtual expert address within that virtual expert address space.

[0128] The receiving end uses the virtual expert address to find the corresponding actual physical address, and then reads the data to be processed from the actual physical address.

[0129] This disclosure enables flexible reading of data from a specific expert model within multiple virtual expert address spaces, providing support for the computation of hybrid expert models.

[0130] It is understandable that the operation corresponding to the data read instruction is the reverse process of the above data write instruction. The specific implementation processes of the two are similar, and will not be described in detail here.

[0131] Figure 7 This is a schematic diagram illustrating data reduction during the Combine stage, as provided in an embodiment of this disclosure. Figure 8 To and Figure 7 A diagram illustrating the second routing information carried by the relevant data read command.

[0132] like Figure 7-8 In this embodiment, we will still use the example of communication involving 4 processor Ranks, with each processor Rank receiving 5 input tokens and each processor Rank corresponding to 2 expert models.

[0133] Taking Token2 as an example, Rank0 invokes the data read instruction inc_v.ldr, and AI Core sends the data read instruction to Rank0. Rank0 converts this data read instruction into a data packet, which includes a data read request and carries the second routing information (including a number bitmap and an expert bitmap), second address information, and weight allocation information corresponding to the data Token2 to be processed. Rank0 then sends the data packet to the switch.

[0134] When the switch receives a data packet sent by Rank0, it can multicast the data packet to the corresponding communication receivers Rank1 and Rank3 based on the number bitmap, thus completing the multicast transmission of the data packet.

[0135] For the receivers Rank1 and Rank3, based on the expert bitmap, multiple expert data needs to be read at the port nodes of experts 2 and 3 within Rank1 and experts 6 and 7 within Rank3, and the response data needs to be calculated based on the weight allocation information.

[0136] That is, Token2.rank3.data=Token2.Expert6.weigth*Token2.Expert6_data+Token2.Expert7.weigth*Token2.Expert7_data.

[0137] Rank1 and Rank3 return the completed response data to the switch. The switch receives the corresponding data from Rank1 and Rank3 and completes the corresponding Reduce calculation task. Finally, the Reduce calculation result is returned to Rank0 and the data is written to the memory that Rank0 has created in advance.

[0138] This disclosure's embodiments pre-create a virtual address space in each execution entity, rather than requesting actual physical memory. When the execution entity at the communication receiving end receives a data write instruction (i.e., a data packet) and accesses the virtual expert address of the virtual address space according to the first address information in the data write instruction, it allocates actual physical space as needed in the address pointed to by the first address information, based on the size of the data to be processed in the data write instruction. The aggregate communication method provided in this disclosure does not require address information synchronization during communication, saving time and improving processing performance. Furthermore, the communication method provided in this disclosure requests the corresponding actual physical space based on the size of the data to be processed only when the data packet arrives and accesses the virtual expert address, preventing excessive memory usage and resource explosion, further improving processing performance.

[0139] Further reference Figure 9 As an implementation of the methods shown in the above figures, this disclosure provides an embodiment of a collective communication system based on a hybrid expert model. This communication system embodiment is similar to... Figure 2 Corresponding to the method embodiments shown, this communication system can be specifically applied to various electronic devices.

[0140] like Figure 9 As shown, the communication system 900 of this embodiment may include: multiple processors 901 and a switch 902. Each processor 901 can act as a communication requesting end to initiate communication requests to other processors 901, or as a communication receiving end to receive communication requests sent by other processors 901.

[0141] When processor 901 receives and sends instructions as a communication requesting end, processor 901 sends the instructions to switch 902.

[0142] The switch 902 is configured to receive instructions, which include data to be processed, address information, and data routing information; and based on the data routing information, multicast the instructions to the corresponding processor.

[0143] When the processor 902 acts as a communication receiver, it is configured to respond to the received instruction as a data write instruction, access the virtual expert address in the pre-created virtual address space according to the address information, and apply for the corresponding actual physical space for the virtual expert address based on the size of the data to be processed; and write the data to be processed into the actual physical space.

[0144] In this embodiment, in the communication system 900: the specific processing of the processor 901 as the communication receiver and the resulting technical effects can be referred to separately. Figure 2 The relevant descriptions of steps 201-204 in the corresponding embodiments will not be repeated here.

[0145] The communication system provided in this disclosure transfers the data multicast operation from the processor to the switch, enabling the switch to multicast instructions based on the data routing information carried in the instructions. This reduces the processor's overhead for multicasting data and further improves the computational efficiency of the communication system. The solution disclosed herein is highly scalable, can meet the acceleration requirements of various aggregated communication operations, and is applicable to various implementation methods such as internal interconnection of hosts and inter-host networking.

[0146] In some optional implementations of this embodiment, the virtual address space includes a virtual numbered address space and a virtual expert address space. Specifically, the processor 901 is configured to, in response to a received instruction being a data write instruction, access the virtual numbered address of the pre-created virtual numbered address space based on address information; and access the virtual expert address of the virtual expert address space through a mapping between the virtual numbered address space and the virtual expert address space.

[0147] In some optional implementations of this embodiment, the routing information includes an expert bitmap and a numbering bitmap. Specifically, the switch 902 is configured to multicast instructions to the corresponding processor 901 based on the numbering bitmap. The processor 901 is configured to, in response to a received instruction being a data write instruction, and in response to at least two virtual expert address spaces corresponding to a virtual numbering address space, access the virtual numbering address of a pre-created virtual numbering address space based on address information; and, according to the expert bitmap, access the virtual expert address of the virtual expert address space through the mapping between the virtual numbering address space and the virtual expert address space.

[0148] In this embodiment, the specific processing of the processor 801 as a communication receiver and the resulting technical effects can be referred to the relevant descriptions of steps 402-403 in the above method embodiment, and will not be repeated here.

[0149] Figure 10 This is a structural diagram of a physical memory allocation device provided in an embodiment of the present disclosure.

[0150] like Figure 10As shown, when the processor 901 at the communication receiving end accesses the virtual expert address in the pre-created virtual address space according to the first address information, it attempts to write the data to be processed into the virtual expert address. The communication interface 1012 within the processor 901 routes to the memory management unit 1022 within the processor 901 through the address access bus 1003 of the virtual address space; the memory management unit 1022 within the processor 901 triggers a PageFault interrupt to the processor driver 1011 of the host 1001; the processor driver 1011 of the host 1001 allocates physical memory for the virtual expert address and writes the page table into the processor's page table memory 1032; finally, the communication interface 1012 can access the actual physical space corresponding to the virtual expert address.

[0151] Figure 11 This is a structural diagram of another physical memory allocation device provided in an embodiment of the present disclosure.

[0152] In some optional implementations of this embodiment, such as Figure 11 As shown, the processor 901 includes a communication interface 1012, a memory management unit 1022, and a memory processing module 1042. The communication interface 1012 is configured to access the memory management unit 1022 via a virtual expert address access bus. The memory management unit 1022 is configured to trigger a page fault interrupt in response to writing data to be processed to the virtual expert address, and to send the interrupt to the memory processing module 1042. The memory processing module 1042 is configured to allocate actual physical space for the virtual expert address, write the mapping relationship between the virtual expert address and the actual physical space into a page table memory, and obtain the actual physical space corresponding to the virtual expert address based on the page table memory.

[0153] Specifically, such as Figure 11 As shown, when the processor 1002 at the communication receiving end accesses the virtual expert address in the pre-created virtual address space according to the first address information, it attempts to write the data to be processed into the virtual expert address. The communication interface 1012 within the processor 1002 carries the address access bus of the virtual address space and routes it to the memory management unit 1022 within the processor 1002. The memory management unit 1022 within the processor 1002 triggers a PageFault interrupt, which is sent to the memory processing module 1042 within the processor 1002 for processing. The memory processing module 1042 within the processor 1002 allocates physical memory for the virtual expert address and writes the page table into the processor's page table memory 1032. Finally, the communication interface 1012 can access the actual physical space corresponding to the virtual expert address. By using a dedicated memory processing module within the processor, the efficiency of memory allocation and mapping can be improved, the intervention of the operating system can be reduced, and thus the overall performance of the system can be improved.

[0154] In some optional implementations of this embodiment, the memory processing module is a page manager core or a hardware page management engine.

[0155] Specifically, the PageManager Core is a software-implemented memory management module that runs as part of the operating system kernel. The PageManager Core is responsible for managing the mapping between virtual and physical memory, handling page faults, and allocating and releasing physical memory pages.

[0156] A hardware page manager engine (HW PageManager Engine) is a hardware-implemented memory management module integrated into the processor or memory controller. It directly handles the translation from virtual addresses to physical addresses without software intervention, thereby improving memory access efficiency.

[0157] The memory processing module 1042 in this embodiment can use a software-implemented page manager core or a hardware-implemented page management engine. Both can effectively allocate actual physical space for virtual expert addresses and establish a mapping relationship between virtual addresses and physical addresses. The specific configuration can be flexibly set according to the actual application scenario.

[0158] In some optional implementations of this embodiment, the processor 901 is further configured to respond to a data read instruction, which also includes weight allocation information. Based on the address information, it reads the data to be processed from the actual physical address of the virtual expert address in the virtual address space, and calculates the response data based on the weight allocation information. The switch 902 is further configured to perform reduction calculation on the response data and feed back the reduction calculation result to the instruction sender.

[0159] In this embodiment, the specific processing procedure of the processor 901 in the communication system 900 receiving the data read and the resulting technical effects can be referred to the relevant descriptions of steps 406 to 408 in the above method embodiment, and will not be repeated here.

[0160] It should be noted that other details of the collective communication system provided in this disclosure are the same as any of the embodiments of the collective communication method described above, and will not be repeated here.

[0161] This embodiment is a communication system embodiment corresponding to the above method embodiment. The communication system provided in this embodiment pre-creates a virtual address space in each processor instead of allocating actual physical memory. When the processor at the communication receiving end receives a data write instruction (i.e., a data packet) and accesses the virtual expert address of the virtual address space according to the address information in the data write instruction, it allocates actual physical space in the address pointed to by the address information on demand, based on the size of the data to be processed in the data write instruction. The collective communication system provided in this disclosure does not require synchronization of address information during communication, saving time overhead and improving processing performance. Moreover, the communication system provided in this disclosure only allocates the corresponding actual physical space based on the size of the data to be processed when the data packet arrives and accesses the virtual expert address, which can prevent excessive memory usage and resource explosion, further improving processing performance. Furthermore, the communication system provided in this disclosure transfers the data multicast operation from the processor to the switch, allowing the switch to multicast the instruction according to the data routing information carried by the instruction, reducing the processor's multicast data overhead and further improving the computational efficiency of the communication system. This publicly disclosed solution is highly scalable and can meet the acceleration requirements of various collective communication operations. It is also applicable to various implementation methods such as internal interconnection of hosts and inter-host networking interconnection.

[0162] Figure 12 A collection communication device 1200 based on a hybrid expert model is provided as an embodiment of this disclosure. Further references... Figure 12 As an implementation of the methods shown in the above figures, this disclosure provides an embodiment of a collective communication device based on a hybrid expert model. This device embodiment is similar to... Figure 2 Corresponding to the method embodiments shown, this device can be specifically applied to various electronic devices.

[0163] like Figure 12 As shown, the aggregate communication device 1200 based on a hybrid expert model in this embodiment may include a receiving unit 1201, an access unit 1202, an application unit 1203, and a writing unit 1204. The receiving unit 1201 is configured to receive a data writing instruction, which includes data to be processed and first address information. The access unit 1202 is configured to access a virtual expert address in a pre-created virtual address space based on the first address information. The application unit 1203 is configured to apply for corresponding physical space for the virtual expert address based on the size of the data to be processed. The writing unit 1204 is configured to write the data to be processed into the physical space.

[0164] In this embodiment, the specific processing of the receiving unit 1201, access unit 1202, request unit 1203, and writing unit 1204 in the hybrid expert model-based aggregate communication device 1200, and the resulting technical effects, can be found in reference to [reference needed]. Figure 2 The relevant descriptions of steps 201-204 in the corresponding embodiments will not be repeated here.

[0165] In some optional implementations of this embodiment, the virtual address space is created based on the total size of all data to be processed in the collection communication.

[0166] In some optional implementations of this embodiment, the size of the virtual address space is determined by the total number of data to be processed in the aggregate communication and the size of the data to be processed.

[0167] In some optional implementations of this embodiment, the virtual address space includes a virtual number address space and a virtual expert address space; the access unit 1202 is further configured to: access the virtual number address of the pre-created virtual number address space according to the first address information; and access the virtual expert address of the virtual expert address space through the mapping between the virtual number address space and the virtual expert address space.

[0168] In some optional implementations of this embodiment, the data writing instruction further includes first routing information, and the access unit 1202 is further configured to: in response to the virtual number address space corresponding to at least two virtual expert address spaces, access the virtual expert address of the virtual expert address space through the mapping between the virtual number address space and the virtual expert address space according to the first routing information.

[0169] In some optional implementations of this embodiment, the application unit 1203 is further configured to: trigger a page fault interrupt in response to writing the data to be processed to the virtual expert address; allocate actual physical space to the virtual expert address based on the page fault interrupt, and write the mapping relationship between the virtual expert address and the actual physical space into the page table memory; and obtain the actual physical space corresponding to the virtual expert address based on the page table memory.

[0170] In some optional implementations of this embodiment, the application unit 1203 is further configured to allocate actual physical space for the virtual expert address through the memory processing module within the processor.

[0171] In some optional implementations of this embodiment, the memory processing module is a page manager core or a hardware page management engine.

[0172] In some optional implementations of this embodiment, the receiving unit 1201 is further configured to receive a data read instruction, which includes second address information and weight allocation information. The device 1200 also includes a data reading unit and a calculation unit. The data reading unit is configured to read data to be processed from the actual physical address of the virtual expert address in the virtual address space according to the second address information; the calculation unit is configured to calculate response data based on the weight allocation information on the data to be processed.

[0173] In some optional implementations of this embodiment, the data read instruction further includes second routing information, the virtual address space includes a virtual number address space and a virtual expert address space, and the data reading unit is further configured to: determine the virtual number address of the virtual number address space according to the second address information; determine the virtual expert address from the virtual expert address space according to the second routing information through the mapping between the virtual number address space and the virtual expert address space; and read the data to be processed from the actual physical address of the virtual expert address.

[0174] This embodiment exists as a device embodiment corresponding to the above method embodiment. The aggregate communication device based on a hybrid expert model provided in this embodiment, by pre-creating a virtual address space instead of allocating actual physical memory, allocates actual physical space as needed in the address pointed to by the first address information only when a data write instruction (i.e., a data packet) is received and the virtual expert address of the virtual address space is accessed according to the first address information in the data write instruction, based on the size of the data to be processed in the data write instruction. The aggregate communication device provided in this disclosure does not need to synchronize address information during communication, saving time overhead and improving processing performance; moreover, the communication device provided in this disclosure only allocates the corresponding actual physical space according to the size of the data to be processed when the data packet arrives and accesses the virtual expert address, which can prevent excessive memory occupation and resource explosion, further improving processing performance.

[0175] According to embodiments of the present disclosure, the present disclosure also provides an electronic device, the electronic device comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to implement the aggregate communication method based on the hybrid expert model described in any of the above embodiments.

[0176] According to embodiments of this disclosure, this disclosure also provides a readable storage medium storing computer instructions that enable a computer to implement the collection communication method based on a hybrid expert model as described in any of the above embodiments when executed.

[0177] According to embodiments of this disclosure, this disclosure also provides a computer program product that, when executed by a processor, can implement the set communication method based on a hybrid expert model as described in any of the above embodiments.

[0178] Figure 13 A schematic block diagram of an example electronic device 1300 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0179] like Figure 13 As shown, the electronic device 1300 includes a computing unit 1301, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 1302 or a computer program loaded from a storage unit 1308 into a random access memory (RAM) 1303. The RAM 1303 may also store various programs and data required for the operation of the electronic device 1300. The computing unit 1301, ROM 1302, and RAM 1303 are interconnected via a bus 1304. An input / output (I / O) interface 1305 is also connected to the bus 1304.

[0180] Multiple components in electronic device 1300 are connected to I / O interface 1305, including: input unit 1306, such as keyboard, mouse, etc.; output unit 1307, such as various types of monitors, speakers, etc.; storage unit 1308, such as disk, optical disk, etc.; and communication unit 1309, such as network card, modem, wireless transceiver, etc. Communication unit 1309 allows electronic device 1300 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0181] The computing unit 1301 can be various general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 1301 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 1301 performs the various methods and processes described above, such as a hybrid expert model-based collective communication method. For example, in some alternative implementations of this embodiment, the hybrid expert model-based collective communication method can be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 1308. In some alternative implementations of this embodiment, part or all of the computer program can be loaded and / or installed on the electronic device 1300 via ROM 1302 and / or communication unit 1309. When the computer program is loaded into RAM 1303 and executed by the computing unit 1301, one or more steps of the hybrid expert model-based collective communication method described above can be performed. Alternatively, in other embodiments, computing unit 1301 may be configured by any other suitable means (e.g., by means of firmware) to perform a collection communication method based on a hybrid expert model.

[0182] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0183] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0184] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0185] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0186] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with embodiments of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0187] Computer systems can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system to address the shortcomings of traditional physical hosts and Virtual Private Server (VPS) services, such as high management difficulty and weak business scalability.

[0188] According to the technical solution of this disclosure, instead of allocating actual physical memory, a virtual address space is pre-created in each processor. When the processor of the communication receiving end receives a data write instruction (i.e., a data packet) and accesses the virtual expert address of the virtual address space according to the address information in the data write instruction, the actual physical space is allocated on demand in the address pointed to by the address information based on the size of the data to be processed in the data write instruction. The aggregated communication scheme provided in this disclosure does not require synchronization of address information during communication, saving time overhead and improving processing performance. Moreover, the communication system provided in this disclosure only allocates the corresponding actual physical space based on the size of the data to be processed when the data packet arrives and accesses the virtual expert address, which can prevent excessive memory usage and resource explosion, further improving processing performance. Furthermore, the scheme provided in this disclosure transfers the data multicast operation from the processor to the switch, enabling the switch to multicast the instruction according to the data routing information carried by the instruction, reducing the processor's multicast data overhead and further improving the computational efficiency of the communication system. The solution of this disclosure has strong scalability, can meet the acceleration requirements of various aggregated communication operations, and is applicable to various implementation methods such as internal interconnection of hosts and inter-host networking interconnection.

[0189] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this disclosure can be achieved, and this is not limited herein.

[0190] The specific embodiments described above do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.

Claims

1. A ensemble communication method based on a hybrid expert model, the method being applied at a communication receiver, comprising: Receive a data write instruction, the data write instruction including data to be processed and first address information; Access the virtual number address of the pre-created virtual number address space according to the first address information; access the virtual expert address of the virtual expert address space through the mapping between the virtual number address space and the virtual expert address space; Based on the size of the data to be processed, apply for the corresponding actual physical space for the virtual expert address; The data to be processed is written into the actual physical space.

2. The method according to claim 1, wherein, The data writing instruction further includes first routing information, and the step of accessing the virtual expert address in the virtual expert address space through the mapping between the virtual number address space and the virtual expert address space includes: In response to the virtual number address space corresponding to at least two virtual expert address spaces, the virtual expert address of the virtual expert address space is accessed through the mapping between the virtual number address space and the virtual expert address space according to the first routing information.

3. The method according to claim 1, wherein, Based on the size of the data to be processed, allocate corresponding physical space for the virtual expert address, including: In response to writing the data to be processed to the virtual expert address, a page error interruption is triggered; Based on the page fault interruption, actual physical space is allocated to the virtual expert address, and the mapping relationship between the virtual expert address and the actual physical space is written into the page table memory; The actual physical space corresponding to the virtual expert address is obtained based on the page table memory.

4. The method according to claim 3, wherein, The process of allocating actual physical space for the virtual expert address includes: The processor allocates actual physical space for the virtual expert address through the memory processing module within the processor. The memory processing module is either a page manager core or a hardware page management engine.

5. The method according to claim 1, further comprising: Receive a data read instruction, the data read instruction including second address information and weight allocation information; Based on the second address information, the data to be processed is read from the actual physical address of the virtual expert address; The response data is obtained by calculating the data to be processed based on the weight allocation information.

6. The method according to claim 5, wherein, The data read instruction further includes second routing information. The step of reading the data to be processed from the actual physical address of the virtual expert address according to the second address information includes: determining the virtual number address of the virtual number address space according to the second address information. Based on the second routing information, the virtual expert address is determined from the virtual expert address space through the mapping between the virtual number address space and the virtual expert address space; The data to be processed is read from the actual physical address of the virtual expert address.

7. The method according to any one of claims 1-6, wherein, The virtual number address space and the virtual expert address space are created based on the total size of all the data to be processed in the collective communication.

8. A collective communication system based on a hybrid expert model, the collective communication system comprising multiple processors and switches, wherein, The switch is configured to receive instructions, which include data to be processed, address information, and data routing information; and to multicast the instructions to the corresponding processor based on the data routing information. The processor is configured to, in response to the received instruction being a data write instruction, access the virtual number address of a pre-created virtual number address space based on the address information; access the virtual expert address of the virtual expert address space through the mapping between the virtual number address space and the virtual expert address space; and request corresponding actual physical space for the virtual expert address based on the size of the data to be processed. And write the data to be processed into the actual physical space.

9. The communication system according to claim 8, wherein, The routing information includes an expert bitmap and a numbered bitmap. The switch is specifically configured to multicast the instructions to the corresponding processor based on the numbered bitmap; The processor is specifically configured to, in response to the received instruction being a data write instruction and in response to the virtual number address corresponding to at least two virtual expert address spaces, access the virtual number address of a pre-created virtual number address space based on the address information; and, according to the expert bitmap, access the virtual expert address of the virtual expert address space through the mapping between the virtual number address space and the virtual expert address space.

10. The communication system according to claim 8, wherein, The processor includes a communication interface, a memory management unit, and a memory processing module. The communication interface is configured to access the memory management unit based on the virtual expert address access bus; The memory management unit is configured to trigger a page fault interrupt in response to writing the data to be processed to the virtual expert address, and to send the interrupt to the memory processing module; The memory processing module is configured to allocate actual physical space for the virtual expert address, write the mapping relationship between the virtual expert address and the actual physical space into the page table memory, and obtain the actual physical space corresponding to the virtual expert address based on the page table memory.

11. The communication system according to claim 8, wherein, The processor is further configured to respond to the instruction as a data read instruction, the data read instruction further including weight allocation information, read the data to be processed from the actual physical address of the virtual expert address according to the address information, and calculate the response data based on the data to be processed based on the weight allocation information; The switch is also configured to perform reduction calculations on the response data and feed back the reduction calculation results to the instruction sending end.

12. A collective communication device based on a hybrid expert model, comprising: The receiving unit is configured to receive a data write instruction, wherein the data write instruction includes data to be processed and first address information; The access unit is configured to access the virtual number address of a pre-created virtual number address space based on the first address information; and to access the virtual expert address of the virtual expert address space through the mapping between the virtual number address space and the virtual expert address space. The application unit is configured to apply for the corresponding physical space for the virtual expert address based on the size of the data to be processed. The writing unit is configured to write the data to be processed into the actual physical space.

13. An electronic device, comprising: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the collection communication method according to any one of claims 1-7.

14. A non-transitory computer-readable storage medium storing computer instructions for causing the computer to perform the collective communication method according to any one of claims 1-7.

15. A computer program product comprising a computer program that, when executed by a processor, implements the steps of the collective communication method according to any one of claims 1-7.

Images