Expert-parallel processing method and system, electronic device, and storage medium

Abstract

The present application discloses an expert-parallel processing method and system, an electronic device, and a storage medium, which solve the problem of load imbalance on computing cards that carry expert execution, thereby improving the processing efficiency of the computing cards. In the expert-parallel processing system, W experts are deployed from a host node to N computing cards by means of expert replication, wherein each of the N computing cards comprises at least two experts having different expert weights. In an embodiment of the present application, on the basis of text processing units respectively activated by the W experts, M text processing units are assigned to at least two experts on each of the N computing cards, so as to obtain text processing units respectively activated by the at least two experts on each of the N computing cards. The total number of text processing units activated by the at least two experts and corresponding to a same expert weight across different computing cards is equal to the number of text processing units corresponding to the same expert weight among the W experts obtained by routing partitioning.

Images