Circuit board, computing system, computer cabinet and computing method

By setting up accelerator card units, switching units, and extended storage units on the circuit board, the storage and transmission of expert models are optimized, solving the problems of high resource consumption and low scheduling efficiency in existing technologies, and achieving efficient computing power utilization and resource scheduling.

WO2026237434A1PCT designated stage Publication Date: 2026-11-19INSPUR SUZHOU INTELLIGENT TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
PCT/CN2026/072584
Authority / Receiving Office
WO · WO
Patent Type
Applications
Current Assignee / Owner
Priority Date
2025-05-15
Filing Date
2026-01-14
Publication Date
2026-11-19

AI Technical Summary

Technical Problem

In existing technologies, increasing computing power by adding graphics processing units (GPUs) consumes a lot of resources and has low timeliness and efficiency in scheduling server resources, which restricts the development of artificial intelligence technology.

Method used

By setting up an accelerator card unit, a switching unit, an extended storage unit, and a monitoring unit on the circuit board, and utilizing different memory types and bus transmission speeds, efficient scheduling and resource optimization of the expert model can be achieved. This includes storing high-frequency model parameters in the first memory, storing low-frequency model parameters in the extended storage unit, and transmitting call frequency information in a timely manner through the monitoring unit.

Benefits of technology

It improved the utilization rate of server computing power, reduced resource consumption, improved the timeliness and efficiency of resource scheduling, and solved the bottleneck problem of resource scheduling.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN2026072584_19112026_PF_FP_ABST
    Figure CN2026072584_19112026_PF_FP_ABST
Patent Text Reader

Abstract

The present application provides a circuit board, a computing system, a computer cabinet and a computing method, which can be applied to the technical field of artificial intelligence. The circuit board comprises: a switching unit; an accelerator card unit, connected to the switching unit, wherein a first memory of the accelerator card unit is used for storing a first model parameter of a first expert model; an extended storage unit, connected to the switching unit, wherein the extended storage unit is used for storing a second model parameter of a second expert model, the calling frequency of the first expert model is different from the calling frequency of the second expert model; and a monitoring unit, connected to the accelerator card unit, and connected to a first control device by means of a plurality of buses, wherein the data transmission speeds of the plurality of buses are different. The circuit board can fully exert the computing power of the accelerator card unit, thereby improving the computing power of a server without the need for additional circuit boards, saving resource consumption, improving the timeliness of resource scheduling, and improving the efficiency of resource scheduling.
Need to check novelty before this filing date? Find Prior Art

Description

Circuit boards, computing systems, computer cabinets, and computing methods

[0001] Cross-references to related applications

[0002] This application claims priority to Chinese Patent Application No. 202510623229.9, filed on May 15, 2025, entitled "Circuit Board, Computing System, Computer Cabinet and Computing Method", the entire contents of which are incorporated herein by reference. Technical Field

[0003] This application relates to the field of memory expansion technology, and in particular to a data read / write method, system, device and storage medium. Background Technology

[0004] We are currently in an era of rapid development in artificial intelligence technology, and the rapid development of large-scale models places higher demands on server computing power than ever before. However, related technologies mainly increase computing power by adding graphics processing units (GPUs). This method consumes a lot of resources, has poor timeliness in scheduling server resources, and is inefficient, thus hindering the development of artificial intelligence technology. Summary of the Invention

[0005] In view of the above problems, this application provides a circuit board, a computing system, a computer cabinet, and a computing method.

[0006] According to a first aspect of this application, a circuit board is provided, comprising: a switching unit; an accelerator card unit connected to the switching unit, wherein a first memory of the accelerator card unit is used to store first model parameters of a first expert model; an extended storage unit connected to the switching unit, wherein the extended storage unit is used to store second model parameters of a second expert model; wherein the calling frequency of the first expert model is different from that of the second expert model; a monitoring unit connected to the accelerator card unit and connected to a first control device via multiple buses; wherein the data transmission speeds of the multiple buses are different, so that the monitoring unit can transmit information on the calling frequency of different expert models by the accelerator card unit to the first control device at different transmission speeds.

[0007] According to a second aspect of this application, a computing system is provided, comprising: a plurality of the aforementioned circuit boards; and a first control device for collecting call frequency information of expert models configured in the plurality of circuit boards, and determining a scheduling strategy for the plurality of circuit boards based on the received call frequency information.

[0008] According to a third aspect of this application, a computer cabinet is provided, comprising: a plurality of computing systems as described above; and a second control device for receiving call frequency information from a first control device of the plurality of computing systems, and determining a scheduling strategy for the plurality of computing systems based on the received call frequency information.

[0009] According to a fourth aspect of this application, a computation method is provided, applied to a circuit board; the method includes: an accelerator card unit of the circuit board obtaining first model parameters from the memory of the accelerator card unit of the circuit board, and / or obtaining second model parameters from the extended storage unit of the circuit board via a switching unit of the circuit board; the accelerator card unit of the circuit board, based on the first model parameters and / or the second model parameters, calling at least one of a first expert model and a second expert model to perform a model processing task, wherein the calling frequency of the first expert model is higher than the calling frequency of the second expert model; and a monitoring unit of the circuit board transmitting information on the calling frequency of different expert models by the accelerator card unit of the circuit board to a first control device at different transmission speeds.

[0010] According to embodiments of this application, during operation, the accelerator card unit can promptly obtain the first model parameters from the first memory, or obtain the second model parameters from an additional extended storage unit via a switching unit. Thus, by expanding the memory accessible to the accelerator card unit, this application avoids the accelerator card unit spending excessive time retrieving model parameters from other storage spaces, reducing the time required for the accelerator card unit to retrieve model parameters. This allows for full utilization of the accelerator card unit's computing power, thereby increasing the server's computing power without adding circuit boards and saving resource consumption.

[0011] Based on this, the monitoring unit connects to the accelerator card unit and can collect call frequency information of expert models with different call frequencies. Simultaneously, the monitoring unit can also connect to the first control device via multiple buses and transmit different call frequency information to the first control device at different speeds via these buses. In this way, information requiring rapid processing (such as the call frequency information of frequently called expert models) can be transmitted to the first control device at a faster speed, enabling the first control device to perform timely resource scheduling for the accelerator card unit, improving the timeliness and efficiency of resource scheduling. Attached Figure Description

[0012] The above-mentioned contents, other objects, features and advantages of this application will become clearer from the following description of embodiments with reference to the accompanying drawings, in which:

[0013] Figure 1 shows a schematic diagram of a circuit board according to a first embodiment of this application.

[0014] Figure 2 shows a schematic diagram of a circuit board according to a second embodiment of this application.

[0015] Figure 3 shows a schematic diagram of a circuit board according to a third embodiment of this application.

[0016] Figure 4 shows a schematic diagram of a computing system according to an embodiment of this application.

[0017] Figure 5 shows a schematic diagram of a first circuit board and a second circuit board according to a first embodiment of this application.

[0018] Figure 6 shows a schematic diagram of a first circuit board and a second circuit board according to a second embodiment of this application.

[0019] Figure 7 shows a schematic diagram of the connection of units in different circuit boards according to embodiments of this application.

[0020] Figure 8 shows a schematic diagram of the connection between the first control device and the circuit board according to an embodiment of this application.

[0021] Figure 9 shows a schematic diagram of a computer cabinet according to a first embodiment of this application.

[0022] Figure 10 shows a schematic diagram of a computer cabinet according to a second embodiment of this application.

[0023] Figure 11 shows a connection diagram of a computer cabinet according to an embodiment of this application.

[0024] Figure 12 shows a schematic diagram of the connection between the circuit boards of the first computing system and the second computing system according to an embodiment of this application.

[0025] Figure 13 shows a schematic diagram of the calculation method according to an embodiment of this application. Detailed Implementation

[0026] The embodiments of this application will now be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of this application. In the following detailed description, numerous specific details are set forth to provide a thorough understanding of the embodiments of this application for ease of explanation. However, it will be apparent that one or more embodiments may be implemented without these specific details. Furthermore, descriptions of well-known structures and technologies are omitted in the following description to avoid unnecessarily obscuring the concepts of this application.

[0027] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the scope of this application. The terms “comprising,” “including,” etc., as used herein indicate the presence of features, steps, operations, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, or components.

[0028] All terms used herein (including technical and scientific terms) have the meanings commonly understood by those skilled in the art, unless otherwise defined. It should be noted that the terms used herein are to be interpreted in a manner consistent with the context of this specification, and not in an idealized or overly rigid way.

[0029] When using expressions such as "at least one of A, B, and C", they should generally be interpreted in accordance with the meaning that is commonly understood by a person skilled in the art (e.g., "a system having at least one of A, B, and C" should include, but is not limited to, a system having A alone, a system having B alone, a system having C alone, a system having A and B, a system having A and C, a system having B and C, and / or a system having A, B, and C, etc.).

[0030] Figure 1 shows a schematic diagram of a circuit board according to a first embodiment of this application.

[0031] As shown in Figure 1, the circuit board of this embodiment includes a switching unit SW, an acceleration card unit AI, a first memory AIS of the acceleration card unit, an extended storage unit ES, and a monitoring unit MON.

[0032] Accelerator card unit AI can be a GPU. Accelerator card unit AI can perform model processing tasks based on Mixed Expert Models (MoE). Accelerator card unit AI can deploy expert models and can utilize the deployed expert models to perform model processing tasks. Model processing tasks can refer to data processing tasks using expert models.

[0033] The accelerator card unit AI is electrically connected to its first memory AIS. The first memory AIS can store the first model parameters of the first expert model. The accelerator card unit AI can obtain the first model parameters from the first memory AIS. For example, the first memory AIS can be a Compression Attached Memory Module (CAMM). This CAMM supports multiple memory types. Specifically, the first memory AIS can be in the form of sixth-generation graphics-dedicated dual data rate synchronous dynamic random access memory (Graphics Double Data Rate 6, GDDR6), which has the advantages of high speed, high capacity, adjustable bandwidth, and low cost.

[0034] The accelerator card unit AI can also be electrically connected to a switching unit SW. For example, the switching unit SW can be a high-speed serial computer expansion bus standard signal (Peripheral Component Interconnect Express, PCIe) switch. The switching unit SW can be electrically connected to an expansion storage unit ES. The expansion storage unit ES can store the second model parameters of the second expert model. In this way, the accelerator card unit AI can obtain the second model parameters from the expansion storage unit ES via the switching unit SW. It should be understood that the accelerator card unit AI can simultaneously read the first model parameters from the first memory AIS and the second model parameters from the expansion storage unit ES, or it can read the first model parameters and the second model parameters separately at different times. This application does not limit this. Furthermore, in the embodiments of this application, in addition to the first memory AIS, the latency of the accelerator card unit AI accessing other memory (such as the memory of the expansion storage unit) can be increased to avoid the accelerator card unit AI obtaining data from other memory affecting the timeliness of obtaining data from the first memory AIS.

[0035] The accelerator card unit (AI) can also be electrically connected to the monitoring unit (MON). For example, the monitoring unit (MON) can be implemented based on an embedded controller (Baseboard Management Controller, BMC), a field-programmable gate array (FPGA), etc. The monitoring unit (MON) can collect relevant information about the accelerator card unit's AI calling the expert model, such as the working status information and call frequency information. The call frequency information refers to the frequency at which the accelerator card unit's AI calls the expert model per unit time. This call frequency can be determined based on the frequency of access to the model parameters stored on the circuit board. Furthermore, the monitoring unit (MON) can also be electrically connected to a first control device (CTD1) independent of the circuit board via multiple buses L1, ..., Ln, and send the monitored relevant information via multiple buses L1, ..., Ln, where n is an integer greater than 1. Since the data transmission speeds of the multiple buses L1, ..., Ln are different, buses with different transmission speeds can be selected to transmit the aforementioned relevant information. For example, multiple buses L1, ..., Ln may include a Universal Serial Bus (USB) bus, an LVDS (Low-Voltage Differential Signaling) tunneling protocol and interface (LTPI) bus, etc. In one embodiment of this application, the first expert model and the second expert model are invoked at different frequencies. The bus with a relatively higher transmission speed among the multiple buses L1, ..., Ln can be used to transmit the expert model with a relatively higher invocation frequency among the first and second expert models.

[0036] Based on this, during operation, the accelerator card unit AI can promptly obtain the first model parameters from the first memory AIS, and can also obtain the second model parameters from the additional extended storage unit ES via the switching unit SW. Thus, by expanding the memory accessible to the accelerator card unit AI, this application avoids the accelerator card unit AI spending excessive time retrieving model parameters from other storage spaces, thereby reducing the time spent retrieving model parameters and fully utilizing the computing power of the accelerator card unit AI. This, in turn, increases the server's computing power without adding circuit boards, saving resource consumption.

[0037] Furthermore, the monitoring unit MON is connected to the accelerator card unit AI and can collect call frequency information of the accelerator card unit AI when calling expert models with different call frequencies. Simultaneously, the monitoring unit MON can also be connected to the first control device CTD1 via multiple buses L1, ..., Ln, and transmit different call frequency information to the first control device CTD1 at different speeds via multiple buses L1, ..., Ln. In this way, information requiring rapid processing (such as the call frequency information of frequently called expert models) can be transmitted to the first control device CTD1 at a faster speed, enabling the first control device CTD1 to perform timely resource scheduling for the accelerator card unit AI, improving the timeliness and efficiency of resource scheduling.

[0038] In this embodiment, the ports of the switching unit SW can be configured to support different protocols based on firmware information. For example, some ports of the switching unit SW can be configured as PCIe ports, and others can be configured as Compute Express Link (CXL) ports. Thus, the switching unit SW can communicate with the accelerator card unit AI via PCIe signals and with the extended storage unit ES via CXL signals.

[0039] In this embodiment, the circuit board may further include a control unit. The control unit may be a central processing unit (CPU). The control unit is electrically connected to the switching unit SW and sends instructions to the switching unit SW. For example, the instructions sent by the control unit to the switching unit SW may be based on PCIe signals, specifically PCIe x16 signals.

[0040] The control unit can be pre-configured with an initial scheduling strategy, for example, the initial scheduling strategy can be sent to the control unit by the aforementioned first control device. This initial scheduling strategy can be implemented based on a predetermined program. Based on the scheduling strategy, the control unit can send scheduling instructions and other data to the switching unit SW. Under the control of the scheduling instructions, the switching unit SW can electrically connect other units to the control unit, enabling data from the control unit to be sent to those other units, thereby scheduling those other units. For example, the switching unit SW can send control instructions from the control unit to the accelerator card unit AI to schedule the accelerator card unit AI to execute model processing tasks.

[0041] The switching unit (SW) can also send the second model parameters from the extended storage unit (ES) to the accelerator card unit (AI) under the control of scheduling instructions. The AI ​​can then use these second model parameters to invoke a second expert model to perform model processing tasks and obtain the model processing results from the second expert model. The CXL link needs to have sufficient bandwidth to match the GPU's requirements, preventing bandwidth from becoming a bottleneck for accessing the extended storage unit (ES).

[0042] Based on this, under the control of control instructions, the accelerator card unit AI can obtain the first model parameters from the first memory AIS, or the second model parameters from the extended storage unit ES via the switching unit SW. Subsequently, the accelerator card unit AI can, based on the obtained first model parameters, call the first expert model to execute model processing tasks and obtain the model processing results of the first expert model; it can also, based on the obtained second model parameters, call the second expert model to execute model processing tasks and obtain the model processing results of the second expert model; or it can simultaneously call both the first and second expert models to execute model processing tasks, thereby obtaining the model processing results of the first and second expert models respectively. The first expert model is called more frequently than the second expert model. Thus, during operation, the accelerator card unit AI can preferentially read the first model parameters from the first memory AIS, which has relatively higher access speed, to execute model processing tasks.

[0043] Based on this, the time it takes for the accelerator card unit AI to read the first model parameters from the first memory AIS is shorter than the time it takes to read the second model parameters from the extended storage unit ES via the switching unit SW. Thus, storing the frequently accessed first model parameters in the first memory AIS, which the accelerator card unit AI can access quickly, facilitates frequent reading of the first expert parameters. Meanwhile, storing the less frequently accessed second model parameters in the extended storage unit, which has a relatively slower read speed, fully utilizes the accelerator card unit AI's computing power for frequent processing of the first expert parameters, reducing wasted computing power and thus reducing the resources consumed to increase server computing power. This improves the efficiency of the accelerator card unit AI in acquiring model parameters and reduces the power consumption associated with this process. Furthermore, pre-storing the less frequently accessed second model parameters in the extended storage unit ES avoids the need for temporary scheduling of the second model parameters' storage location when needed, thus avoiding additional scheduling time. This allows for timely reading of the relatively infrequently accessed expert model parameters from the extended storage unit ES. This improves the efficiency of the accelerator card unit AI in executing model processing tasks.

[0044] Furthermore, the monitoring unit MON can collect information on the different call frequencies of different expert models by the accelerator card unit AI, and transmit this information to the first control device via multiple buses L1, ..., Ln. For example, the monitoring unit MON can collect the call frequency information of the first expert model when the accelerator card unit AI calls the first expert model, and it can also collect the call frequency information of the second expert model when the accelerator card unit AI calls the second expert model.

[0045] For example, multiple buses L1, ..., Ln may include a first bus and a second bus. The monitoring unit MON can be electrically connected to the first control device CTD1 via the first and second buses. MON can send the call frequency information of the first expert model to the first control device via the first bus at a first transmission speed, and send the call frequency information of the second expert model to the second control device via the second bus at a second transmission speed. The first transmission speed is higher than the second transmission speed. Thus, for the first expert model with a relatively high call frequency, using a bus with a relatively faster transmission speed facilitates the first control device's timely acquisition of the accelerator card unit AI's call status for the frequently called expert model. This allows for timely resource scheduling for the accelerator card unit AI, improving the timeliness and efficiency of resource scheduling.

[0046] The embodiments of this application are not limited to this. The monitoring unit MON can also select multiple buses L1, ..., Ln respectively based on the storage location of the model parameters of different expert models, and thus transmit different call frequency information to the first control device CTD1 via multiple buses L1, ..., Ln. For example, when the accelerator card unit AI reads model parameters from the first memory, the monitoring unit MON can select the first bus to transmit the first call frequency information of the model parameters stored in the first memory AIS. Alternatively, when the accelerator card unit AI reads model parameters from the extended storage unit ES, the monitoring unit MON can select the second bus to transmit the second call frequency information of the model parameters stored in the extended storage unit ES.

[0047] The first control device CTD1 can pre-store a mapping relationship between first call frequency information, second call frequency information, and scheduling strategies. For example, this mapping relationship can be in the form of a mapping table. Upon receiving the first and second call frequency information, it can utilize its stored mapping relationship to determine the scheduling strategy associated with the first and second call frequency information and send the queried scheduling strategy to the control unit. Thus, based on the correlation between the storage location of model parameters and the call frequency of expert models described above, the bus corresponding to the specific storage location of the model parameter can be used to transmit the call frequency information of the expert model to which the model parameter belongs. This avoids the problem that, in some cases, changes in the call frequency of the expert model can lead to a mismatch between the speed of the bus used to transmit the call frequency information and the call frequency information itself, thereby affecting the timeliness of transmitting the call frequency information of frequently called expert models. The embodiments of this application are not limited to this. The above mapping relationship can also be stored in the terminal device. The first control device CTD1 can also send the first call frequency information and the second call frequency information to the terminal device. The terminal device performs the above operation to obtain the scheduling strategy and sends the scheduling strategy to the first control device CTD1. Then, the first control device CTD1 sends the scheduling strategy to the control unit.

[0048] In other embodiments of this application, each model processing task may include multiple model processing flows. In different model processing flows, the frequency at which the accelerator card unit AI calls the same expert model may differ. For example, in the first model processing flow of the accelerator card unit AI, since the call frequency of the first expert model is higher than that of the second expert model, the first memory AIS can store the first model parameters, and the extended storage unit ES can store the second model parameters, so that the accelerator card unit AI can promptly obtain the model parameters of the frequently called expert model. In this case, the monitoring unit MON can send different call frequency information to the first control device CTD1 via multiple buses L1, ..., Ln, so that the first control device CTD1 can determine the second model processing flow after the first model processing flow based on the received call frequency information. For example, the first control device CTD1 can pre-store a first mapping relationship between call frequency information and model processing flows, and a second mapping relationship between model processing flows and scheduling strategies; for example, the first and second mapping relationships can be in the form of a mapping table. Thus, the first control device CTD1 can determine the current model processing flow as the first model processing flow based on the received call frequency information and using the first mapping relationship, and determine the second model processing flow following the first model processing flow based on the order of multiple model processing flows in the first mapping relationship. Then, based on the second model processing flow, a scheduling strategy corresponding to the second model processing flow can be determined using the second mapping relationship. Afterwards, the determined scheduling strategy can be sent to the control unit when the first model processing flow ends. Specifically, the end of the first model processing flow can be determined when the accelerator card unit AI stops calling the expert model. In another embodiment of this application, the call frequency information can also be sent to the terminal device via the first control device CTD1, so that the terminal device can determine the scheduling strategy based on the call frequency information.

[0049] At least one of the first and second model parameters may be stored in a different location in the second model processing flow than in the first model processing flow. For example, the call frequency of the first expert model and the call frequency of the second expert model may change in the second model processing flow relative to the first model processing flow. Thus, the model parameters stored in the first memory AIS may change; for example, they may be changed from storing the first model parameters to storing the second model parameters. Correspondingly, the model parameters stored in the extended storage unit ES may change from the second model parameters to the first model parameters. In this case, the storage locations of the first and second model parameters change. For example, after receiving the scheduling policy from the first control device CTD1, the control unit can update its previously deployed initial scheduling policy to a new scheduling policy. Under the control of this scheduling policy, the control unit can send the first model parameters to the extended storage unit ES via the switching unit SW to write the first model parameters into the extended storage unit, thereby changing the storage location of the first model parameters, and / or send the second model parameters to the accelerator card unit AI via the switching unit SW to write the second model parameters into the first memory, thereby changing the storage location of the second model parameters. For example, after writing the first model parameters to the extended storage unit ES, the first model parameters can overwrite the second model parameters originally stored in the extended storage unit ES. Similarly, after writing the second model parameters to the first memory, the second model parameters can overwrite the first model parameters originally stored in the extended storage unit ES. By changing the storage location of the model parameters, the transmission speed of the call frequency information of the first and second expert models can be adjusted simultaneously, thereby matching the transmission speed of the call frequency information of the first and second expert models with the processing flow of the second model.

[0050] Specifically, the storage locations of the first model parameters and the second model parameters have changed. The first model parameters, as model parameters of the first expert model that are called infrequently, are stored in the extended storage unit (ES), while the second model parameters, as model parameters of the second expert model that are called infrequently, are stored in the first memory unit (AIS). Based on this, in the second model processing flow, which is different from the first model processing flow, the first expert model becomes a low-frequency expert model, and the second expert model becomes a high-frequency expert model. Consequently, the call frequency information of the second expert model can be transmitted at the first transmission speed, and the call frequency information of the first expert model can be transmitted at the second transmission speed.

[0051] The embodiments of this application are not limited to this. In the embodiments of this application, the first model processing flow and the second model processing flow may belong to different model processing tasks. For example, the calculation result obtained in the first model processing flow may not be used as input information for the second model processing flow. Specifically, the second model processing flow may be the model processing flow of a model processing task executed by the AI ​​unit of an accelerator card on another circuit board, etc.

[0052] Building upon this, and addressing the issue in some solutions where models deployed on a single GPU (e.g., deep learning model architectures based on self-attention mechanisms, such as Transformers) can only execute multiple model processing flows of the same task sequentially in a fixed pipeline during the execution of the same model processing task, making it difficult to schedule GPU resources in a timely manner, the solution described in this application decouples the sequentially executed model processing tasks and uses multiple circuit boards to execute the originally pipelined sequential model processing flows in parallel. In the parallel execution of the model processing flows, the monitoring units of each circuit board executing the parallel model processing flows can send call frequency information corresponding to the model parameters at a corresponding transmission speed based on the storage location of the model parameters. For example, the monitoring unit transmits the call frequency information of expert models with relatively high call frequencies at a relatively high speed. Thus, even if the AI ​​of the accelerator card unit of any circuit board receives a scheduling strategy belonging to the model processing flow of other circuit boards and executes the same model processing task in parallel with other circuit boards, the first control device CTD1 can promptly obtain the location of the model parameters stored in the accelerator card unit AI of each of the multiple circuit boards that are processing the same model processing task in parallel, and promptly send the call frequency information of the frequently called expert models deployed on each circuit board to the first control device CTD1. This allows the first control device CTD1 to promptly perform resource scheduling for multiple circuit boards based on the received call frequency information of the frequently called expert models, thereby improving the timeliness and efficiency of resource scheduling.

[0053] Figure 2 shows a schematic diagram of a circuit board according to a second embodiment of this application.

[0054] As shown in Figure 2, the extended control unit CTRL can include an extended control subunit ES_1 and multiple extended storage subunits. For example, the multiple extended storage subunits can include a first extended storage subunit ES_21 and a second extended storage subunit ES_22. For example, the extended control subunit ES_1 can be a memory extension controller (MXC). The extended storage subunits can be in the form of Double Data Rate Synchronous Dynamic Random Access Memory (DDR), which is a physical form of a memory resource pool expanded based on the CXL protocol, and its capacity can be maximized according to bandwidth. In addition to the accelerator card unit AI interacting with this storage subunit, the control unit CTRL can also interact with this storage subunit via the switching unit SW through the Global Interconnect Manager (GIM) of CXL3.1. Memory resource functions are accessed by the control unit CTRL. In this way, two forms of acceleration can be achieved based on the CPU as a general computing node and the GPU as an intelligent computing node, solving the problems of insufficient memory capacity or other memory-related issues in both general computing and intelligent computing. The extended control subunit ES_1 can control at least one of the multiple extended storage subunits to store data transmitted via the switching unit SW, and can also send data stored in at least one of the multiple extended storage subunits to the switching unit SW for transmission to other units. By setting the extended control subunit ES_1 and the extended storage subunit on the circuit board, the extended control subunit ES_1 and the extended storage subunit can be used to store model parameters with low call frequency. Thus, when the computing power of the circuit board is insufficient, the model parameters stored in the extended storage subunit can be sent to other circuit boards for calculation. In this way, the strategy of exchanging memory for computing power adopted in this application can solve the problem of insufficient computing power, which greatly improves the quality of activated experts and greatly increases the number of inactive experts on a single circuit board. Furthermore, in this application, the novel architecture of the extended control subunit ES_1 and the extended storage subunit, combined with the CXL cache coherence protocol, is used to expand the memory of the AI ​​in the accelerator card unit. In this way, the latency problem can be improved by using multi-level caching and increasing the cache capacity of the extended control subunit ES_1 and the extended storage subunit. It should be understood that the number of extended control subunits ES_1 and extended storage subunits in Figure 2 is only an example, and this application does not limit the number of extended control subunits ES_1 and extended storage subunits.

[0055] In this application, in addition to the first and second expert models, the accelerator card unit AI can also call a third expert model to perform model processing tasks. The first expert model is called more frequently than the third expert model, and the third expert model is called more frequently than the second expert model. The second memory CTRL of the control unit can be used to store the third model parameters of the third expert model. The second memory CTRL can be in the form of ordinary DDR chips, which is compatible with the platform on which the CPU resides. Furthermore, when using DDR, using an LGA (Land Grid Array) socket will result in lower signal transmission loss and improved signal rate.

[0056] The accelerator card unit AI can also, under the control of control instructions, obtain third model parameters from the second memory CTRLS, and based on the third model parameters, call the third expert model to execute model processing tasks, obtaining the model processing results of the third expert model. For example, the first expert model, the second expert model, and the third expert model can be divided based on a predetermined call frequency range. In the embodiments of this application, the first expert model, the second expert model, and the third expert model can be used to perform the same type of model processing tasks. For example, expert models set on the same circuit board can be used to process the same type of smallest text unit (token). For example, the first expert model, the second expert model, and the third expert model can all be used to process punctuation marks, or all be used to process words, etc.

[0057] The second memory (CTRLS) of the control unit CTRL can store the first model parameters, the second model parameters, and the third model parameters. In this embodiment, the control unit CTRL can pre-schedule the resources of the first memory (AIS) and the extended storage unit (ES) of the accelerator card unit AI to store the model parameters before the AI ​​executes the model processing task. For example, the control unit CTRL can send a scheduling instruction, the first model parameters, and the second model parameters to the switching unit SW according to the initial scheduling strategy. Under the control of the scheduling instruction, the switching unit SW can send the first model parameters to the accelerator card unit AI to write the first model parameters into the AI's first memory (AIS), or it can send the second model parameters to the extended storage unit ES to write the second model parameters into the extended storage unit ES.

[0058] Figure 3 shows a schematic diagram of a circuit board according to a third embodiment of this application.

[0059] As shown in Figure 3, in addition to the control unit CTRL, switching unit SW, accelerator card unit AI, extended storage unit ES, and monitoring unit MON mentioned above, the circuit board of this embodiment may also include a communication unit INT.

[0060] The communication unit INT can be a Network Interface Controller (NIC). The communication unit INT can be electrically connected to the switching unit SW, and receive data from the control unit CTRL via the switching unit SW. Based on the communication connection with other circuit boards, the received data can be sent to other circuit boards.

[0061] The monitoring unit MON can be electrically connected to the first interface CON1, which can be a high-speed connector. The monitoring unit MON can output bus signals of various speeds via the first interface CON1. ​​For example, the first interface CON1 can include a USB interface, LTPI, etc. The USB interface can be used to transmit USB signals. The LTPI interface can be used to transmit two-wire serial bus (Inter-Integrated Circuit, I2C) signals, Universal Asynchronous Receiver / Transmitter (UART) signals, General Purpose Input / Output (GPIO) signals, etc.

[0062] The switching unit SW can be electrically connected to the second interface CON2, which can be a high-speed connector. The switching unit SW can be connected to the switching units SW of other circuit boards via the second interface CON2.

[0063] The accelerator card unit AI can be electrically connected to the third interface CON3, which can be a high-speed connector. The accelerator card unit AI can be connected to other accelerator card unit AIs on other circuit boards via the third interface CON3.

[0064] The communication unit INT can be electrically connected to the fourth interface CON4, which can be a high-speed connector. The communication unit INT can be connected to the communication units INT of other circuit boards via the fourth interface CON4.

[0065] Specifically, after the circuit board is powered on and begins operation, it can be used not only as a computing core, but also as a GPU and a memory pool. When the accelerator card unit's AI is working, in addition to obtaining the model parameters of the first memory AIS, it can also obtain the model parameters of the second memory CTRLS and the extended storage unit ES. For example, the accelerator card unit's AI can call the Compute Unified Device Architecture (CUDA) programming model to allocate a region on the first memory AIS, or it can call CUDA to allocate the same size space on the second memory CTRLS, then initialize the data, and copy the initialized data from the first memory AIS to the second memory CTRLS. Furthermore, the unified memory access technology (UMA) can automatically take over the memory placement work. Thus, by simply calling the fixed application programming interface (API) in CUDA, UMA can handle the automatic processing.

[0066] However, in cases with multiple nodes (such as the aforementioned circuit board) within the chassis and rack, the bus becomes a performance bottleneck as the number of processors increases because the processors on each node access memory via a shared bus. This slows down memory access speed, impacting overall system performance. Furthermore, the memory that the control unit CTRL can carry is limited. Therefore, accessing the memory of the extended storage unit ES is particularly important and convenient. Using CXL bus technology, the CXL resources of the control unit CTRL can be transformed into a memory extension resource pool, allowing the accelerator card unit AI to easily access the extended memory resources. The prerequisite for access is:

[0067] ①CXL Interface Integration: The model processing subunit AI_H supports the CXL.memory protocol, or can access the extended storage subunit through the extended control subunit ES_1. The CXL.memory protocol allows the control unit CTRL and the accelerator card unit AI to directly access or pool the memory of the extended storage subunit.

[0068] ②Physical connection: CXL extended memory is connected to the accelerator card unit AI and control unit CTRL via high-speed interconnect (such as CXL link) to ensure low latency and high bandwidth.

[0069] ③ Virtualization support: The CXL extended memory is mapped to a unified virtual address space, and the accelerator card unit AI accesses the memory of the extended storage subunit through its built-in Memory Management Unit (MMU).

[0070] ④ Page Table Management: The accelerator card unit AI driver works with the control unit CTRL to manage the page table of the extended storage sub-unit's memory, and handles the address translation requests of the accelerator card unit AI through the Input-Output Memory Management Unit (IOMMU) or MMU.

[0071] ⑤CXL.cache protocol: The accelerator card unit AI supports cache consistency. CXL.cache is used to maintain the cache state between the accelerator card unit AI and the control unit CTRL to avoid data inconsistency.

[0072] ⑥ Hardware Coherence Domain: A coherence domain is built within the circuit board, enabling the accelerator card unit AI and the control unit CTRL to transparently access the extended storage sub-unit without explicit refresh.

[0073] ⑦ At the driver and operating system layer: The control unit CTRL recognizes the access to the extended storage subunit as an allocatable resource, provides APIs for the CXL.mem module driver of the Linux system to manage, and can modify the default driver of the accelerator card unit AI to enable the accelerator card unit AI to support memory allocation of the extended storage subunit, and enable the accelerator card unit AI to handle DMA (Direct Memory Access) requests and address mapping.

[0074] Even after the memory of the extended storage subunit is accessed, the system still requires further optimization. Therefore, based on the access frequencies of the first, second, and third expert models, the positions of the model parameters stored in the first memory AIS, the extended storage unit ES, and the second memory CTRL can be adjusted. Frequently accessed data is placed in the first memory AIS, while less frequently accessed data is stored in the extended storage subunit. This access frequency data is recorded by the monitoring unit MON, which can send the acquired access frequency information to the terminal device to update the scheduling strategy of the control unit CTRL.

[0075] On a single circuit board, a small-scale hybrid expert model can be deployed based on the capacity of the first memory AIS, the extended storage sub-unit, and the second memory control unit CTRLS. The number of expert models that can be deployed on a single circuit board can be calculated using the following formula:

[0076] Maximum number of expert models = [(Capacity of the first memory AIS - Overhead of the 2GB circuit board) + (Total memory capacity available for the accelerator card unit AI × Predetermined model parameter calling factor)] / (Number of parameters of the expert model × Number of bytes processed by the accelerator card unit AI per unit time + Activation value occupancy) × (50% ~ 70%) (1)

[0077] As can be seen from the above formula, 50 expert models can be deployed on a single circuit board, which is higher than the 20 expert models in related technologies, thus increasing the number of models that can be deployed and facilitating the improvement of the computing power of the circuit board in this application.

[0078] Furthermore, multiple expert models can be deployed on a single circuit board, each including expert models of the same type. Specifically, the architecture and workflow of MOE can be represented by the following two formulas: MOE(x)=∑K(Gi(x)Ei(x)) (2) G(x)=TopK(Softmax(Wg(x)+€)): (3)

[0079] Where Ei(x) is the result of the input token being processed by the i-th expert model. Gi(x) is the output weight (between 0 and 1) of the i-th expert model. G(x) represents the total result of the model processing. TopK is used to select the K largest values ​​from the input data. Softmax represents the activation function. Wg represents the predefined weights. x represents the input data. € represents the predefined variable.

[0080] As can be seen from the architecture and workflow of MOE, in addition to having fewer activation parameters and faster inference speed, it also has strong scalability. By increasing or decreasing the number of expert models, the parameter scale of the expert models can be adjusted at will, thereby adapting to more different tasks.

[0081] Based on the working principle of the MOE architecture, this application further addresses the deployment of 1 to 2 expert models on a single circuit board CD for performing the same type of model processing tasks. This can further reduce the number of parameters activated on a single circuit board CD, thereby better allocating memory resources to the expert models deployed on a single circuit board CD. This reduces the communication overhead caused by data interaction between multiple circuit board CDs during the operation of the computing system CS, and avoids memory-related problems such as video memory fragmentation.

[0082] Compared to related technologies that excessively stack the number of GPUs and increase GPU power consumption, this application further refines the hardware architecture based on the MOE model. It not only increases the number of expert models in the MOE model to improve computing power, but also makes each computing hardware node more refined. Instead of deploying expert models across all GPUs in the entire machine cluster, the expert models are deployed on specific single computing chips according to the characteristics of the business processing. This means replacing the large Feedforward Neural Network (FFN) in the Transformer model with multiple smaller expert models (also FFN structures), and then employing a sparse activation mechanism, activating only 1-2 expert models of the same type during each inference. This significantly reduces the number of computational parameters required for inference, thereby improving the inference efficiency of the entire MOE model. Furthermore, while increasing the number of stored parameters, it improves the interpretability of model processing and the ability to acquire local information. It is important to note that each expert model is also an FFN structure. The expert models in this application can be compressed, for example, by means of sparsity, distillation, and quantization.

[0083] Therefore, the number of experts in this application can be adjusted according to system needs. Since only the same type of expert model is activated at a time in this application, the parallel capabilities of the expert models need to be improved to handle dedicated tasks, which places higher demands on the memory capacity and bandwidth of the current accelerator card unit AI.

[0084] Figure 4 shows a schematic diagram of a computing system according to an embodiment of this application.

[0085] As shown in Figure 4, the computing system of this embodiment includes multiple circuit boards (CDs) and a first control device (CTD1). The circuit boards (CDs) can be any of the types described above. A single computing system can be deployed within a single chassis.

[0086] The first control device CTD1 can collect the call frequency information of expert models configured in multiple circuit boards CD and send the call frequency information to the terminal device. For example, the first control device CTD1 can receive the call frequency information from the monitoring unit MON of the circuit board CD and determine the scheduling strategy of the circuit board based on the call frequency information. For example, a mapping relationship between call frequency information and scheduling strategy can be preset, so that the scheduling strategy corresponding to the call frequency information can be determined using this mapping relationship.

[0087] In some embodiments of this application, the multiple circuit boards CD include a first circuit board CD1 and a second circuit board CD2. The model parameters stored in the first circuit board CD1 and the second circuit board CD2 are used to perform different types of model processing tasks. For example, the model parameters stored in the first circuit board CD1 and the second circuit board CD2 can be used to process different types of data. Specifically, the model parameters stored in the first circuit board CD1 and the second circuit board CD2 are used to process different types of the smallest unit (token). For example, the model parameters of the first circuit board CD1 can be used to process punctuation marks, while the model parameters of the second circuit board CD2 can be used to process words, etc.

[0088] In this embodiment, the model parameters stored on the first circuit board CD1 can be target model parameters used to perform target model processing tasks. The first circuit board CD1 is electrically connected to the second circuit board CD2 and can send the target model parameters to the second circuit board CD2. Upon receiving the target model parameters, the second circuit board CD2 can, based on the target model parameters, call the target expert model to perform a target-type model processing task and obtain the target model processing result. Therefore, by scheduling the computing resources of the second circuit board CD2 to assist in processing the target model parameters of the first circuit board CD1, problems such as the first circuit board CD1 malfunctioning due to excessively high frequencies of calling the target expert model can be avoided, thereby improving the reliability of the circuit board CDs and increasing the redundancy of the computing system.

[0089] Figure 5 shows a schematic diagram of a first circuit board and a second circuit board according to a first embodiment of this application.

[0090] In the embodiment shown in Figure 5, the first circuit board CD1 includes a first control unit CTRL1 and an accelerator card unit AI1 electrically connected via a switching unit SW1 of the first circuit board. The first control unit CTRL1 can send a first scheduling instruction, a first control instruction, and target model parameters to the switching unit SW1 of the first circuit board according to a first target scheduling strategy. Under the control of the first scheduling instruction, the switching unit SW1 of the first circuit board can send a first control instruction and target model parameters to the accelerator card unit AI1 of the first circuit board based on the first target scheduling strategy. It should be understood that in this application, both the first circuit board CD1 and the second circuit board CD2 are circuit boards described above. Therefore, they also have structures corresponding to the circuit boards described above. For example, the accelerator card unit AI1 of the first circuit board and the accelerator card unit AI2 of the second circuit board are both accelerator card units described above. The other units in the first circuit board CD1 and the second circuit board CD2 are similar and will not be described in detail here. Similarly, the circuit boards described in the following sections of this application are also similar to the circuit boards described above, and the units of the circuit boards described in the following sections are also similar to those described above. These will not be described in detail here.

[0091] The second circuit board CD2 includes an accelerator card unit AI2. The accelerator card unit AI1 of the first circuit board is electrically connected to the accelerator card unit AI2 of the second circuit board, and can send target model parameters to the accelerator card unit AI2 under the control of a first control command, so as to write the target model parameters into the memory of the accelerator card unit AI2. Here, "memory" refers to the first memory of the aforementioned accelerator card unit.

[0092] Figure 6 shows a schematic diagram of a first circuit board CD1 and a second circuit board CD2 according to a second embodiment of this application.

[0093] In the embodiment shown in Figure 6, the first circuit board CD1 includes a first control unit CTRL1 and an accelerator card unit AI1 electrically connected via a switching unit SW1 of the first circuit board. The first control unit CTRL1 can send a second scheduling command, a second control command, and target model parameters to the switching unit SW1 of the first circuit board according to a second target scheduling strategy.

[0094] The second circuit board CD2 may include a switching unit SW2 and an extended storage unit ES2. The switching unit SW1 of the first circuit board is electrically connected to the switching unit SW2 of the second circuit board and, under the control of a second scheduling instruction, can send target model parameters to the switching unit SW2 based on a second target scheduling strategy. The switching unit SW2 of the second circuit board is electrically connected to the extended storage unit ES2. Thus, via the switching unit SW2 of the second circuit board, the switching unit SW1 of the first circuit board can send the target model parameters to the extended storage unit ES2, thereby writing the target model parameters into the extended storage unit ES2.

[0095] The embodiments of this application are not limited to these. The first circuit board CD1 may further include a first communication unit, and the second circuit board CD2 may further include a second communication unit. The switching unit SW1 of the first circuit board may also be electrically connected to the first communication unit. Thus, the first communication unit can receive target model parameters via the switching unit SW1 of the first circuit board. The first communication unit is communicatively connected to the second communication unit and can send target model parameters to the second communication unit. The second communication unit is electrically connected to the switching unit SW2 of the second circuit board and can send target model parameters to the extended storage unit ES2 of the second circuit board via the switching unit SW2 of the second circuit board.

[0096] Figure 7 shows a schematic diagram of the connection of units in different circuit boards according to embodiments of this application.

[0097] As shown in Figure 7, accelerator card units on different circuit boards in the computing system can be connected to each other. For example, the first accelerator card unit can be connected to the second accelerator card unit, the first and second accelerator card units can be connected to the Nth accelerator card unit, and so on. In this way, each accelerator card unit can be connected to other accelerator card units, thereby facilitating data transmission between accelerator card units. Here, the first accelerator card unit can refer to the accelerator card unit on the first circuit board; the other accelerator card units are similar, and will not be elaborated further here.

[0098] Similarly, switching units on different circuit boards in a computing system can be connected to each other. For example, the first switching unit can be connected to the second switching unit, the first and second switching units can be connected to the Nth switching unit, and so on. In this way, each switching unit can be connected to other switching units, thereby facilitating data transmission between switching units. Here, the first switching unit can refer to the switching unit on the first circuit board; the other switching units are similar and will not be elaborated further here.

[0099] Similarly, communication units on different circuit boards in a computing system can be connected to each other. For example, the first communication unit can be connected to the second communication unit, the first and second communication units can be connected to the Nth communication unit, and so on. In this way, each communication unit can be connected to other communication units, facilitating data transmission between them. Here, the first communication unit can refer to the communication unit on the first circuit board; the other communication units are similarly connected and will not be elaborated upon here. It should be noted that, for illustrative purposes, the first and Nth communication units are connected by a black line in Figure 7, but this should be understood as merely illustrative; in reality, communication units can be interconnected.

[0100] Furthermore, in this application, the switching units of multiple circuit boards are connected to form a FABRIC (structure) link. Within a single computing system, this network architecture exhibits excellent nanosecond-level low latency performance. Based on this, the accelerator card units of multiple circuit boards are connected to achieve peer-to-peer communication (P2P) between the accelerator card units, thus forming a dual-ring network with the aforementioned FABRIC network architecture. In this way, frequently called model parameters can be transmitted between connected accelerator card units, and frequently called model parameters can also be transmitted between connected switching units. The network formed by connected accelerator card units and the network formed by connected switching units are mutually redundant, reducing the amount of data related to the reduce function in the link to half of its original value. This satisfies both general low-latency transmission requirements and the specific needs of the communication library. In addition, the aforementioned FABRIC link can also be used to transmit data stored in the extended storage unit and data in the memory of the control unit. Since the network formed between accelerator card units does not require a third-party unit to relay data, the latency caused by the third-party unit's data relay can be avoided. Therefore, the dedicated link used to transmit frequently accessed data can also be regarded as a dedicated link for accelerator card units to access each other's memory.

[0101] The data throughput of the network formed by the connected communication units is larger than that of the dual-ring network mentioned above. However, the data needs to be demodulated during data transmission, so it can be used to transmit data with a low call frequency other than high-frequency data.

[0102] Based on this, three balanced topologies can be formed by connecting the switching units, accelerator card units and communication units of the different circuit boards, thereby reducing the amount of data related to the reduce function in the link and facilitating the optimization of system resource allocation.

[0103] In some embodiments of this application, the first control device can send target call frequency information of the target expert model from the first circuit board to the terminal device. For example, the target call frequency information can be sent to the terminal device via an RJ45 (Registered Jack) network interface. After receiving the target call frequency information, the terminal device can display the target call frequency information through a visual interface, so that relevant personnel can send a target scheduling strategy to the first control device based on the displayed target call frequency information. In response to receiving the target scheduling strategy corresponding to the target call frequency information from the terminal device, the first control device can send the target scheduling strategy to the first circuit board to control the first circuit board to send target model parameters to the second circuit board, thereby realizing the scheduling of resources of the second circuit board. In embodiments of this application, the target scheduling strategy can also be determined based on other information such as GPU load information, first memory usage information, second memory control unit usage information, and circuit board power consumption information. The method is similar to that described above and will not be repeated here.

[0104] For some exemplary server hardware architectures, their high complexity and management difficulty lead to high operation and maintenance costs, high management difficulty, and high overall costs. Furthermore, such server hardware architectures have a large number of nodes, and the computing power of some nodes may not be fully utilized, resulting in resource waste and affecting the computing efficiency of the architecture. Moreover, such server hardware architectures are subject to hardware limitations. Since the hardware resources of a single server have an upper limit, it is difficult to further expand the server's hardware resources after reaching the limit, thus limiting the server's computing power. Furthermore, since a large number of resources are deployed on the same server, a failure of one server node may cause the entire system to crash. Additionally, the above-mentioned server hardware architectures are inefficient at handling a large number of small memory read and write tasks. This application employs hardware-aware routing technology and dynamic monitoring technology to determine the calling frequency and other status information of circuit boards, thereby enabling the transmission of model parameters stored on any circuit board to other circuit boards for processing when the computing power of the accelerator card unit on that circuit board is insufficient. Thus, when any circuit board invokes the first target expert model for executing the target model processing task of the target type, that circuit board can send model parameters of the second target expert model for the same target model processing task to other circuit boards. In this way, the aforementioned circuit board can execute a portion of the data processing for the target model processing task, while other circuit boards can invoke the second target expert model to execute the remaining portions of the data processing for that target model processing task, achieving pipeline parallelism context tasks and solving the problem of low parallel computing capability for large models in related technologies. Furthermore, the allocation of expert models between the aforementioned circuit board and other circuit boards can be dynamically adjusted using the aforementioned update and adjustment strategy, thereby achieving dynamic batching. This solves the problem of insufficient memory leading to insufficient computing power for large models in related technologies and can balance the differences in computation time between different expert models, improving computational efficiency.

[0105] In this way, by achieving cross-device, cross-system, and even cross-architecture scheduling, the limitations of hardware architecture on the computing power scheduling of accelerator card units can be overcome, and the resources of accelerator card units can be fully utilized, reducing resource waste and lowering costs. This can solve the problems of insufficient large model parameters, few expert models, and low capabilities in related technologies, and can also solve the problem of resource waste caused by the current fixed deployment method of large models.

[0106] In some embodiments of this application, multiple circuit boards are stacked vertically in sequence according to the number of times the configured expert models are called. A first circuit board is adjacent to a second circuit board, and the expert model on the first circuit board is called more times than that on the second circuit board. Thus, when the expert model on the first circuit board is called more frequently, by scheduling the resources of the second circuit board adjacent to the first circuit board to execute model processing tasks based on the model parameters of the first circuit board, problems such as the first circuit board failing due to excessively high call frequencies to the target expert model can be avoided, thereby improving the reliability of the circuit boards. It should be understood that in the embodiments of this application, the resources of other circuit boards called by the first circuit board are not limited to those adjacent to the first circuit board, but may also include other circuit boards with lower call frequencies of expert models. Based on this, for a single computing system, the circuit boards in this application use a single accelerator card unit connected to a single control unit, and the expert model is specifically deployed on a single circuit board to realize a single circuit board as a small-scale basic model. On this basis, dynamic routing of model parameters between multiple circuit boards within a single computing system can constitute a medium-scale model. By deploying a medium-sized model in this way, the number of expert models within the computing system, the resources that experts can schedule, and the data flow and gating network between expert models can be optimized. Furthermore, in this embodiment, a hierarchical communication strategy can be adopted to deploy frequently interacting expert groups on the same circuit board or within the same computing system, utilizing NVLink communication technology or PCIe Fabric interconnect technology to improve communication speed.

[0107] In some embodiments of this application, the first control device is electrically connected to any one of a plurality of circuit boards via multiple buses. The data transmission speeds of the multiple buses are different.

[0108] The first control device can collect call frequency information of different types of expert models from any circuit board via multiple buses. The type of expert model represents its historical call frequency. For example, expert models can be classified according to their historical call frequency and call frequency range. For instance, the first, second, and third expert models described above are different types of expert models. In this embodiment, call frequency information of expert models with high call frequency (such as the first expert model) and call frequency information of expert models with low call frequency (such as the second expert model) can be collected from any circuit board via multiple buses. The collected expert model call frequencies can be sent to the terminal device so that the circuit board can be scheduled to call expert models to perform model processing tasks in a timely manner based on the number of calls to different types of expert models.

[0109] Figure 8 shows a schematic diagram of the connection between the first control device and the circuit board according to an embodiment of this application.

[0110] As shown in Figure 8, multiple buses include a first bus L1 and a second bus L2. The data transmission speed of the first bus L1 is higher than that of the second bus L2. The monitoring unit MON of the circuit board CD can collect the call frequency information of the expert model of the circuit board CD. The first control device CTD1 may include a data acquisition unit ACQ, which can be electrically connected to the monitoring unit MON via the first bus L1 and the second bus L2. The data acquisition unit ACQ collects the first call frequency information of the first type of expert model from the monitoring unit MON via the first bus L1, and the second call frequency information of the second type of expert model from the monitoring unit MON via the second bus L2. The first control device CTD1 may also include a device control unit ECT. The device control unit ECT can send the first call frequency information and the second call frequency information to the terminal device TE.

[0111] The acquisition unit ACQ may include a first acquisition subunit and a second acquisition subunit. The first bus L1 may be the aforementioned USB bus. The monitoring unit MON may be electrically connected to the first acquisition subunit via the first bus L1, and the first acquisition subunit may be connected to a USB hub unit. Data can be quickly transmitted to the USB hub unit for real-time processing via the USB bus, and then the USB hub unit can transmit the acquired information to the device control unit ECT. The second bus L2 may be electrically connected to the second acquisition subunit. The second bus L2 may be the aforementioned LTPI bus. The FPGA may acquire model parameters with medium call frequency from the monitoring unit MON via the second bus L2. For model parameters with low call frequency, the FPGA may acquire the call frequency information via I2C signals or CPIO signals.

[0112] In some embodiments of this application, the first control device CTD1 further includes a system switching unit SSW. For example, the system switching unit may be implemented based on a PCIe switch unit. The system switching unit SSW is electrically connected to the switching units SW of each of the multiple circuit boards CD. The device control unit ECT is electrically connected to the system switching unit SSW and is used to send a target scheduling policy to the system switching unit SSW, so that the switching units of the circuit boards CD receive the target scheduling policy via the system switching unit, and then write the target scheduling policy into the control unit via the switching unit. For example, the switching unit of the first circuit board receives the target scheduling policy via the system switching unit, and then writes the target scheduling policy into the first control unit via the switching unit of the first circuit board.

[0113] Figure 9 shows a schematic diagram of a computer cabinet according to a first embodiment of this application.

[0114] As shown in Figure 9, the computer cabinet in this embodiment includes multiple computing systems CS and a second control device CTD2. The computing system CS here can be any of the computing systems described above.

[0115] The second control device CTD2 can receive call frequency information from the first control device of multiple computing systems CS, and can determine the scheduling strategy based on the received call frequency information. The specific method is similar to that described above and will not be repeated here. For example, a computer cabinet can correspond to a server cabinet.

[0116] Figure 10 shows a schematic diagram of a computer cabinet according to a second embodiment of the present application, and Figure 11 shows a connection schematic diagram of a computer cabinet according to an embodiment of the present application.

[0117] As shown in Figures 10 and 11, the computer cabinet also includes a data acquisition device (CLD). The CLD can be a Remote Power Distribution Unit (RPDU). The CLD can be electrically connected to the circuit boards of multiple computing systems (CS) and collect power consumption information from the circuit boards. In one embodiment of this application, the CLD can directly send power consumption information to the terminal device. In another embodiment, the CLD can be electrically connected to a second control device (CTD2) and send power consumption information to the terminal device via the CTD2. After receiving the power consumption information, the terminal device can determine a scheduling strategy based on the power consumption information and send the scheduling strategy to the second control device (CTD2). The second control device (CTD2) can then send the scheduling strategy to the circuit boards in the computing system (CS) to schedule the resources of those circuit boards.

[0118] In some embodiments of this application, the second control device CTD2 can be an industrial-grade CPU based on an edge series. This CPU features wide temperature range and fast response. The second control device CTD2 may include a system control unit SCT, a first architecture switching unit SYC1, and a second architecture switching unit SYC2. The first architecture switching unit SYC1 can be implemented based on a PCIe switch unit. The second architecture switching unit SYC2 can be implemented based on an Ethernet switch (ETH SW) unit. The switching units SW of the circuit boards of multiple computing systems CS can be connected to the first architecture switching unit SYC1 and perform data transmission via the first architecture switching unit SYC1. For example, the switching units SW can also be connected to the first architecture switching unit SYC1 via the aforementioned system switching unit. The communication units of the circuit boards of multiple computing systems CS can be connected to the second architecture switching unit SYC2 and perform data transmission via the second architecture switching unit SYC2. In one embodiment of this application, the system control unit SCT can receive the scheduling policy corresponding to the power consumption information from the terminal device and send it to the switching unit SW of the circuit board via the first architecture switching unit SYC1, so that the scheduling policy can be written into the control unit of the circuit board via the switching unit SW. In another embodiment of this application, the system control unit SCT can receive the scheduling strategy corresponding to the power consumption information from the terminal device and send it to the communication unit INT of the circuit board via the second architecture switching unit SYC2, so as to write the scheduling strategy into the control unit of the circuit board via the communication unit INT and the switching unit SW connected to the communication unit INT.

[0119] In some embodiments of this application, multiple computing systems include a first computing system and a second computing system. The first computing system includes a circuit board for the first computing system. The second computing system includes a circuit board for the second computing system. A data acquisition device (CLD) is electrically connected to the circuit boards of the first and second computing systems and can acquire power consumption information from both circuit boards. The CLD can then send the power consumption information to a terminal device and send a scheduling policy corresponding to the power consumption information received from the terminal device to the circuit board of the first computing system. The circuit board of the first computing system can send its model parameters to the circuit board of the second computing system according to the scheduling policy corresponding to the power consumption information. The circuit board of the second computing system can, based on the model parameters of the circuit board of the first computing system, invoke the expert model of the circuit board of the first computing system to perform model processing tasks and obtain the model processing results of the circuit board of the first computing system.

[0120] Multiple architectures can also be connected via the first architecture switching unit SYC1 and the second architecture switching unit SYC2 described above. This provides the physical conditions for multiple chassis within the same architecture and for multiple architectures to form a network cluster. Based on this, RDMA technology can be used to enable mutual access between multiple first control devices, ensuring efficient all-to-all communication and reducing the latency of gradient synchronization between experts. Furthermore, it provides conditions for PCIe networking between multiple chassis within a rack, which can be applied more extensively in low-latency scenarios. The second control device CTD2 can also include a rack management controller (RMC). The RMC can supervise and manage the entire architecture. For example, it can identify high-power computing systems (CS) within the architecture. These CS systems might be those with high memory usage of accelerator card units and high expert model call frequencies. The RMC can send power consumption information to terminal devices and receive feedback scheduling policies. Scheduling policies can also be added to the image file loaded by the RMC. The RMC can send power consumption information via the RJ45 network interface. Resources of high-power computing systems can be prioritized and redundantly allocated to them using other computing systems, ensuring their stable operation. Multiple server chassis can be housed within a single rack, using two types of switches—SYC1 and SYC2—to create multiple ports for resource exchange between chassis and nodes. For the MOE model, localized communication can be used for optimization. Frequently interacting expert groups can be deployed in the same rack, and technologies like NVLink can be used for tuning. Asynchronous updates of global parameters for non-critical paths (such as gated networks) can reduce communication congestion.

[0121] Figure 12 shows a schematic diagram of the connection between the circuit boards of the first computing system and the second computing system according to an embodiment of this application.

[0122] As shown in Figure 12, the circuit board TCD1 of the first computing system includes a control unit TCTRL1 and a communication unit TINT1 of the first computing system circuit board, which are electrically connected via the switching unit TSW1 of the first computing system circuit board. The circuit board TCD2 of the second computing system includes a communication unit TINT2 and an extended storage unit TES2 of the second computing system circuit board, which are electrically connected via the switching unit TSW2 of the second computing system circuit board.

[0123] The control unit TCTRL1 of the circuit board of the first computing system can send a first target scheduling instruction, a first target control instruction, and model parameters of the circuit board TCD1 of the first computing system to the switching unit TSW1 of the circuit board of the first computing system according to the scheduling strategy corresponding to the power consumption information.

[0124] In one embodiment of this application, the switching unit TSW1 of the first computing system circuit board can, under the control of the second target scheduling instruction, send the model parameters of the first computing system circuit board TCD1 to the communication unit TINT1 of the first computing system circuit board based on the scheduling strategy corresponding to the power consumption information. The model parameters of the first computing system circuit board TCD1 are then sent to the communication unit TINT2 of the second computing system circuit board via the communication unit TINT1 of the first computing system circuit board. The model parameters of the first computing system circuit board TCD1 received via the communication unit TINT2 of the second computing system circuit board are then written into the extended storage unit TES2 of the second computing system circuit board.

[0125] The circuit board TCD2 of the second computing system may also include a switching unit TSW2 of the circuit board of the second computing system. The communication unit TINT2 of the circuit board of the second computing system can write the model parameters of the circuit board TCD1 of the first computing system into the extended storage unit TES2 of the circuit board of the second computing system via the switching unit TSW2 of the circuit board of the second computing system.

[0126] The embodiments of this application are not limited to this. The circuit board TCD2 of the second computing system may also include an accelerator card unit of the circuit board of the second computing system. The accelerator card unit of the circuit board of the second computing system may be electrically connected to the switching unit TSW2 of the circuit board of the second computing system. Thus, in another embodiment of this application, the communication unit TINT2 of the circuit board of the second computing system may also write the model parameters of the circuit board TCD1 of the first computing system into the memory of the accelerator card unit of the circuit board of the second computing system.

[0127] In the embodiments of this application, multiple computer cabinets can be deployed, and the second control devices of the multiple computer cabinets can be interconnected to enable data interaction between the multiple computer cabinets. The high-speed network card signal of the communication unit can be electrically transmitted within the cabinet and can also be transmitted optically in long-distance transmission scenarios across cabinets. By adopting a network clustering approach, it is possible to achieve interconnection of hundreds, thousands, and tens of thousands of cards across chassis and cabinets in data nodes.

[0128] Figure 13 shows a schematic diagram of the calculation method according to an embodiment of this application.

[0129] As shown in Figure 13, the calculation method of this embodiment can be applied to a circuit board. The calculation method may include: operation S1310 to operation S1330.

[0130] During operation S1310, the accelerator card unit of the circuit board obtains the first model parameters from the memory of the accelerator card unit of the circuit board, and / or obtains the second model parameters from the extended storage unit of the circuit board via the switching unit of the circuit board.

[0131] During operation S1320, the accelerator card unit of the circuit board, based on the first model parameters and / or the second model parameters, calls at least one of the first expert model and the second expert model to perform a model processing task and obtain the model processing result;

[0132] During operation of S1330, the monitoring unit of the circuit board transmits information on the frequency of calls to different expert models by the accelerator card unit of the circuit board to the first control device at different transmission speeds.

[0133] In the embodiments of this application, operations S1310 to S1330 are similar to the operations performed by the circuit board described above.

[0134] For example, the monitoring unit of the circuit board transmits the call frequency information of the accelerator card unit of the circuit board to the first control device at different transmission speeds. This may include: the monitoring unit selects multiple buses based on the storage location of the model parameters of each expert model, and then transmits different call frequency information to the control device via the multiple buses, so that the control device can determine the scheduling strategy of the circuit board based on the received call frequency information.

[0135] For example, in the first model processing flow of the accelerator card unit, the memory is used to store the first model parameters, and the extended storage unit is used to store the second model parameters. The control device determines the scheduling strategy of the circuit board based on the received call frequency information, which may include: the control device determining a second model processing flow following the first model processing flow based on the received call frequency information, and determining the scheduling strategy based on the second model processing flow, wherein at least one of the first and second model parameters is stored in a different location in the second model processing flow than in the first model processing flow.

[0136] For example, the calculation method may further include: the control unit of the circuit board receiving a scheduling strategy from a control device; under the control of the scheduling strategy, the control unit sending a first model parameter to an extended storage unit via a switching unit to write the first model parameter into the extended storage unit, thereby changing the storage location of the first model parameter; and / or, sending a second model parameter to an accelerator card unit via a switching unit to write the second model parameter into memory, thereby changing the storage location of the second model parameter; wherein, after changing the storage location, the transmission speed of the call frequency information of the first expert model and the second expert model respectively matches the second model processing flow.

[0137] For example, multiple buses include a first bus and a second bus; transmitting different call frequency information to the control device via multiple buses may include: the monitoring unit sending the call frequency information of the model parameters stored in the memory to the control device via the first bus at a first transmission speed; and / or sending the call frequency information of the model parameters stored in the extended storage unit to the control device via the second bus at a second transmission speed; wherein the first transmission speed is higher than the second transmission speed.

[0138] For example, the above calculation method may further include the following: In the case where the accelerator card unit of the circuit board performs a model processing task based on one of the first and second model parameters; In the case where the accelerator card unit of the circuit board performs a model processing task based on one of the first and second model parameters; the accelerator card unit of the circuit board sends another model parameter from the first and second model parameters to other circuit boards to write the other model parameter into the other circuit boards, so that the other circuit boards perform model processing tasks based on the other model parameter. It should be understood that the calculation method of this application is not limited to this, and will not be elaborated further here.

[0139] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0140] Those skilled in the art will understand that the features described in the various embodiments of this application can be combined and / or combined in various ways, even if such combinations or combinations are not explicitly described in this application. In particular, the features described in the various embodiments of this application can be combined and / or combined in various ways without departing from the spirit and teachings of this application. All such combinations and / or combinations fall within the scope of this application.

[0141] The embodiments of this application have been described above. However, these embodiments are merely illustrative and not intended to limit the scope of this application. Although various embodiments have been described above, this does not mean that the measures in the various embodiments cannot be used advantageously in combination. Without departing from the scope of this application, those skilled in the art can make various substitutions and modifications, all of which should fall within the scope of this application.

Claims

1. A circuit board, characterized by, include: Switching unit; An accelerator card unit is connected to the switching unit, and the first memory of the accelerator card unit is configured to store the first model parameters of the first expert model. An extended storage unit is connected to the switching unit, and the extended storage unit is configured to store the second model parameters of the second expert model; the call frequency of the first expert model and the call frequency of the second expert model are different; A monitoring unit is connected to the accelerator card unit and connected to a first control device via multiple buses; the data transmission speeds of the multiple buses are different, so that the monitoring unit can transmit the call frequency information of the accelerator card unit calling different expert models to the first control device at different transmission speeds.

2. The circuit board according to claim 1, characterized in that, The accelerator card unit is configured to: obtain the first model parameters from the first memory to call the first expert model to perform a model processing task; and / or, obtain the second model parameters from the extended storage unit via the switching unit to call the second expert model to perform a model processing task; the call frequency of the first expert model is higher than the call frequency of the second expert model.

3. The circuit board according to claim 2, characterized in that, The monitoring unit is also configured to collect information on the different call frequencies of different expert models by the accelerator card unit, and transmit the different call frequency information to the first control device via the multiple buses respectively.

4. The circuit board according to claim 3, characterized in that, The monitoring unit is further configured to select the plurality of buses based on the storage location of the model parameters of the different expert models, thereby transmitting the different call frequency information to the first control device via the plurality of buses, so that the first control device can determine the scheduling strategy of the circuit board based on the received call frequency information.

5. The circuit board of claim 4, wherein, In the first model processing flow of the accelerator card unit, the first memory is configured to store the first model parameters, and the extended storage unit is configured to store the second model parameters; The monitoring unit is further configured to send the different call frequency information to the first control device via the plurality of buses, so that the first control device determines the second model processing flow after the first model processing flow based on the received call frequency information, and determines the scheduling strategy based on the second model processing flow. Wherein, at least one of the first model parameter and the second model parameter is stored in a different location in the second model processing flow than in the first model processing flow.

6. The circuit board of claim 5, wherein Also includes: The control unit, connected to the switching unit, is configured to: Receive the scheduling strategy from the first control device; Under the control of the scheduling strategy, the first model parameters are sent to the extended storage unit via the switching unit to write the first model parameters into the extended storage unit, thereby changing the storage location of the first model parameters; And / or, The second model parameters are sent to the accelerator card unit via the switching unit to write the second model parameters into the first memory, thereby changing the storage location of the second model parameters; After the storage location is changed, the transmission speed of the call frequency information of the first expert model and the second expert model is matched with the processing flow of the second model.

7. The circuit board of claim 6, wherein The control unit is also configured to: Upon receiving the scheduling policy, the system updates its own initial scheduling policy to the specified scheduling policy.

8. The circuit board of claim 7, wherein, The control unit is further configured to: According to the initial scheduling strategy, the initial scheduling instruction, the first model parameter, and the second model parameter are sent to the switching unit; The switching unit is configured to operate under the control of the initial scheduling instruction: Send the first model parameters to the accelerator card unit to write the first model parameters into the first memory of the accelerator card unit; The second model parameters are sent to the extended storage unit to write the second model parameters into the extended storage unit.

9. The circuit board according to any one of claims 6 to 8, characterized by The second memory of the control unit is configured to store the third model parameters of the third expert model; The accelerator card unit is also configured to: Under the control of control instructions received from the control unit via the switching unit, the third model parameters are retrieved from the second memory; and Based on the third model parameters, the third expert model is invoked to perform the model processing task.

10. The circuit board of claim 9, wherein, The first expert model is called more frequently than the third expert model, and the third expert model is called more frequently than the second expert model.

11. The circuit board according to any one of claims 3 to 8, characterized by The plurality of buses includes a first bus and a second bus; the monitoring unit is connected to the first control device via the first bus and the second bus; The monitoring unit is also configured to: The frequency information of the model parameters stored in the first memory is sent to the first control device via the first bus at a first transmission speed. and / or The frequency information of the model parameters stored in the extended storage unit is sent to the first control device via the second bus at a second transmission speed. The first transmission speed is higher than the second transmission speed.

12. A computing system, comprising: Multiple circuit boards as described in any one of claims 1 to 11; as well as The first control device is configured to collect the call frequency information of expert models configured in multiple circuit boards, and determine the scheduling strategy of multiple circuit boards based on the received call frequency information.

13. The computing system of claim 12, wherein, The plurality of circuit boards include a first circuit board and a second circuit board, wherein the first circuit board is configured to store target model parameters; the model parameters stored on the first circuit board and the model parameters stored on the second circuit board are respectively configured to perform different types of model processing tasks. The first circuit board is electrically connected to the second circuit board and is configured to send the target model parameters to the second circuit board; The second circuit board is configured to invoke the target expert model to perform a model processing task of the target type based on the target model parameters.

14. The computing system according to claim 13, characterized in that, The first control device is also configured to: Send the target invocation frequency information of the target expert model from the first circuit board to the terminal device; In response to receiving a target scheduling policy from the terminal device corresponding to the target call frequency information, the target scheduling policy is sent to the first circuit board to control the first circuit board to send the target model parameters to the second circuit board.

15. The computing system of claim 14, wherein, The first control device further includes a device control unit and a system switching unit, wherein the system switching unit is electrically connected to the switching units of multiple circuit boards respectively; The device control unit is electrically connected to the system switching unit and is configured to send the target scheduling policy to the system switching unit, so that the switching unit of the first circuit board receives the target scheduling policy through the system switching unit and writes the target scheduling policy into the first circuit board through the switching unit of the first circuit board.

16. The computing system according to any one of claims 14-15, characterized in that, The accelerator card unit of the first circuit board is configured to send the target model parameters to the accelerator card unit of the second circuit board to write the target model parameters into the first memory of the accelerator card unit of the second circuit board.

17. The computing system of claim 16, wherein, The target scheduling strategy includes a first target scheduling strategy; The switching unit of the first circuit board is configured to send the target model parameters to the acceleration card unit of the first circuit board based on the first target scheduling strategy.

18. The computing system of any of claims 14-15, wherein, The target scheduling strategy includes a second target scheduling strategy; The switching unit of the first circuit board is configured to send the target model parameters to the switching unit of the second circuit board based on the second target scheduling strategy, so as to send the target model parameters to the extended storage unit of the second circuit board via the switching unit of the second circuit board, so as to write the target model parameters into the extended storage unit of the second circuit board.

19. The computing system of any of claims 13-15, wherein, The first circuit board is adjacent to the second circuit board, and the expert model of the first circuit board is called more times than the expert model of the second circuit board.

20. The computing system of any of claims 13-15, wherein, The first control device includes: The acquisition unit, electrically connected to the monitoring unit of any circuit board via a first bus and a second bus, is configured to acquire first call frequency information from the monitoring unit via the first bus and second call frequency information from the monitoring unit via the second bus; The device control unit is configured to determine the scheduling strategy for any circuit board based on the first call frequency information and the second call frequency information.

21. A computer cabinet, characterized by include: Multiple computing systems as described in any one of claims 12 to 20; as well as The second control device is configured to receive call frequency information from the first control device of multiple computing systems, and determine the scheduling strategy of the multiple computing systems based on the received call frequency information.

22. The computer cabinet of claim 21, wherein, The plurality of computing systems includes a first computing system and a second computing system; The computer cabinet also includes a data acquisition device, electrically connected to the circuit boards of the first computing system and the second computing system, configured as follows: Collect power consumption information of the circuit boards of the first computing system and the second computing system; Send the power consumption information to the terminal device; The scheduling strategy corresponding to the power consumption information received from the terminal device is sent to the circuit board of the first computing system; The circuit board of the first computing system is configured to send the model parameters of the circuit board of the first computing system to the circuit board of the second computing system according to a scheduling strategy corresponding to the power consumption information. The circuit board of the second computing system is configured to call the expert model of the circuit board of the first computing system to perform model processing tasks based on the model parameters of the circuit board of the first computing system.

23. The computer cabinet of claim 22, wherein, The circuit board of the first computing system includes a switching unit and a communication unit of the first computing system circuit board electrically connected to the first computing system circuit board, and the circuit board of the second computing system includes a communication unit of the second computing system circuit board electrically connected to the second computing system circuit board via the switching unit of the second computing system circuit board and an extended storage unit of the second computing system circuit board. The switching unit of the first computing system's circuit board is configured to send the model parameters of the first computing system's circuit board to the communication unit of the first computing system's circuit board based on a scheduling strategy corresponding to the power consumption information, so as to send the model parameters of the first computing system's circuit board to the communication unit of the second computing system's circuit board via the communication unit of the first computing system's circuit board, so as to write the model parameters of the first computing system's circuit board into the extended storage unit of the second computing system's circuit board via the switching unit of the second computing system's circuit board through the communication unit of the second computing system's circuit board.

24. A computing method, comprising: Applied to the circuit board as described in any one of claims 1 to 11, The method includes: The accelerator card unit of the circuit board obtains the first model parameters from the memory of the accelerator card unit of the circuit board, and / or obtains the second model parameters from the extended storage unit of the circuit board via the switching unit of the circuit board. The accelerator card unit of the circuit board calls at least one of the first expert model and the second expert model to perform model processing tasks based on the first model parameters and / or the second model parameters, wherein the calling frequency of the first expert model is higher than that of the second expert model. The monitoring unit of the circuit board transmits information on the frequency of calls to different expert models by the accelerator card unit of the circuit board to the first control device at different transmission speeds.

25. The computing method of claim 24, wherein, The monitoring unit of the circuit board transmits information on the frequency of calls to different expert models by the accelerator card unit of the circuit board to the first control device at different transmission speeds, including: The monitoring unit of the circuit board selects multiple buses based on the storage location of the model parameters of the different expert models, and transmits different call frequency information to the control device through the multiple buses, so that the control device can determine the scheduling strategy of the circuit board based on the received call frequency information.

26. The computing method of claim 25, wherein, In the first model processing flow of the accelerator card unit, the memory is configured to store the first model parameters, and the extended storage unit is configured to store the second model parameters; The control device determines the scheduling strategy for the circuit board based on the received call frequency information, including: The control device determines the second model processing flow following the first model processing flow based on the received call frequency information, and determines the scheduling strategy based on the second model processing flow. Wherein, at least one of the first model parameter and the second model parameter is stored in a different location in the second model processing flow than in the first model processing flow.

27. The computing method of claim 26, wherein, The method further includes: The control unit of the circuit board receives the scheduling strategy from the control device; Under the control of the scheduling strategy, the control unit of the circuit board sends the first model parameter to the extended storage unit via the switching unit to write the first model parameter into the extended storage unit, thereby changing the storage location of the first model parameter; and / or, sends the second model parameter to the accelerator card unit via the switching unit to write the second model parameter into the memory, thereby changing the storage location of the second model parameter. After the storage location is changed, the transmission speed of the call frequency information of the first expert model and the second expert model is matched with the processing flow of the second model.

28. The computing method of any one of claims 25-27, wherein, The plurality of buses includes a first bus and a second bus; The transmission of the different call frequency information to the control device via multiple buses includes: The monitoring unit of the circuit board sends the call frequency information of the model parameters stored in the memory to the control device via the first bus at a first transmission speed; and / or sends the call frequency information of the model parameters stored in the extended storage unit to the control device via the second bus at a second transmission speed. The first transmission speed is higher than the second transmission speed.

29. The computing method of any one of claims 25-27, wherein, The method further includes, when the accelerator card unit of the circuit board performs a model processing task based on one of the first model parameters and the second model parameters: the accelerator card unit of the circuit board sends another model parameter, one of the first model parameters and the second model parameters, to other circuit boards to write the other model parameter into the other circuit boards, so that the other circuit boards perform a model processing task based on the other model parameter; wherein, the number of times the expert model of the circuit board is called is higher than the number of times the expert model of the other circuit boards is called.