Automatic model parallel scheduling strategy generation method and device based on heterogeneous computing power
By analyzing task characteristics and acceleration card performance, generating task scheduling strategies, and formulating data communication routing strategies based on network topology, the problems of acceleration card load imbalance and network congestion in heterogeneous computing power systems are solved, and system performance and stability are improved sex.
Patent Information
- Application Number
- CN202410924027.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-10
- Publication Date
- 2024-11-12
Smart Images

Figure CN118939391A_ABST
Abstract
Description
Technical Field
[0001] The present disclosure belongs to the field of heterogeneous computing technology, and specifically relates to a method and device for generating an automatic model parallel scheduling strategy based on heterogeneous computing power. Background Art
[0002] In the parallel scheduling of heterogeneous computing power automatic models, different types of accelerator cards need to work together, which involves issues such as task scheduling, data communication, storage, and synchronization optimization. These issues have a significant impact on system performance, power consumption, reliability, and other indicators. In order to solve the application development and system optimization problems of heterogeneous systems, many parallel programming models for heterogeneous computing architectures have emerged in recent years.
[0003] However, the differences in task division, scheduling, communication, storage, synchronization and other mechanisms among different computing accelerator cards have brought new challenges to system management. Summary of the invention
[0004] The present disclosure aims to solve at least one of the technical problems existing in the prior art, and provides a method and device for generating an automatic model parallel scheduling strategy based on heterogeneous computing power.
[0005] In a first aspect, an embodiment of the present disclosure provides a method for generating an automatic model parallel scheduling strategy based on heterogeneous computing power, and the method for generating an automatic model parallel scheduling strategy based on heterogeneous computing power includes:
[0006] Analyze task characteristics based on input model information;
[0007] Evaluate the current real-time available performance of different accelerator cards based on their remaining memory, memory bandwidth, and computing power;
[0008] A task scheduling strategy is generated according to the task characteristics and the current real-time available performance of different accelerator cards.
[0009] In some embodiments, based on the remaining memory, memory bandwidth, and computing power of different accelerator cards, the current real-time available performance of different accelerator cards is evaluated, including:
[0010] Use rap=leftMem*α+computility*β+bandWidth*γ to calculate the current real-time availability of different accelerator cards;
[0011] Among them, rap represents the current real-time availability of the accelerator card, leftMem represents the remaining space of the accelerator card, α represents the weighted proportion of the remaining memory of the accelerator card, computility represents the computing power of the accelerator card, β represents the weighted proportion of the computing power of the accelerator card, bandWidth represents the memory bandwidth of the accelerator card, and γ represents the weighted proportion of the memory bandwidth of the accelerator card.
[0012] In some embodiments, the current real-time available performance of different accelerator cards is evaluated based on the remaining memory, memory bandwidth, and computing power of different accelerator cards, and then the following is further included:
[0013] Develop data communication routing strategies based on network topology, traffic patterns, and link quality;
[0014] A task scheduling strategy is generated according to the generated data communication routing strategy.
[0015] In some embodiments, generating a data communication routing strategy according to a network topology, a traffic pattern, and a link quality includes:
[0016] Collect network traffic data in real time;
[0017] Analyzing the network traffic data to identify traffic patterns and congestion points;
[0018] Develop data communication routing strategies based on traffic patterns and congestion points.
[0019] In some embodiments, the current real-time available performance of different accelerator cards is evaluated based on the remaining memory, memory bandwidth, and computing power of different accelerator cards, which also includes:
[0020] According to the port information of different acceleration cards, the communication protocols of the different acceleration cards are integrated.
[0021] In some embodiments, the task characteristics include: model size, data volume, and computational requirements of the task.
[0022] In a second aspect, an embodiment of the present disclosure provides an automatic model parallel scheduling strategy generation device based on heterogeneous computing power, and the automatic model parallel scheduling strategy generation device based on heterogeneous computing power includes:
[0023] An analysis module, configured to analyze task characteristics based on input model information;
[0024] A real-time availability evaluation module is configured to evaluate the current real-time availability performance of different accelerator cards according to the remaining memory, memory bandwidth, and computing power of the different accelerator cards;
[0025] The task scheduling strategy generating module is configured to generate a task scheduling strategy according to the task characteristics and the current real-time available performance of different acceleration cards.
[0026] In some embodiments, the automatic model parallel scheduling strategy generation device based on heterogeneous computing power further includes:
[0027] a communication routing strategy formulation module, configured to formulate a data communication routing strategy based on a network topology, a traffic pattern, and a link quality;
[0028] The task scheduling strategy generating module is further configured to generate a task scheduling strategy according to the generated data communication routing strategy.
[0029] In a third aspect, an embodiment of the present disclosure provides a computer device, the computer device comprising:
[0030] one or more processors;
[0031] A memory for storing one or more programs;
[0032] When the one or more programs are executed by the one or more processors, the one or more processors implement the automatic model parallel scheduling strategy generation method based on heterogeneous computing power as provided above;
[0033] One or more I / O interfaces are connected between the processor and the memory and are configured to implement information interaction between the processor and the memory.
[0034] In a fourth aspect, an embodiment of the present disclosure provides a computer non-volatile readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the automatic model parallel scheduling strategy generation method based on heterogeneous computing power provided above is executed. BRIEF DESCRIPTION OF THE DRAWINGS
[0035] Figure 1 A flowchart of a method for generating an automatic model parallel scheduling strategy based on heterogeneous computing power is provided in an embodiment of the present disclosure.
[0036] Figure 2 A flowchart of another method for generating automatic model parallel scheduling strategies based on heterogeneous computing power provided in an embodiment of the present disclosure.
[0037] Figure 3 A flowchart of a method for formulating a data communication routing strategy provided by an embodiment of the present disclosure.
[0038] Figure 4 A schematic diagram of the structure of an automatic model parallel scheduling strategy generation device based on heterogeneous computing power provided in an embodiment of the present disclosure.
[0039] Figure 5 A schematic diagram of the structure of a computer device provided in some embodiments of the present disclosure. DETAILED DESCRIPTION
[0040] In order to enable those skilled in the art to better understand the technical solution of the present disclosure, the present disclosure is further described in detail below in conjunction with the accompanying drawings and specific embodiments. Obviously, the described embodiments are only part of the embodiments of the present disclosure, rather than all of the embodiments. The components of the embodiments of the present disclosure generally described and shown in the drawings herein can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present disclosure provided in the drawings is not intended to limit the scope of the present disclosure claimed for protection, but merely represents selected embodiments of the present disclosure. Based on the embodiments of the present disclosure, all other embodiments obtained by those skilled in the art without making creative efforts are within the scope of protection of the present disclosure. .
[0041] Unless otherwise defined, the technical terms or scientific terms used in the present disclosure should be understood by people with ordinary skills in the field to which the present disclosure belongs. The "first", "second" and similar words used in the present disclosure do not indicate any order, quantity or importance, but are only used to distinguish different components. Similarly, similar words such as "one", "one" or "the" do not indicate quantity restrictions, but indicate that there is at least one. Similar words such as "include" or "comprise" mean that the elements or objects appearing before the word cover the elements or objects listed after the word and their equivalents, without excluding other elements or objects. Similar words such as "connect" or "connected" are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. "Up", "down", "left", "right" and the like are only used to indicate relative positional relationships. When the absolute position of the described object changes, the relative positional relationship may also change accordingly.
[0042] For ease of understanding, special terms involved in the embodiments of the present disclosure are first described in detail.
[0043] Heterogeneous computing power is a computing system composed of computing units with different types of instruction sets and architectures, designed to optimize performance and cost. This computing method coordinates the use of machines with different performance and structures to meet different computing needs, so that codes or code segments are executed in a way that maximizes overall performance. Heterogeneous computing power has a wide range of application scenarios, including but not limited to data science acceleration, diversified computing power needs in the intelligent era, high-performance computing, artificial intelligence, Internet of Things and edge computing, 5G communications, and multimedia processing.
[0044] Accelerator cards generally refer to physics processors, which are microprocessors used in computers to process physical calculations, especially in video games. Physical effects such as rigid body dynamics, collision detection, fluid dynamics, hair simulation, clothing simulation, and object destruction can all be calculated using this processor. This concept comes from reducing the burden on the central processing unit (CPU), which is the same as using the graphics core to replace the CPU for graphics calculations.
[0045] In the automatic model parallel scheduling of heterogeneous computing power, different types of accelerator cards need to work together. However, the current automatic model parallel scheduling of heterogeneous computing power has an unbalanced load when assigning tasks to different accelerator cards. It is easy for some accelerator cards to be overloaded and some accelerator cards to be relatively idle, resulting in low resource utilization. At the same time, the data transmission and communication mechanisms between different accelerator cards may be different, which is prone to network congestion, data transmission problems, and increased data communication overhead.
[0046] In order to solve at least one of the above-mentioned technical problems, the embodiments of the present disclosure provide a method and device for generating an automatic model parallel scheduling strategy based on heterogeneous computing power. The following will further describe in detail the method and device for generating an automatic model parallel scheduling strategy based on heterogeneous computing power provided by the embodiments of the present disclosure in combination with the accompanying drawings and specific implementation methods.
[0047] In a first aspect, the embodiments of the present disclosure provide a method for generating an automatic model parallel scheduling strategy based on heterogeneous computing power. Figure 1 A flow chart of a method for generating an automatic model parallel scheduling strategy based on heterogeneous computing power provided by an embodiment of the present disclosure is shown in FIG. Figure 1 As shown, the method for generating an automatic model parallel scheduling strategy based on heterogeneous computing power includes the following steps S101 to S103.
[0048] S101, analyzing task characteristics according to input model information.
[0049] In the above step S101, different model information may be input into the system, wherein the task characteristics may include: model size, data volume, task computing requirements, etc. It is understandable that the task characteristics may also include: communication efficiency, power consumption, etc., and one or more of the above factors may be selected according to actual needs to accurately analyze the task characteristics.
[0050] S102, evaluating the current real-time available performance of different accelerator cards according to the remaining memory, memory bandwidth, and computing power of different accelerator cards.
[0051] In the above step S102, the remaining memory of the accelerator card refers to the remaining available memory space of the accelerator card. A larger remaining memory can store larger data. The memory bandwidth of the accelerator card refers to the data transmission rate on the accelerator card. A larger memory bandwidth can transmit data faster. The computing power of the accelerator card refers to the theoretical maximum number of floating-point operations per second that a computer has. A larger computing power means a faster calculation speed. Since different types of accelerator cards have different parameters and may come from different manufacturers, their current real-time availability is also different.
[0052] S103, generating a task scheduling strategy according to the task characteristics and the current real-time available performance of different accelerator cards.
[0053] In the above step S103, different tasks can be assigned to different accelerator cards for calculation according to the task characteristics and the current real-time availability of different accelerator cards to generate a task scheduling strategy. For example, tasks with larger model size, larger data volume, and larger task calculation requirements are assigned to accelerator cards with larger remaining memory, larger memory bandwidth, and stronger computing power, and tasks with smaller model size, smaller data volume, and smaller task calculation requirements are assigned to accelerator cards with smaller remaining memory, smaller memory bandwidth, and weaker computing power.
[0054] In the automatic model parallel scheduling strategy generation method based on heterogeneous computing power provided by the embodiment of the present disclosure, by accurately evaluating the real-time availability of different accelerator cards and automatically scheduling, the computing resources in the cluster can be used more efficiently, resource waste can be reduced, and tasks can be assigned to the accelerator card that best suits their needs for execution, thereby effectively improving the overall computing performance and processing speed of the system. At the same time, by dynamically adjusting task allocation, load balancing between different accelerator cards can be achieved, thereby avoiding the problem of some accelerator cards being overloaded while some accelerator cards are idle, thereby improving resource utilization and improving system stability.
[0055] In some embodiments, the current real-time available performance of different accelerator cards is evaluated based on the remaining memory, memory bandwidth and computing power of different accelerator cards, including: using rap=leftMem*α+computility*β+bandWidth*γ (Formula 1) to calculate the current real-time availability of different accelerator cards; wherein rap represents the current real-time availability of the accelerator card, leftMem represents the remaining space of the accelerator card, α represents the weight ratio of the remaining memory of the accelerator card, computility represents the computing power of the accelerator card, β represents the weight ratio of the computing power of the accelerator card, bandWidth represents the memory bandwidth of the accelerator card, and γ represents the weight ratio of the memory bandwidth of the accelerator card.
[0056] Using the above formula 1, the current real-time availability of different accelerator cards can be accurately evaluated. This is a novel performance quantification method that can dynamically evaluate the current performance status of the accelerator card. Among them, the weights of the above three factors can be the same or different. Generally, α+β+γ=1. It can be understood that the factors affecting the real-time availability of the accelerator card can include other factors in addition to the above three factors, and other influencing factors can be added according to actual needs. They will not be listed one by one here.
[0057] Figure 2 A flow chart of another method for generating an automatic model parallel scheduling strategy based on heterogeneous computing power provided by an embodiment of the present disclosure is shown in FIG. Figure 2 As shown, according to the remaining memory, memory bandwidth and computing power of different accelerator cards, the current real-time available performance of different accelerator cards is evaluated, and then steps S104 and S105 are included.
[0058] S104, formulating a data communication routing strategy according to the network topology, traffic pattern and link quality.
[0059] In the above step S104, a data communication routing strategy is formulated according to the network topology, traffic pattern and link quality to ensure that the data packets can be efficiently and reliably transmitted to the destination. At the same time, the optimal data transmission path for the task can be set according to the changes in network traffic monitored in real time. This data communication routing strategy is particularly suitable for heterogeneous computing environments with complex network environments and large traffic fluctuations.
[0060] S105: Generate a task scheduling strategy according to the generated data communication routing strategy.
[0061] In addition to generating a task scheduling strategy based on task characteristics and the current real-time available performance of different accelerator cards in step S103, the method for generating an automatic model parallel scheduling strategy based on heterogeneous computing power provided by the embodiment of the present disclosure may also include step S105, which may generate a task scheduling strategy based on the generated data communication routing strategy. In this way, the optimized data communication routing strategy reduces unnecessary data transmission, reduces communication delay and bandwidth consumption, and thus can improve task execution efficiency.
[0062] In some embodiments, Figure 3 A flow chart of a method for formulating a data communication routing strategy provided by an embodiment of the present disclosure is shown as follows: Figure 3 As shown, the above step S104 formulates a data communication routing strategy according to the network topology, traffic pattern and link quality, and specifically includes the following steps S1041 to S1043.
[0063] S1041, collect network traffic data in real time.
[0064] In the above step S1041, different acceleration cards can communicate data through routing, and the network traffic data monitoring device can be used to collect network traffic data in real time.
[0065] S1042, analyzing network traffic data to identify traffic patterns and congestion points.
[0066] In the above step S1042, the collected network traffic data may be analyzed, for example, the size of the network data and the data transmission speed between different routes may be analyzed, so that the traffic pattern and the congestion points between different routes may be identified.
[0067] S1043, formulate a data communication routing strategy based on the traffic pattern and congestion points.
[0068] In the above step S1043, during the data distribution process, the traffic patterns between different routes can be adapted to avoid transmitting large amounts of data between routes where congestion points occur. This reduces unnecessary data transmission, reduces communication delays and bandwidth consumption, and thus improves task execution efficiency.
[0069] In some embodiments, Figure 1 and Figure 2 As shown, step S101 evaluates the current real-time available performance of different accelerator cards according to the remaining memory, memory bandwidth and computing power of different accelerator cards, and also includes: step S100, integrating the communication protocols of different accelerator cards according to the port information of different accelerator cards.
[0070] In the above step S100, since different types of accelerator cards have different parameters and may come from different manufacturers, the communication protocols of different accelerator cards can be integrated according to the port information of different accelerator cards, so that different accelerator cards can use a unified communication protocol for data transmission, support accelerator cards of different types and performances, so that the system can be flexibly expanded to adapt to the ever-changing hardware environment.
[0071] In a second aspect, the embodiment of the present disclosure provides an automatic model parallel scheduling strategy generation device based on heterogeneous computing power, Figure 4 A schematic diagram of a structure of an automatic model parallel scheduling strategy generation device based on heterogeneous computing power provided by an embodiment of the present disclosure, such as Figure 4 As shown, the automatic model parallel scheduling strategy generation device based on heterogeneous computing power includes: an analysis module 401, a real-time availability evaluation module 402 and a task scheduling strategy generation module 403.
[0072] The analysis module 401 is configured to analyze the task characteristics according to the input model information.
[0073] Different model information can be input into the system, wherein the task characteristics may include: model size, data volume, task computing requirements and other factors. It is understandable that the task characteristics may also include: communication efficiency. Power consumption and other factors. One or more of the above factors may be selected according to actual needs to accurately analyze the task characteristics.
[0074] The real-time availability evaluation module 402 is configured to evaluate the current real-time availability performance of different accelerator cards according to the remaining memory, memory bandwidth and computing power of different accelerator cards.
[0075] The remaining memory of an accelerator card refers to the remaining available memory space of the accelerator card. A larger remaining memory can store larger data. The memory bandwidth of an accelerator card refers to the data transmission rate on the accelerator card. A larger memory bandwidth can transmit data faster. The computing power of an accelerator card refers to the theoretical maximum number of floating-point operations per second that a computer can have. A larger computing power means a faster computing speed. Since different types of accelerator cards have different parameters and may come from different manufacturers, their current real-time availability is also different.
[0076] The task scheduling strategy generating module 403 is configured to generate a task scheduling strategy according to the task characteristics and the current real-time available performance of different accelerator cards.
[0077] The task scheduling strategy generation module 403 can allocate different tasks to different accelerator cards for calculation according to the task characteristics and the current real-time availability of different accelerator cards to generate a task scheduling strategy. For example, tasks with larger model size, larger data volume, and larger task calculation requirements are allocated to accelerator cards with larger remaining memory, larger memory bandwidth, and stronger computing power, and tasks with smaller model size, smaller data volume, and smaller task calculation requirements are allocated to accelerator cards with smaller remaining memory, smaller memory bandwidth, and weaker computing power.
[0078] The automatic model parallel scheduling strategy generation device based on heterogeneous computing power provided by the embodiment of the present disclosure can more efficiently utilize the computing resources in the cluster, reduce resource waste, and ensure that tasks can be assigned to the most suitable acceleration card for execution, thereby effectively improving the overall computing performance and processing speed of the system. At the same time, by dynamically adjusting task allocation, load balancing between different acceleration cards can be achieved, thereby avoiding the problem of some acceleration cards being overloaded while some acceleration cards are idle, thereby improving resource utilization and improving system stability.
[0079] In some embodiments, Figure 4 As shown, the automatic model parallel scheduling strategy generation device based on heterogeneous computing power also includes: a communication routing strategy formulation module 404.
[0080] The communication routing strategy formulation module 404 is configured to formulate a data communication routing strategy according to the network topology, traffic pattern and link quality.
[0081] The communication routing strategy formulation module 404 can formulate a data communication routing strategy based on the network topology, traffic pattern and link quality, which can ensure that the data packet can be efficiently and reliably transmitted to the destination. At the same time, the optimal data transmission path for the task can be set according to the real-time monitored network traffic changes. This data communication routing strategy is particularly suitable for heterogeneous computing environments with complex network environments and large traffic fluctuations.
[0082] The task scheduling strategy generating module 403 is further configured to generate a task scheduling strategy according to the generated data communication routing strategy.
[0083] The task scheduling strategy generation module 403 can also generate a task scheduling strategy based on the generated data communication routing strategy. In this way, the optimized data communication routing strategy reduces unnecessary data transmission, reduces communication delay and bandwidth consumption, and thus can improve task execution efficiency.
[0084] In a third aspect, an embodiment of the present disclosure provides a computer device, Figure 5 is a schematic diagram of the structure of a computer device provided in some embodiments of the present disclosure, such as Figure 5 As shown, the computer device includes: one or more processors 501; a memory 502, used to store one or more programs, when the one or more programs are executed by one or more processors, the one or more processors implement the automatic model parallel scheduling strategy generation method based on heterogeneous computing power provided by any of the above embodiments; one or more I / O interfaces 503, connected between the processor and the memory, and configured to realize information interaction between the processor and the memory.
[0085] Among them, the processor 501 is a device with data processing capabilities, including but not limited to a central processing unit (CPU); the memory 502 is a device with data storage capabilities, including but not limited to random access memory (RAM, more specifically SDRAM, DDR, etc.), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory (FLASH); the I / O interface (read-write interface) 503 is connected between the processor 501 and the memory 502, and can realize information interaction between the processor 501 and the memory 502, including but not limited to a data bus (Bus), etc.
[0086] In some embodiments, the processor 501 , the memory 502 , and the I / O interface 503 are connected to each other through a bus, and further connected to other components of the computing device.
[0087] In a fourth aspect, the present embodiment provides a computer non-volatile readable storage medium, on which a computer program is stored. When the program is executed by a processor, the method for generating an automatic model parallel scheduling strategy based on heterogeneous computing power provided in any of the above embodiments is implemented.
[0088] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a machine-readable medium, and the computer program contains a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication part, and / or installed from a removable medium. When the computer program is executed by a central processing unit (CPU), the above-mentioned functions defined in the system of the present disclosure are executed.
[0089] It should be noted that the computer non-transient readable medium shown in the present disclosure may be a computer readable signal medium or a computer readable storage medium or any combination of the above two. The computer readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, a computer readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, device or device. In the present disclosure, a computer readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer readable program code. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable non-transient storage medium other than a computer-readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device. The program code contained on the computer-readable non-transient storage medium may be transmitted using any suitable medium, including but not limited to: wireless, wire, optical cable, RF, etc., or any suitable combination of the above.
[0090] The flow charts and block diagrams in the accompanying drawings illustrate the possible architecture, functions and operations of the devices, methods and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flow chart or block diagram can represent a module, a program segment, or a part of a code, and the aforementioned module, program segment, or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two connected boxes can actually represent that they are executed in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flow chart, and the combination of the boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.
[0091] The circuits or subcircuits involved in the embodiments described in the present disclosure may be implemented by software or by hardware. The described circuits or subcircuits may also be arranged in a processor, for example, may be described as: a processor comprising: a receiving circuit and a processing circuit, the processing module comprising a writing subcircuit and a reading subcircuit. The names of these circuits or subcircuits do not, in some cases, constitute limitations on the circuits or subcircuits themselves, for example, a receiving circuit may also be described as "receiving a signal".
[0092] It is to be understood that the above embodiments are merely exemplary embodiments used to illustrate the principles of the present disclosure, but the present disclosure is not limited thereto. For those of ordinary skill in the art, various modifications and improvements can be made without departing from the spirit and substance of the present disclosure, and these modifications and improvements are also considered to be within the scope of protection of the present disclosure.
Claims
1. A method for generating automatic model parallel scheduling strategies based on heterogeneous computing power. It is characterized in that The method for generating an automatic model parallel scheduling strategy based on heterogeneous computing power includes: Analyze task characteristics based on input model information; Evaluate the current real-time available performance of different accelerator cards based on their remaining memory, memory bandwidth, and computing power; A task scheduling strategy is generated according to the task characteristics and the current real-time available performance of different accelerator cards.
2. The method for generating an automatic model parallel scheduling strategy based on heterogeneous computing power according to claim 1, It is characterized in that Evaluate the current real-time available performance of different accelerator cards based on their remaining memory, memory bandwidth, and computing power, including: Use rap=leftMem*α+computility*β+bandWidth*γ to calculate the current real-time availability of different accelerator cards; Among them, rap represents the current real-time availability of the accelerator card, leftMem represents the remaining space of the accelerator card, α represents the weighted proportion of the remaining memory of the accelerator card, computility represents the computing power of the accelerator card, β represents the weighted proportion of the computing power of the accelerator card, bandWidth represents the memory bandwidth of the accelerator card, and γ represents the weighted proportion of the memory bandwidth of the accelerator card.
3. The method for generating automatic model parallel scheduling strategy based on heterogeneous computing power according to claim 1, It is characterized in that Evaluate the current real-time available performance of different accelerator cards based on their remaining memory, memory bandwidth, and computing power, followed by: Develop data communication routing strategies based on network topology, traffic patterns, and link quality; A task scheduling strategy is generated according to the generated data communication routing strategy.
4. The method for generating an automatic model parallel scheduling strategy based on heterogeneous computing power according to claim 3, It is characterized in that The generating of data communication routing strategies according to network topology, traffic patterns and link quality includes: Collect network traffic data in real time; Analyzing the network traffic data to identify traffic patterns and congestion points; Develop data communication routing strategies based on traffic patterns and congestion points.
5. The method for generating automatic model parallel scheduling strategy based on heterogeneous computing power according to claim 1, It is characterized in that Evaluate the current real-time available performance of different accelerator cards based on their remaining memory, memory bandwidth, and computing power. Previously, this also included: According to the port information of different acceleration cards, the communication protocols of the different acceleration cards are integrated.
6. The method for generating automatic model parallel scheduling strategy based on heterogeneous computing power according to claim 1, It is characterized in that The task characteristics include: model size, data volume, and task computing requirements.
7. An automatic model parallel scheduling strategy generation device based on heterogeneous computing power, It is characterized in that The automatic model parallel scheduling strategy generation device based on heterogeneous computing power includes: An analysis module, configured to analyze task characteristics based on input model information; A real-time availability evaluation module is configured to evaluate the current real-time availability performance of different accelerator cards according to the remaining memory, memory bandwidth, and computing power of the different accelerator cards; The task scheduling strategy generating module is configured to generate a task scheduling strategy according to the task characteristics and the current real-time available performance of different acceleration cards.
8. The automatic model parallel scheduling strategy generation device based on heterogeneous computing power according to claim 7, It is characterized in that The automatic model parallel scheduling strategy generation device based on heterogeneous computing power also includes: a communication routing strategy formulation module, configured to formulate a data communication routing strategy based on a network topology, a traffic pattern, and a link quality; The task scheduling strategy generating module is further configured to generate a task scheduling strategy according to the generated data communication routing strategy.
9. A computer device, It is characterized in that The computer device comprises: one or more processors; A memory for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the method for generating an automatic model parallel scheduling strategy based on heterogeneous computing power according to any one of claims 1 to 6; One or more I / O interfaces are connected between the processor and the memory and are configured to implement information interaction between the processor and the memory.
10. A computer non-transitory readable storage medium, It is characterized in that The computer non-volatile readable storage medium stores a computer program, and when the computer program is executed by a processor, the method for generating an automatic model parallel scheduling strategy based on heterogeneous computing power according to any one of claims 1 to 6 is executed.
Citation Information
Cited By
Server, communication control method, communication method, device, medium and product
CN119232657A
Intelligent computing center computing power asymmetric collaborative scheduling method oriented to common computing power
CN119938284A
Intelligent Computing Center Computing Power Asymmetric Cooperative Scheduling Method for Inclusive Computing Power
CN119938284B
Method for acquiring acceleration card performance, acceleration card performance test equipment and product
CN120295883A
