Accelerator card hybrid deployment method and device and electronic equipment

Abstract

The invention provides an accelerator card hybrid deployment method and device and electronic equipment, and the method comprises the steps: carrying out the segmentation of an overall calculation graph, and obtaining a segmented calculation graph; inputting the calculation characteristic index of the segmentation calculation graph into a performance prediction model to obtain performance prediction data, output by the performance prediction model, of the segmentation calculation graph executed by various acceleration cards; and based on the performance prediction data of the segmentation calculation graph, carrying out acceleration card hybrid deployment on the target task. According to the method provided by the invention, the integral calculation graph is segmented to obtain the segmented calculation graph; inputting the calculation characteristic index of the segmentation calculation graph into a performance prediction model to obtain performance prediction data, output by the performance prediction model, of the segmentation calculation graph executed by various acceleration cards; based on the performance prediction data of the segmentation calculation graph, accelerator card hybrid deployment is performed on the target task, so that accurate accelerator card hybrid deployment is realized, and then the execution efficiency of executing the target task is improved and the power consumption required by execution is reduced in a hybrid deployment environment.

Description

Technical Field

[0001] The present invention relates to the technical field of accelerator cards, and in particular to a method, device, and electronic device for hybrid deployment of accelerator cards. Background Art

[0002] With the rapid development of artificial intelligence (AI) technology, the demand for AI computing is exploding. The computing power of a single accelerator card is no longer sufficient to meet this growing demand, prompting the emergence and rapid development of distributed training. However, the market currently offers a wide variety of accelerator devices, including GPUs, TPUs, FPGAs, and ASICs, each with its own advantages, and they are coexisting and developing. Against this backdrop, hybrid deployment solutions comprised of different AI accelerator cards are gaining increasing attention.

[0003] Therefore, how to accurately optimize hybrid deployment accelerator cards and give full play to the advantages of various accelerator cards to improve the execution efficiency of tasks in a hybrid deployment environment is a key issue that needs to be solved urgently and is extremely challenging. Summary of the Invention

[0004] The present invention provides a method, device and electronic device for hybrid deployment of accelerator cards, which are used to solve the defect of low task execution efficiency in a hybrid deployment environment of accelerator cards in the prior art.

[0005] The present invention provides a hybrid deployment method for accelerator cards, comprising: Get the overall computational graph of the target task; Segmenting the overall computation graph to obtain a segmented computation graph; Inputting the computational characteristic index of the segmented computation graph into a performance prediction model to obtain performance prediction data of various accelerator cards executing the segmented computation graph respectively output by the performance prediction model; Performing hybrid deployment of accelerator cards for the target task based on the performance prediction data of the segmented computation graph; The performance prediction model is constructed based on a neural network.

[0006] According to a hybrid deployment method for accelerator cards provided by the present invention, the performance prediction model training step includes: Obtain sample computational characteristic metrics of the test split computation graph, as well as an initial performance prediction model; Executing the test split calculation graph based on the various types of accelerator cards to collect actual performance data of the various types of accelerator cards; Inputting the sample computing characteristic index into the initial performance prediction model to obtain sample performance prediction data of each type of accelerator card executing the test split computing graph, output by the initial performance prediction model; Calculating a prediction loss based on the sample performance prediction data and the actual performance data; Based on the prediction loss, the model parameters of the initial performance prediction model are adjusted, and the performance prediction model is obtained through training.

[0007] According to a hybrid deployment method for accelerator cards provided by the present invention, the actual performance data includes at least one of: computing time, energy consumption, memory occupancy rate, and communication delay time.

[0008] According to a hybrid deployment method for accelerator cards provided by the present invention, hybrid deployment of accelerator cards for the target task based on the performance prediction data of the segmented computation graph includes: Screening the performance prediction data of the segmentation computation graph to obtain candidate performance prediction data of the segmentation computation graph; The accelerator card corresponding to the candidate performance prediction data is used as the target accelerator card for the segmented computation graph; Based on the target accelerator card, a segmentation calculation graph corresponding to the target accelerator card is executed to implement hybrid deployment of accelerator cards for the target task.

[0009] According to a hybrid deployment method for accelerator cards provided by the present invention, screening the performance prediction data of the segmented computation graph to obtain candidate performance prediction data of the segmented computation graph includes: Performing weighted calculation on each type of performance prediction data of the segmentation calculation graph to obtain a comprehensive performance score; The comprehensive performance scores are sorted, and the performance prediction data corresponding to the comprehensive performance score with the highest value is selected as the candidate performance prediction data for the segmentation calculation graph.

[0010] According to a hybrid deployment method for accelerator cards provided by the present invention, segmenting the overall computation graph to obtain a segmented computation graph includes: Obtaining the parallel scale of each operator in the overall computation graph; Based on the parallel scale, the overall computation graph is segmented to obtain the segmented computation graph.

[0011] According to a hybrid deployment method of accelerator cards provided by the present invention, the computing characteristic indicators of the split computing graph include: input and output dimensions, input and output data size, number of operations, and at least one of parallel scale.

[0012] According to a hybrid deployment method for accelerator cards provided by the present invention, the target task includes a training task of a target model, and the target model is constructed based on a deep learning algorithm.

[0013] The present invention also provides an accelerator card hybrid deployment device, comprising: Get the unit to get the overall computational graph of the target task; A segmentation unit, which segments the overall computation graph to obtain a segmented computation graph; a performance prediction unit, inputting the computational characteristic index of the segmented computation graph into a performance prediction model, and obtaining performance prediction data output by the performance prediction model for each type of accelerator card executing the segmented computation graph; A hybrid deployment unit, which performs hybrid deployment of accelerator cards for the target task based on the performance prediction data of the segmented computation graph; The performance prediction model is constructed based on a neural network.

[0014] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the accelerator card hybrid deployment method described in any one of the above is implemented.

[0015] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements any of the above-described accelerator card hybrid deployment methods.

[0016] The present invention also provides a computer program product, including a computer program, which, when executed by a processor, implements any of the above-mentioned accelerator card hybrid deployment methods.

[0017] The accelerator card hybrid deployment method, device, and electronic device provided by the present invention obtain a segmented calculation graph by segmenting the overall calculation graph; inputting the computing characteristic indicators of the segmented calculation graph into a performance prediction model to obtain performance prediction data of various types of accelerator cards that respectively execute the segmented calculation graph output by the performance prediction model; based on the performance prediction data of the segmented calculation graph, accelerator cards are hybrid deployed for target tasks, thereby achieving accurate hybrid deployment of accelerator cards, and thereby improving the execution efficiency of executing the target tasks and reducing the power consumption required for execution in a hybrid deployment environment. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the technical solutions in the present invention or the prior art, a brief introduction is given below to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0019] Figure 1 This is one of the flow diagrams of the hybrid deployment method of accelerator cards provided by the present invention; Figure 2 It is a flow chart of the method for obtaining performance prediction data provided by the present invention; Figure 3 This is the second flow chart of the hybrid deployment method of accelerator cards provided by the present invention; Figure 4 It is a structural diagram of the accelerator card hybrid deployment device provided by the present invention; Figure 5 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0020] To make the objectives, technical solutions, and advantages of the present invention more clear, the technical solutions of the present invention will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the embodiments described are only some of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts shall fall within the scope of protection of the present invention.

[0021] The present invention provides a hybrid deployment method for accelerator cards to achieve precise hybrid deployment of accelerator cards. Figure 1 This is one of the flow charts of the hybrid deployment method of the accelerator card provided by the present invention, such as Figure 1 As shown, the method includes: Step 110, obtaining the overall computation graph of the target task; Here, the target task refers to a computationally intensive task, such as training a deep learning model or scientific computing. Furthermore, the overall computational graph refers to the directed acyclic graph (DAG) containing all the computational nodes of the target task, where a computational node can refer to a single operator.

[0022] Specifically, the static graph mode or dynamic graph capture tool of a deep learning framework can be used to automatically extract the operators and data dependencies in the task model corresponding to the target task, thereby constructing the computational graph for the target task. For deep learning frameworks such as TensorFlow and PyTorch, the capture tool can be the PyTorch JIT. The operators can be used to perform convolutions and matrix multiplications. It can be understood that by obtaining the overall computational input of the target task, all computational operations of the target task are captured, enabling the subsequent hybrid deployment of accelerator cards for the target task.

[0023] Step 120, segmenting the overall computation graph to obtain a segmented computation graph; Here, a split computation graph refers to multiple subgraphs obtained by segmenting the overall computation graph. Specifically, a graph partitioning algorithm can be used to segment the overall computation graph based on metrics such as computational load balancing, communication overhead, and parallel scale, resulting in multiple split computation graphs. Thus, a single split computation graph can include a single or multiple operators. It is understood that the resulting split computation graphs can be optimized for accelerator card deployment for each split computation graph, thereby achieving precise mixed accelerator card deployment for the overall computation graph.

[0024] Step 130: Inputting the computational characteristic index of the segmentation computation graph into a performance prediction model to obtain performance prediction data of various accelerator cards executing the segmentation computation graph, as output by the performance prediction model; the performance prediction model is constructed based on a neural network; Here, computational characteristic metrics refer to quantitative parameters used to reflect the computational characteristics of a partitioned computation graph. These metrics can include computational effort, communication volume, and resource requirements. It is understood that by using computational characteristic metrics of a partitioned computation graph to clarify the computational and communication dimensions, the performance prediction model can comprehensively evaluate the performance of the partitioned computation graph on different types of accelerator cards.

[0025] The performance prediction data here refers to the expected execution metrics of the segmentation computation graph output by the performance prediction model on various types of accelerator cards, including execution time and resource consumption information. Accelerator cards here include GPUs, TPUs, FPGAs, ASICs, and other types of accelerator cards.

[0026] Specifically, for any segmentation computation graph, the computational characteristic indicators of the segmentation computation graph can be input into a performance prediction model, and the performance prediction model can extract features from the input computational characteristic indicators to obtain computational characteristic features. The computational characteristic features can then be input into a multi-layer perceptron or Transformer in the performance prediction model to output performance prediction values ​​for each type of accelerator card executing the segmentation computation graph, which can be used as performance prediction data. It is understood that the same method can be used for other segmentation computation graphs to obtain performance prediction data for each type of accelerator card executing the segmentation computation graph.

[0027] The performance prediction model is constructed based on a neural network. This can be achieved by constructing a sample dataset, training an initial neural network model, and adjusting the model parameters of the initial neural network model to obtain a final, applicable performance prediction model. For example, a dataset can be generated through offline benchmark testing, sample calculation characteristic indicators can be input, and the initial neural network model can output sample prediction performance data. The initial neural network model is trained until the mean square error converges, thereby obtaining the final performance prediction model.

[0028] It should be noted that compared to actually deploying the split calculations on different types of accelerator cards separately to obtain the actual performance data collected, by inputting the computational characteristics of the split calculation graph into the performance prediction model, the performance prediction data of each type of accelerator card executing the split calculation graph is obtained as output by the performance prediction model. This enables efficient and reliable performance prediction, thereby supporting real-time optimization decisions and realizing dynamic mixed deployment of accelerator cards.

[0029] Step 140: Performing hybrid deployment of accelerator cards for the target task based on the performance prediction data of the segmented computation graph; Specifically, various accelerator cards with better performance prediction data can be screened from the performance prediction data of the split calculation graph and used as target accelerator cards for executing the corresponding split calculation graph, so as to achieve mixed deployment of accelerator cards for the target task.

[0030] It should be noted that the performance prediction model outputs performance prediction data for each type of accelerator card executing the split calculation graph respectively. It can combine the advantages and hardware performance of different types of accelerator cards, give full play to the hardware performance of each type of accelerator card, enable it to execute the split calculation graph that better matches its hardware performance, optimize the mixed deployment of accelerator cards, and thus achieve accurate mixed deployment of accelerator cards.

[0031] Furthermore, by predicting the performance of various accelerator cards executing split computation graphs and achieving precise mixed deployment of accelerator cards for target tasks, we can achieve parallel execution of split computation graphs, reducing the overall execution time of target tasks and improving their efficiency. For example, distributed training is currently emerging and rapidly developing. If we can achieve precise mixed deployment of accelerator cards, we can greatly improve the efficiency of distributed training and reduce the power consumption required for training tasks in this hybrid deployment environment.

[0032] The method provided by an embodiment of the present invention obtains a segmented calculation graph by segmenting the overall calculation graph; inputs the calculation characteristic indicators of the segmented calculation graph into a performance prediction model to obtain performance prediction data of various types of accelerator cards output by the performance prediction model for respectively executing the segmented calculation graph; based on the performance prediction data of the segmented calculation graph, the accelerator cards are mixedly deployed for the target task, thereby achieving accurate mixed deployment of the accelerator cards, and thereby improving the execution efficiency of the target task and reducing the power consumption required for execution in a mixed deployment environment.

[0033] Based on any of the above embodiments, the step of training the performance prediction model includes: Obtain sample computational characteristic metrics of the test split computation graph, as well as an initial performance prediction model; Executing the test split calculation graph based on the various types of accelerator cards to collect actual performance data of the various types of accelerator cards; Inputting the sample computing characteristic index into the initial performance prediction model to obtain sample performance prediction data of each type of accelerator card executing the test split computing graph, output by the initial performance prediction model; Calculating a prediction loss based on the sample performance prediction data and the actual performance data; Based on the prediction loss, the model parameters of the initial performance prediction model are adjusted, and the performance prediction model is obtained through training.

[0034] The initial performance prediction model here can be obtained by obtaining a neural network model whose parameters are to be adjusted, such as a multilayer perceptron or Transformer. Furthermore, a test split computation graph refers to a computational subgraph obtained by splitting the overall computation graph corresponding to any test target task.

[0035] Specifically, first, during the data construction phase, we can obtain sample computational characteristic indicators of the test split computation graph and an initial performance prediction model. Then, we can actually deploy various accelerator cards to execute the test split computation graph and collect actual performance data from each accelerator card in real time. It is understood that this real-time performance data can serve as sample labels for the initial performance prediction model.

[0036] During the training phase of the performance prediction model, sample computing characteristic indicators are input into the initial performance prediction model, which then outputs sample performance prediction data for each accelerator card executing the test split computation graph. The prediction loss can then be calculated using the sample performance prediction data and the actual performance data. For example, this can be achieved by calculating the mean squared error (MSE). The model parameters of the initial performance prediction model can then be adjusted with the goal of reducing the prediction loss. Through multiple rounds of training iterations, the final performance prediction model is obtained.

[0037] In one embodiment, Figure 2 : is a flow chart of the method for obtaining performance prediction data provided by the present invention, such as Figure 2 As shown, first, different accelerator cards are trained using a test operator. Specifically, each accelerator card executes a test split computation graph, collecting actual performance data for each accelerator card. The training parameters are then saved. Specifically, the initial performance prediction model is fed with sample computational characteristic indicators to obtain sample performance prediction data for each accelerator card executing the test split computation graph. The initial performance prediction model then outputs sample performance prediction data for each accelerator card executing the test split computation graph. Prediction loss is calculated based on the sample performance prediction data and actual performance data. Based on the prediction loss, the model parameters of the initial performance prediction model are adjusted.

[0038] Furthermore, by obtaining the computational characteristic indicators of the segmented computation graph, the performance prediction model is used to output the performance prediction data of each type of accelerator card executing the segmented computation graph.

[0039] It should be noted that by actually collecting the actual performance data of various types of accelerator cards, constructing sample labels, and using the sample computing characteristic indicators of the test segmentation calculation graph as sample data, the initial performance prediction model is trained to adjust and optimize the model parameters of the initial performance prediction model, thereby obtaining an accurate performance prediction model, improving the accuracy of the output performance prediction data, and thereby improving the accuracy of hybrid deployment of accelerator cards.

[0040] Based on any of the above embodiments, the actual performance data includes at least one of: calculation time, energy consumption, memory occupancy rate and communication delay time.

[0041] Specifically, during the model training phase, actual performance data can be collected in real time when various accelerator cards execute the test split calculation graph. For example, the calculation time can be calculated by recording the timestamps before and after the accelerator card executes all operators in the test split calculation graph. In terms of energy consumption, the power consumption of the accelerator card when executing operators can be measured through external power monitoring equipment. In terms of memory usage, the memory usage of the accelerator card when executing all operators in the test split calculation graph can be viewed through memory monitoring tools. In addition, in terms of communication delay duration, the communication data packets between accelerator cards can be captured and analyzed through network analysis tools to measure the communication delay.

[0042] It should be noted that by obtaining actual performance data of at least one of the calculation time, energy consumption, memory occupancy rate and communication delay time, comprehensive performance data collection of the accelerator card execution test split calculation graph is achieved, so that the performance prediction data output by the trained performance prediction model is more comprehensive and more fully reflects the execution performance of the accelerator card, thereby enabling more accurate optimization of the hybrid deployment of accelerator cards.

[0043] Based on any of the foregoing embodiments, the hybrid deployment of the accelerator cards for the target task based on the performance prediction data of the segmented computation graph includes: Screening the performance prediction data of the segmentation computation graph to obtain candidate performance prediction data of the segmentation computation graph; The accelerator card corresponding to the candidate performance prediction data is used as the target accelerator card for the segmented computation graph; Based on the target accelerator card, a segmentation calculation graph corresponding to the target accelerator card is executed to implement hybrid deployment of accelerator cards for the target task.

[0044] Specifically, for performance prediction data of a single split computation graph, we can filter out candidate performance prediction data that meets specific performance requirements by selecting performance prediction data from various accelerator cards executing this split computation graph. For example, we can select performance prediction data with the shortest computation time and lowest memory usage as candidate performance prediction data. Alternatively, we can use a multi-objective optimization algorithm to select performance prediction data that performs well across multiple performance metrics as candidate performance prediction data.

[0045] The accelerator card corresponding to the candidate performance prediction data can then be used as the target accelerator card for each split computation graph. The split computation graph corresponding to the target accelerator card is then executed through the target accelerator card, achieving mixed deployment of accelerator cards for the target task.

[0046] It should be noted that by selecting the optimal target accelerator card for each split computation graph, high-performance execution of a single split computation graph is achieved, thereby achieving high-performance execution of the target task as a whole and realizing the hybrid deployment of the optimal accelerator card.

[0047] Based on any of the foregoing embodiments, screening the performance prediction data of the segmentation computation graph to obtain candidate performance prediction data of the segmentation computation graph includes: Performing weighted calculation on each type of performance prediction data of the segmentation calculation graph to obtain a comprehensive performance score; The comprehensive performance scores are sorted, and the performance prediction data corresponding to the comprehensive performance score with the highest value is selected as the candidate performance prediction data for the segmentation calculation graph.

[0048] Specifically, the performance prediction data of the computation graph can be segmented to calculate a comprehensive performance score. For example, the numerical values ​​corresponding to each performance prediction data item can be weighted and the resulting comprehensive performance scores can be ranked. The performance prediction data here includes at least one of computation time, energy consumption, memory usage, and communication delay. Each performance prediction data item can be assigned an initial weight.

[0049] Then, the performance prediction data corresponding to the comprehensive performance score with the highest numerical value can be selected as the candidate performance prediction data for splitting the computation graph.

[0050] The method provided by an embodiment of the present invention calculates a comprehensive score for the performance prediction data of the segmented calculation graphs executed by various types of accelerator cards, sorts the comprehensive performance scores, selects the performance prediction data corresponding to the highest comprehensive performance score as the candidate performance prediction data for the segmented calculation graph, and realizes the optimal accelerator card deployment for a single segmented calculation graph, thereby improving the execution efficiency of each segmented calculation graph.

[0051] Based on any of the above embodiments, step 120 includes: Obtaining the parallel scale of each operator in the overall computation graph; Based on the parallel scale, the overall computation graph is segmented to obtain the segmented computation graph.

[0052] Specifically, by analyzing the overall computational graph, we can understand the type, computational complexity, data dependencies, and other characteristics of each operator in the graph. We can then evaluate the potential for parallel execution for each operator, taking into account factors such as the operator's computational intensity, data parallelism, and model parallelism, to determine the operator's parallel scale. Parallel scale refers to the size or amount of computational tasks or data that can be processed in parallel when executing an operator.

[0053] Next, based on the evaluation results, the parallel scale of each operator is determined. This parallel scale can include determining the number of threads, number of blocks, batch size, etc. for the operator on the accelerator card.

[0054] Furthermore, a partitioning strategy for the overall computational graph can be developed based on the parallel scale of each operator to minimize communication overhead and maximize parallel efficiency. For example, the overall computational graph can be divided into multiple subgraphs, each executed by one or more accelerator cards. Finally, the partitioned computational graphs are verified to ensure their computational logic and dependencies are correct.

[0055] It should be noted that by dividing the overall computation graph according to the parallel scale of each operator in the overall computation graph, the execution splitting strategy is implemented to maximize the parallel efficiency, so as to improve the execution efficiency and resource utilization of the target task.

[0056] Based on any of the above embodiments, the computing characteristic indicators of the segmented computing graph include: at least one of input and output dimensions, input and output data size, number of operations, and parallel scale.

[0057] Here, the input-output dimension describes the spatial or structural characteristics of the split computation graph's input and output data. The number of operations refers to the number of computational operations required by an operator or subgraph during execution. This includes, but is not limited to, basic operations such as addition, multiplication, and comparison. Parallelism refers to the size or amount of computational tasks or data that can be processed in parallel when executing a split computation graph.

[0058] It should be noted that by comprehensively considering the computing characteristic indicators, the performance data such as the execution time and memory usage of each type of accelerator card when executing the split calculation graph can be predicted more accurately.

[0059] Based on any of the above embodiments, the target task includes a training task of a target model, and the target model is constructed based on a deep learning algorithm.

[0060] It should be noted that the target task includes the training task of the target model, and the target model is constructed based on a deep learning algorithm. For example, it can be an image recognition model constructed based on a deep learning algorithm, and the target task is the training task of the image recognition model. Therefore, an overall computational graph can be constructed for the image recognition model. By segmenting the overall computational graph, a segmented computational graph is obtained; the computational characteristic indicators of the segmented computational graph are input into the performance prediction model to obtain the performance prediction data of each type of accelerator card executing the segmented computational graph respectively output by the performance prediction model; based on the performance prediction data of the segmented computational graph, a mixed deployment of accelerator cards is performed for the target task, thereby improving the utilization rate of the accelerator cards in the mixed deployment environment, and thus achieving efficient execution of the target task.

[0061] Based on any of the above embodiments, Figure 3 This is the second flow chart of the hybrid deployment method of the accelerator card provided by the present invention, such as Figure 3 As shown, the method includes: First, the model computation graph is segmented. The model computation graph here refers to the target task computation graph, which is the model training task.

[0062] Next, the operators of the split computation graph are analyzed to obtain the computational characteristics of the split computation graph. These include operator input and output dimensions, input and output data size, number of multiplications and additions, and parallelization scale. Next, the computational characteristics of the split computation graph are fed into a performance prediction model to obtain performance prediction data for each type of accelerator card executing the split computation graph. Furthermore, the performance prediction data for the split computation graphs is sorted, and the accelerator card to be deployed for each split computation graph is determined based on the sorting results, thereby achieving mixed deployment of accelerator cards for the model computation graph.

[0063] Based on any of the above embodiments, Figure 4 This is a schematic diagram of the structure of the accelerator card hybrid deployment device provided by the present invention. Figure 4 As shown, the device includes: An acquisition unit 410 acquires an overall computation graph of a target task; A segmentation unit 420 segments the overall computation graph to obtain a segmented computation graph; The performance prediction unit 430 inputs the computing characteristic index of the segmented computing graph into a performance prediction model to obtain performance prediction data of various accelerator cards executing the segmented computing graph respectively output by the performance prediction model; A hybrid deployment unit 440 performs hybrid deployment of accelerator cards for the target task based on the performance prediction data of the segmented computation graph; The performance prediction model is constructed based on a neural network.

[0064] The device provided by the embodiment of the present invention obtains a segmented calculation graph by segmenting the overall calculation graph; inputs the computing characteristic indicators of the segmented calculation graph into a performance prediction model to obtain performance prediction data of various types of accelerator cards output by the performance prediction model for respectively executing the segmented calculation graph; based on the performance prediction data of the segmented calculation graph, the accelerator cards are mixedly deployed for the target task, thereby achieving accurate mixed deployment of the accelerator cards, and thereby improving the execution efficiency of the target task and reducing the power consumption required for execution in a mixed deployment environment.

[0065] Based on any of the above embodiments, the device further includes a training unit, which is specifically configured to: Obtain sample computational characteristic metrics of the test split computation graph, as well as an initial performance prediction model; Executing the test split calculation graph based on the various types of accelerator cards to collect actual performance data of the various types of accelerator cards; Inputting the sample computing characteristic index into the initial performance prediction model to obtain sample performance prediction data of each type of accelerator card executing the test split computing graph, output by the initial performance prediction model; Calculating a prediction loss based on the sample performance prediction data and the actual performance data; Based on the prediction loss, the model parameters of the initial performance prediction model are adjusted, and the performance prediction model is obtained through training.

[0066] Based on any of the above embodiments, the actual performance data includes at least one of: calculation time, energy consumption, memory occupancy rate and communication delay time.

[0067] Based on any of the above embodiments, the hybrid deployment unit is specifically configured to: Screening the performance prediction data of the segmentation computation graph to obtain candidate performance prediction data of the segmentation computation graph; The accelerator card corresponding to the candidate performance prediction data is used as the target accelerator card for the segmented computation graph; Based on the target accelerator card, a segmentation calculation graph corresponding to the target accelerator card is executed to implement hybrid deployment of accelerator cards for the target task.

[0068] Based on any of the foregoing embodiments, the hybrid deployment unit is further specifically configured to: Performing weighted calculation on each type of performance prediction data of the segmentation calculation graph to obtain a comprehensive performance score; The comprehensive performance scores are sorted, and the performance prediction data corresponding to the comprehensive performance score with the highest value is selected as the candidate performance prediction data for the segmentation calculation graph.

[0069] Based on any of the above embodiments, the segmentation unit is specifically configured to: Obtaining the parallel scale of each operator in the overall computation graph; Based on the parallel scale, the overall computation graph is segmented to obtain the segmented computation graph.

[0070] Based on any of the above embodiments, the computing characteristic indicators of the segmented computing graph include: at least one of input and output dimensions, input and output data size, number of operations, and parallel scale.

[0071] Based on any of the above embodiments, the target task includes a training task of a target model, and the target model is constructed based on a deep learning algorithm.

[0072] Figure 5 An example of a physical structure diagram of an electronic device is shown below. Figure 5 As shown, the electronic device may include: a processor 510, a communications interface 520, a memory 530, and a communications bus 540, wherein the processor 510, the communications interface 520, and the memory 530 communicate with each other via the communications bus 540. The processor 510 may call logic instructions in the memory 530 to execute a method for hybrid deployment of accelerator cards, the method comprising: obtaining an overall computation graph of a target task; segmenting the overall computation graph to obtain a segmented computation graph; inputting computational characteristic indicators of the segmented computation graph into a performance prediction model to obtain performance prediction data output by the performance prediction model for each type of accelerator card executing the segmented computation graph; and hybrid deployment of accelerator cards for the target task based on the performance prediction data of the segmented computation graph; the performance prediction model is constructed based on a neural network.

[0073] Furthermore, the logic instructions in the aforementioned memory 530 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the portion that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as a USB flash drive, a mobile hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0074] On the other hand, the present invention also provides a computer program product, which includes a computer program, which can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the accelerator card hybrid deployment method provided by the above methods, which includes: obtaining the overall calculation graph of the target task; segmenting the overall calculation graph to obtain a segmented calculation graph; inputting the computing characteristic indicators of the segmented calculation graph into a performance prediction model to obtain performance prediction data of each type of accelerator card output by the performance prediction model to execute the segmented calculation graph respectively; based on the performance prediction data of the segmented calculation graph, performing accelerator card hybrid deployment on the target task; the performance prediction model is constructed based on a neural network.

[0075] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the accelerator card hybrid deployment method provided by the above-mentioned methods, the method comprising: obtaining an overall calculation graph of the target task; segmenting the overall calculation graph to obtain a segmented calculation graph; inputting the computing characteristic indicators of the segmented calculation graph into a performance prediction model to obtain performance prediction data of each type of accelerator card output by the performance prediction model for executing the segmented calculation graph respectively; performing accelerator card hybrid deployment on the target task based on the performance prediction data of the segmented calculation graph; the performance prediction model is constructed based on a neural network.

[0076] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, i.e., they may be located in one location or distributed across multiple network units. Some or all of the modules may be selected based on actual needs to achieve the objectives of the present embodiment. Persons of ordinary skill in the art will be able to understand and implement the present invention without inventive effort.

[0077] Through the above description of the embodiments, those skilled in the art will clearly understand that each embodiment can be implemented using software plus a necessary general-purpose hardware platform, or of course, hardware. Based on this understanding, the essence of the above technical solution, or the portion that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a magnetic disk, or an optical disk, and includes a number of instructions for causing a computer device (such as a personal computer, server, or network device) to execute the methods described in each embodiment or certain portions of the embodiments.

[0078] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A hybrid deployment method for accelerator cards, characterized in that: include: Get the overall computational graph of the target task; Segmenting the overall computation graph to obtain a segmented computation graph; Inputting the computational characteristic index of the segmented computation graph into a performance prediction model to obtain performance prediction data of various accelerator cards executing the segmented computation graph respectively output by the performance prediction model; Performing hybrid deployment of accelerator cards for the target task based on the performance prediction data of the segmented computation graph; The performance prediction model is constructed based on a neural network.

2. The accelerator card hybrid deployment method according to claim 1, characterized in that: The training steps of the performance prediction model include: Obtain sample computational characteristic metrics of the test split computation graph, as well as an initial performance prediction model; Executing the test split calculation graph based on the various types of accelerator cards to collect actual performance data of the various types of accelerator cards; Inputting the sample computing characteristic index into the initial performance prediction model to obtain sample performance prediction data of each type of accelerator card executing the test split computing graph, output by the initial performance prediction model; Calculating a prediction loss based on the sample performance prediction data and the actual performance data; Based on the prediction loss, the model parameters of the initial performance prediction model are adjusted, and the performance prediction model is obtained through training.

3. The hybrid deployment method of accelerator cards according to claim 2, characterized in that: The actual performance data includes at least one of calculation time, energy consumption, memory occupancy rate and communication delay time.

4. The method for hybrid deployment of accelerator cards according to any one of claims 1 to 3, characterized in that: The hybrid deployment of the accelerator cards for the target task based on the performance prediction data of the segmented computation graph includes: Screening the performance prediction data of the segmentation computation graph to obtain candidate performance prediction data of the segmentation computation graph; The accelerator card corresponding to the candidate performance prediction data is used as the target accelerator card for the segmented computation graph; Based on the target accelerator card, a segmentation calculation graph corresponding to the target accelerator card is executed to implement hybrid deployment of accelerator cards for the target task.

5. The method for hybrid deployment of accelerator cards according to claim 4, characterized in that: The step of screening the performance prediction data of the segmentation computation graph to obtain candidate performance prediction data of the segmentation computation graph comprises: Performing weighted calculation on each type of performance prediction data of the segmentation calculation graph to obtain a comprehensive performance score; The comprehensive performance scores are sorted, and the performance prediction data corresponding to the comprehensive performance score with the highest value is selected as the candidate performance prediction data for the segmentation calculation graph.

6. The method for hybrid deployment of accelerator cards according to any one of claims 1 to 3, characterized in that: The step of segmenting the overall computation graph to obtain a segmented computation graph includes: Obtaining the parallel scale of each operator in the overall computation graph; Based on the parallel scale, the overall computation graph is segmented to obtain the segmented computation graph.

7. The method for hybrid deployment of accelerator cards according to any one of claims 1 to 3, characterized in that: The computational characteristic indicators of the segmented computation graph include at least one of input and output dimensions, input and output data size, number of operations, and parallel scale.

8. The method for hybrid deployment of accelerator cards according to any one of claims 1 to 3, characterized in that: The target task includes a training task of a target model, and the target model is constructed based on a deep learning algorithm.

9. A hybrid deployment device for accelerator cards, characterized in that: include: Get the unit to get the overall computational graph of the target task; A segmentation unit, which segments the overall computation graph to obtain a segmented computation graph; a performance prediction unit, inputting the computational characteristic index of the segmented computation graph into a performance prediction model, and obtaining performance prediction data output by the performance prediction model for each type of accelerator card executing the segmented computation graph; A hybrid deployment unit, which performs hybrid deployment of accelerator cards for the target task based on the performance prediction data of the segmented computation graph; The performance prediction model is constructed based on a neural network.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the accelerator card hybrid deployment method according to any one of claims 1 to 8 is implemented.

Images