Bandwidth equivalent calculation method, device, equipment, medium and product for data transport

Abstract

The present invention discloses a method, device, equipment, medium and product for calculating bandwidth equivalence during data transfer. The method includes: densely measuring data of multiple data transfer types and multiple tensor shapes under a first data type, performing first measured bandwidth data when data transfer is performed between caches at various levels of a chip, and constructing a target bandwidth model for the first data type; sparsely measuring data of multiple data transfer types and multiple tensor shapes under a second data type, and performing second measured bandwidth data when data transfer is performed between caches at various levels of the chip; based on the first and second measured bandwidth data, constructing a bandwidth equivalence calculation model between the first data type and the second data type, obtaining data of the target data transfer type and target tensor shape under the second data type, and performing predicted bandwidth data when data transfer is performed between two target caches of the chip. The technical solution of the embodiment of the present invention can effectively reduce the various resources consumed in the bandwidth data testing process.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence (AI) technology, and in particular to a method, device, equipment, medium, and product for calculating bandwidth equivalence during data transfer. Background Art

[0002] In high-performance computing scenarios for AI chips, operator performance is a crucial component of chip performance evaluation. Operator performance largely determines the chip's performance ceiling. However, many factors influence operator performance, with bandwidth being a crucial factor. However, due to the complexity of DMA (Direct Memory Access) hardware, calculating the DMA bandwidth for data transmission with varying tensor shapes and precision often requires complex methods, making operator bandwidth evaluation difficult.

[0003] In related technologies, the DMA bandwidth data of various tensor shapes and data precisions can be measured on actual AI chips to train the machine learning model, and the powerful fitting capabilities of the machine learning model can be used to fit the characteristics related to DMA hardware.

[0004] However, during the implementation of this invention, the inventors discovered that accurate bandwidth prediction for deep learning models requires a large amount of bandwidth data with varying tensor shapes and precision measured on the AI ​​chip. The bandwidth data for a single data type that meets the model's precision requirements often requires millions of units. Testing bandwidth data for all tensor shapes and data of all data types would consume significant time, labor costs, and AI chip computing resources. Summary of the Invention

[0005] Embodiments of the present invention provide a method, apparatus, device, medium, and product for calculating bandwidth equivalence during data transport, so as to effectively reduce various resources consumed during bandwidth data testing.

[0006] According to one aspect of an embodiment of the present invention, a method for calculating bandwidth equivalence during data transport is provided, comprising:

[0007] Running a bandwidth test program in the target AI chip to intensively measure data of multiple data transfer types and multiple tensor shapes under a first data type, and first measured bandwidth data when performing data transfer between cache levels of the target AI chip;

[0008] Constructing a target bandwidth model for the first data type according to the first measured bandwidth data;

[0009] Running a bandwidth test program in the target AI chip to sparsely measure data of multiple data transfer types and multiple tensor shapes under the second data type, and generating second measured bandwidth data when performing data transfer between cache levels of the target AI chip;

[0010] Constructing a bandwidth equivalence calculation model between the first data type and the second data type according to the first measured bandwidth data and the second measured bandwidth data;

[0011] According to the target bandwidth model and the bandwidth equivalent calculation model, data on the target data transfer type and the target tensor shape under the second data type are obtained, and the predicted bandwidth data when performing data transfer between the two target caches of the target AI chip is obtained.

[0012] According to another aspect of an embodiment of the present invention, a device for calculating bandwidth equivalence during data transport is provided, comprising:

[0013] A first bandwidth data acquisition module is configured to run a bandwidth test program in the target AI chip to intensively measure data of various data transfer types and multiple tensor shapes under a first data type, and obtain first measured bandwidth data when performing data transfer between cache levels of the target AI chip;

[0014] a target bandwidth model building module, configured to build a target bandwidth model for the first data type based on the first measured bandwidth data;

[0015] A second bandwidth data acquisition module is configured to run a bandwidth test program in the target AI chip, sparsely measure data of multiple data transfer types and multiple tensor shapes under the second data type, and generate second measured bandwidth data when performing data transfer between cache levels of the target AI chip;

[0016] An equivalent model construction module, configured to construct a bandwidth equivalent calculation model between the first data type and the second data type based on the first measured bandwidth data and the second measured bandwidth data;

[0017] The predicted bandwidth data acquisition module is used to obtain data on the target data transfer type and target tensor shape under the second data type based on the target bandwidth model and the bandwidth equivalent calculation model, and the predicted bandwidth data when performing data transfer between the two target caches of the target AI chip.

[0018] According to another aspect of an embodiment of the present invention, an electronic device is provided, comprising:

[0019] at least one processor; and

[0020] a memory communicatively connected to the at least one processor; wherein,

[0021] The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can perform the bandwidth equivalent calculation method for data transfer according to any embodiment of the present invention.

[0022] According to another aspect of an embodiment of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the bandwidth equivalent calculation method for data transfer according to any embodiment of the present invention when executed.

[0023] According to another aspect of an embodiment of the present invention, a computer program product is provided, including a computer program. When the computer program is executed by a processor, the steps of the bandwidth equivalent calculation method during data transfer as described in any embodiment of the present invention are implemented.

[0024] The technical solution of the embodiment of the present invention is to run a bandwidth test program in the target AI chip to densely measure multiple data transfer types and multiple tensor-shaped data under a first data type, and perform first measured bandwidth data when data transfer is performed between caches at all levels of the target AI chip; based on the first measured bandwidth data, a target bandwidth model for the first data type is constructed; run a bandwidth test program in the target AI chip to sparsely measure multiple data transfer types and multiple tensor-shaped data under a second data type, and perform second measured bandwidth data when data transfer is performed between caches at all levels of the target AI chip; based on the first measured bandwidth data and the second measured bandwidth data, a bandwidth equivalent calculation model between the first data type and the second data type is constructed. type; according to the target bandwidth model and the bandwidth equivalent calculation model, the data of the target data transfer type and the target tensor shape under the second data type are obtained, and the implementation method of the predicted bandwidth data when performing data transfer between the two target caches of the target AI chip only requires intensive testing of the bandwidth data of various tensor shapes under one data type when performing data transfer, and then combining the small amount of bandwidth data of various tensor shapes under other data types obtained by sparse testing when performing data transfer. Based on the deep learning model trained based on intensive bandwidth data, combined with the bandwidth ratio relationship between different data accuracies, the corresponding predicted bandwidth data under the corresponding data accuracy can be accurately obtained, saving a lot of manpower, material resources and chip computing power of the AI ​​chip.

[0025] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present invention, nor is it intended to limit the scope of the present invention. Other features of the present invention will become readily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0027] Figure 1 This is a flowchart of a bandwidth equivalent calculation method for data transfer provided by an embodiment of the present invention;

[0028] Figure 2 This is a flowchart of another method for calculating bandwidth equivalence during data transfer provided by an embodiment of the present invention;

[0029] Figure 3 It is a data table applicable to the embodiment of the present invention for displaying the bandwidth ratio relationship between two different data types;

[0030] Figure 4 This is a flowchart of another method for calculating bandwidth equivalence during data transfer provided by an embodiment of the present invention;

[0031] Figure 5 2 is a schematic diagram of a device for calculating bandwidth equivalence during data transfer according to an embodiment of the present invention;

[0032] Figure 6 A schematic diagram of the structure of an electronic device that implements the bandwidth equivalent calculation method for data transfer according to an embodiment of the present invention. DETAILED DESCRIPTION

[0033] In order to enable those skilled in the art to better understand the solutions of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the embodiments described are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of the present invention.

[0034] It should be noted that the terms "first", "second", etc. in the description and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that the numbers used in this way can be interchanged where appropriate so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0035] Figure 1 A flowchart of a method for calculating bandwidth equivalence during data transfer provided by an embodiment of the present invention is applicable to situations where bandwidth data of a second data type is accurately predicted based on a bandwidth model of the first data type and a bandwidth equivalence calculation model between the first data type and the second data type. The method can be performed by a device for calculating bandwidth equivalence during data transfer, which can be implemented in the form of hardware and / or software and can generally be configured in various electronic devices with data processing capabilities, such as terminals or servers.

[0036] Correspondingly, such as Figure 1 As shown, the method includes:

[0037] S110. Run a bandwidth test program in the target AI chip to intensively measure multiple data transfer types and multiple tensor-shaped data under the first data type, and obtain first measured bandwidth data when performing data transfer between cache levels of the target AI chip.

[0038] Among them, the first data type can be understood as a specific data type of the data (typically, tensor data) carried on the target AI chip. The data type can be understood as the numerical precision of the tensor data, specifically referring to the numerical representation format of each element in the tensor data, for example, INT8, FP16 or FP32. The data handling type can be understood as the characteristics of the data reorganization method when the data is moved between different levels of cache. The data handling type may include: in-situ handling (Copy), transpose handling (Transpose), slice handling (Slice) and compression handling (Compress). The tensor shape can be understood as the dimensional structure of the data tensor. The tensor shape defines the number of elements of the data in each dimension. For example, for a two-dimensional tensor, its tensor shape can be [1024,1024].

[0039] Generally speaking, the target AI chip often has a multi-level cache architecture. For example, when the target AI chip has a three-level cache architecture, the three-level cache architecture is, in order from near to far from the computing core in the target AI chip, the first-level cache (also called L1 or L1 cache), the second-level cache (also called L2 or L2 cache) and the third-level cache (also called L3 or L3 cache).

[0040] The first measured bandwidth data when transferring data between cache levels on the target AI chip can be understood as the specific bandwidth data measured when using DMA hardware in the target AI chip to transfer specific data from cache A to cache B. This first bandwidth data refers to the amount of data transferred per unit time.

[0041] In this optional implementation, a corresponding bandwidth test program can be written based on the minimum granularity DMA operation of the target AI chip. By running the bandwidth test program on the target AI chip, data under the first data transfer type, multiple data types and multiple tensor shape combinations can be measured, and the first measured bandwidth data when performing data transfer between various levels of cache of the target AI chip can be obtained.

[0042] It's understandable that once the tensor shape, data type, and data transfer type of the data being transferred are determined, the bandwidth of transferring that data between two predetermined cache levels (for example, from L3 to L2) can be measured. By constructing a bandwidth test program based on this information and running it on the target AI chip, the corresponding bandwidth data, also known as the measured bandwidth data, can be measured and obtained.

[0043] Among them, intensive testing can be understood as selecting as many tensor shapes of data as possible that can complete data transfer between different levels of cache, combining the above-mentioned various tensor shapes with various optional data transfer types as much as possible, obtaining as many forms of data to be transferred as possible, and actually testing the first measured bandwidth data when various forms of data to be transferred are actually transferred between caches at all levels.

[0044] S120: Construct a target bandwidth model for the first data type according to the first measured bandwidth data.

[0045] In this embodiment, intensive testing is required to obtain the first measured bandwidth data because the first measured bandwidth data is used as training samples to train a target bandwidth model for the first data type. A greater amount of bandwidth data contained in the first measured bandwidth data indicates more data handling involved in the actual test, and the resulting trained target bandwidth model will have a higher prediction accuracy.

[0046] The input of the target bandwidth model is the set data transfer type, the set tensor shape, and the two set caches when performing data transfer. The output is the data of the data transfer type and the tensor shape under the first data type predicted by the target bandwidth model, and the predicted bandwidth data when performing data transfer between the two set caches of the target AI chip.

[0047] S130: Run a bandwidth test program in the target AI chip to sparsely measure multiple data transfer types and multiple tensor-shaped data under the second data type, and obtain second measured bandwidth data when performing data transfer between cache levels of the target AI chip.

[0048] It is understandable that if intensive testing such as S110 is performed for each data type, for various data transfer types and data of various tensor shapes, a corresponding target bandwidth model can be trained for each data type. However, it is foreseeable that such intensive testing of data of all data types would require frequent or prolonged running of bandwidth test programs suitable for various test requirements on the target AI chip, generating a large amount (on the order of millions) of measured bandwidth data. This operation would consume a lot of time, manpower, and machine resources.

[0049] Based on this, the inventors creatively proposed a new solution, namely: after intensively testing various measured bandwidth data under a data type (that is, the first data type), and sparsely testing various measured bandwidth data under other data types, based on the above-mentioned measured bandwidth data, learn the bandwidth ratio relationship between other data types and the first data type. Furthermore, when it is necessary to predict bandwidth data of a set form under other data types (for example, the second data type), the target bandwidth model corresponding to the first data type can be used to obtain the predicted bandwidth data under the first data type, and then the predicted bandwidth data under the second data type can be directly obtained by combining the bandwidth ratio relationship between the second data type and the first data type. Through the above setting, the amount of bandwidth data that needs to be measured on the target AI chip can be greatly reduced.

[0050] Among them, when performing sparse testing, this embodiment mainly considers reducing the types of tensor shapes that need to be tested to achieve the purpose of reducing the total amount of measured bandwidth data. However, it is still necessary to test each data transfer type and between any two caches to obtain a small amount of typical second measured bandwidth data.

[0051] In a specific example, assume that the cache capacity of the L2 cache on the target AI chip is 64MB, which can be split into two data dimensions of 8192*8192 at most. Assume that there are two parallel physical computing modules on the target AI chip, each of which contains 12 parallel computing units. Furthermore, assume that the data type to be moved from the L3 cache to the L2 cache is FP16. If a complete test is to be implemented, the number of tensor shapes that need to be constructed is: (8192 / 2)*(8192 / 2)*2*12=402653184, reaching the million level.

[0052] Of course, considering the various software and hardware limitations of the target AI chip, for example, each data dimension needs to be aligned to 16 bits or 32 bits, all the tensor shapes required for intensive testing can be obtained by performing certain data cropping or trade-offs on all the above tensors.

[0053] Furthermore, when performing sparse testing, consider selecting some special or frequently used tensor shapes from all the tensor shapes used in the above-mentioned dense testing. At this time, the data level of the second measured bandwidth data required to be obtained is much smaller than the first measured bandwidth data. It can be understood that the more types of data that can be used in the target AI chip, the more manpower, material resources and chip computing power of the AI ​​chip can be reduced by the technical solution of the embodiment of the present invention.

[0054] S140: Construct a bandwidth equivalence calculation model between the first data type and the second data type based on the first measured bandwidth data and the second measured bandwidth data.

[0055] It can be understood that for each specific second measured bandwidth data, paired first measured data with the same tensor shape, data type, and two caches for performing data transfer can be obtained from all first measured bandwidth data.

[0056] Furthermore, by using each second measured bandwidth data item and its paired first measured data item as a set of training samples, a bandwidth equivalence calculation model can be trained on the specified machine learning model. Based on this bandwidth equivalence calculation model, the numerical proportional relationship between the bandwidth data item of the first data type and the bandwidth data item of the second data type can be mined when the tensors have the same shape and data type, and the two caches performing data transfer are the same.

[0057] S150. According to the target bandwidth model and the bandwidth equivalent calculation model, obtain data on the target data transfer type and the target tensor shape under the second data type, and predict bandwidth data when performing data transfer between the two target caches of the target AI chip.

[0058] In this embodiment, even if the corresponding bandwidth data of many tensor shapes under the second data type is not measured, as long as the bandwidth data of the tensor shape under the first data type can be obtained by using the target bandwidth model and then combining it with the bandwidth equivalent calculation model, a relatively accurate bandwidth prediction result for the tensor shape under the second data type can be obtained.

[0059] The technical solution of the embodiment of the present invention is to run a bandwidth test program in the target AI chip to densely measure multiple data transfer types and multiple tensor-shaped data under a first data type, and perform first measured bandwidth data when data transfer is performed between caches at all levels of the target AI chip; based on the first measured bandwidth data, a target bandwidth model for the first data type is constructed; run a bandwidth test program in the target AI chip to sparsely measure multiple data transfer types and multiple tensor-shaped data under a second data type, and perform second measured bandwidth data when data transfer is performed between caches at all levels of the target AI chip; based on the first measured bandwidth data and the second measured bandwidth data, a bandwidth equivalent calculation model between the first data type and the second data type is constructed. type; according to the target bandwidth model and the bandwidth equivalent calculation model, the data of the target data transfer type and the target tensor shape under the second data type are obtained, and the implementation method of the predicted bandwidth data when performing data transfer between the two target caches of the target AI chip only requires intensive testing of the bandwidth data of various tensor shapes under one data type when performing data transfer, and then combining the small amount of bandwidth data of various tensor shapes under other data types obtained by sparse testing when performing data transfer. Based on the deep learning model trained based on intensive bandwidth data, combined with the bandwidth ratio relationship between different data accuracies, the corresponding predicted bandwidth data under the corresponding data accuracy can be accurately obtained, saving a lot of manpower, material resources and chip computing power of the AI ​​chip.

[0060] Figure 2 This is a flowchart of another bandwidth equivalent calculation method for data transfer provided by an embodiment of the present invention. This embodiment is optimized based on the above embodiments. In this embodiment, the operations of constructing a target bandwidth model and a bandwidth equivalent calculation model are concretized.

[0061] Correspondingly, such as Figure 2 As shown, the method may include:

[0062] S210: Run a bandwidth test program in the target AI chip to intensively measure data of multiple data transfer types and multiple tensor shapes under the first data type, and obtain first measured bandwidth data when performing data transfer between cache levels of the target AI chip.

[0063] In this embodiment, a general bandwidth test program template can be pre-written, which takes a data transfer type, a data type, a tensor shape, and two predetermined buffers as input variables. Furthermore, when the general bandwidth test program template is reconstructed using the predetermined data transfer type, the first data type, the predetermined tensor shape, and the two predetermined buffers, the reconstructed bandwidth test program, when run on the target AI chip, can obtain data for multiple data transfer types and multiple tensor shapes under the first data type, as well as first measured bandwidth data when performing data transfer between cache levels of the target AI chip.

[0064] Optionally, when constructing a specific bandwidth test program, data of multiple different tensor shapes of the same data transfer type and the same data type can be obtained simultaneously during the execution of a bandwidth test program, and the actual bandwidth data when performing data transfer between the same two caches of the target AI chip can be obtained to improve test efficiency.

[0065] S220: Obtain, from the first measured bandwidth data, data of the same data transfer type under the first data type, measured bandwidth data when data transfer is performed between the same two caches, and construct a plurality of measured bandwidth data groups.

[0066] Specifically, each measured bandwidth data group includes data of multiple different tensor shapes of the same data transfer type under the first data type, and the first measured bandwidth data when performing data transfer between the same two caches of the target AI chip.

[0067] Accordingly, each measured bandwidth data group may be named based on the set data transport type, the first data type, and the two buffers.

[0068] For example, if measured bandwidth data group 1 is named "L3->L2_fp16_slice," this indicates that measured bandwidth data group 1 stores the first measured bandwidth data for data with various tensor shapes, whose data transfer type is slice and whose first data type is fp16, when transferred from the L3 cache (L3) to the L2 cache (L2) on the target AI chip. Alternatively, the tensor shape can be understood as the size of the data block transferred from the L3 cache to the L2 cache.

[0069] After obtaining the aforementioned measured bandwidth data groups, different bandwidth sub-models can be trained based on each measured bandwidth data group. Accordingly, the data in each measured bandwidth data group can be preprocessed, for example, by removing unreasonable data or distributing and shuffling the data. Afterward, each item of first measured bandwidth data in each measured bandwidth data group can be divided into a training set and a test set.

[0070] S230: Add hardware attribute information of the matching direct access storage hardware in the target AI chip to each first measured bandwidth data in each measured data group.

[0071] In this embodiment, to ensure a higher degree of correlation and compatibility between the target bandwidth model ultimately obtained through training and the data handling hardware of the actual target AI chip, and thus to increase the accuracy of the resulting bandwidth data, hardware attribute information of the direct access storage hardware matching the target AI chip is added to the training and test samples in each measured bandwidth data group (i.e., each piece of first measured bandwidth data).

[0072] In a specific example, the hardware attribute information of the DMA hardware may include data such as bus bandwidth, bus frequency, cache capacity, and cache level of the DMA hardware, which is not limited in this embodiment.

[0073] S240: Use each measured bandwidth data group to train a preset machine learning model to obtain multiple bandwidth sub-models, and organize each bandwidth sub-model to obtain a target bandwidth model for the first data type.

[0074] The bandwidth sub-model corresponds to two buffers for setting the data transfer type and executing the data transfer.

[0075] In this embodiment, the inventors have found through multiple experiments that when the Extreme Gradient Boosting (XGBoost) model or the Multi-Layer Perceptron (MLP) model is selected for model training to obtain the target bandwidth model, the complex nonlinear relationship between input and output can be better fitted. Furthermore, the above-mentioned preset machine learning model can choose to use the XGBoost model or the MLP model.

[0076] As mentioned above, each measured bandwidth data group can be used to train the set XGBoost model or MLP model respectively to obtain a bandwidth sub-model corresponding to each measured bandwidth data group.

[0077] Furthermore, bandwidth sub-models can be constructed that simultaneously correspond to the specified data transfer type, the first data type, and the two caches. For example, a bandwidth sub-model named L3->L2_fp16_slice_model can be used to predict the bandwidth of data with a specific tensor shape, whose data transfer type is slice and the first data type is fp16, when moving from the third-level cache (L3) to the second-level cache (L2) on the target AI chip.

[0078] That is, after selecting the above L3->L2_fp16_slice_model from the target bandwidth model based on a data transfer behavior of the bandwidth data to be predicted, we can input only the tensor shape in the data transfer behavior into the bandwidth sub-model to obtain the corresponding bandwidth data (that is, the prediction result of the actual bandwidth data).

[0079] S250. Run a bandwidth test program in the target AI chip, sparsely measure multiple data transfer types and multiple tensor-shaped data under the second data type, and perform second measured bandwidth data when performing data transfer between cache levels of the target AI chip.

[0080] S260. Obtain, based on the first measured bandwidth data and the second measured bandwidth data, a bandwidth ratio relationship between a first data type and a second data type when data of a set data transfer type and a set tensor shape is transferred between two set caches in the target AI chip.

[0081] Among them, since the second measured bandwidth data is a sparse test result, and the first measured bandwidth data is a dense test result, then, a second measured bandwidth data can be obtained in sequence as the current second measured bandwidth data, and then the target data handling type, target tensor shape, and two target caches corresponding to the current second measured bandwidth data are obtained when performing data handling. Then, the current paired first measured bandwidth data corresponding to the target data handling type, target tensor shape, and the above-mentioned two target caches can be obtained in each first measured bandwidth data; finally, the bandwidth ratio relationship between the above-mentioned current second measured bandwidth data and the current paired first measured bandwidth data can be obtained. For example, if the current paired first measured bandwidth data is 3MB / s, and the current second measured bandwidth data is 2.5MB / s, then for this set of paired data, the bandwidth ratio relationship between the first data type and the second data type under the target data handling type, target tensor shape, and the above-mentioned two target caches is 1.2. It is understandable that if multiple sets of paired data are obtained for the target data handling type, the target tensor shape, and the two target caches, multiple bandwidth ratio relationships for the target data handling type, the target tensor shape, and the two target caches can be calculated based on the multiple sets of paired data.

[0082] In a specific example, Figure 3 A data table for displaying the bandwidth ratio relationship between two different data types applicable to an embodiment of the present invention is shown in FIG. Figure 3 As shown, the first data type is FP16 (fp16_bw), the second data type is INT8 (int8_bw), block_n is the number of physical computing modules used in the target AI chip, and thread is the number of parallel computing units used by each physical computing module in the target AI chip. The two dst_dim values ​​represent the two dimensional shapes of the data tensor, out_size is the total number of elements contained in the data tensor, and ratio represents the bandwidth ratio between the first and second data types.

[0083] S270: Construct an associated data set according to the bandwidth ratio relationship, and construct a bandwidth equivalence calculation model between the first data type and the second data type according to the associated data set.

[0084] In this embodiment, the multiple paired data sets obtained after the operation in step S260 can be used as associated data to form associated data sets. Furthermore, the associated data sets can be used as training samples to train a bandwidth equivalence calculation model between the first data type and the second data type.

[0085] In an optional implementation of this embodiment, constructing a bandwidth equivalence calculation model between the first data type and the second data type based on the associated data set may include:

[0086] After adding hardware attribute information of the matching direct access storage hardware in the target AI chip to each associated data in the associated data set, the set machine learning model is trained using the key data set to obtain a bandwidth equivalence calculation model between the first data type and the second data type.

[0087] The bandwidth equivalent calculation model is used to output a predicted bandwidth ratio between a first data type and a second data type according to the input settings of two caches, a data transfer type, and a tensor shape.

[0088] In a specific example, the hardware attribute information of the DMA hardware may also include data such as the bus bandwidth, bus frequency, cache capacity, and cache level of the DMA hardware, which is not limited in this embodiment. In addition, the machine learning model used to train the wide equivalent computing model may also be an Extreme Gradient Boosting (XGBoost) model or a Multi-Layer Perceptron (MLP) model, which is not limited in this embodiment.

[0089] S280. According to the target bandwidth model and the bandwidth equivalent calculation model, obtain data on the target data transfer type and the target tensor shape under the second data type, and predict bandwidth data when performing data transfer between the two target caches of the target AI chip.

[0090] The technical solution of the embodiment of the present invention, through in-depth analysis of the characteristics of the DMA hardware in the target AI chip, finds that when data transfer is performed between caches of various levels for data of different data types under the same data transfer type, the actual bandwidth has a certain correlation. Based on this, it is creatively proposed to actually test a large amount of bandwidth data of different tensor shapes under the set data type, and then test a small amount of bandwidth data of special tensor shapes under other data types. After fitting the proportional relationship between the bandwidth of the intensively tested data type and one or more other data types with the same tensor shape based on the deep learning model, the bandwidth of these sparsely tested data types can be efficiently and accurately estimated to minimize the use of time, manpower and equipment resources.

[0091] Figure 4 This is a flowchart of another method for calculating bandwidth equivalence during data transfer provided by an embodiment of the present invention. This embodiment is optimized based on the above embodiments. In this embodiment, the operation of "obtaining data on the target data transfer type and target tensor shape under the second data type based on the target bandwidth model and the bandwidth equivalent calculation model, and predicting bandwidth data when performing data transfer between the two target caches of the target AI chip" is concretized.

[0092] Correspondingly, such as Figure 4 As shown, the method may include:

[0093] S410: Run a bandwidth test program in the target AI chip to intensively measure multiple data transfer types and multiple tensor-shaped data under a first data type, and obtain first measured bandwidth data when performing data transfer between cache levels of the target AI chip.

[0094] S420: Construct a target bandwidth model for the first data type according to the first measured bandwidth data.

[0095] S430: Run a bandwidth test program in the target AI chip to sparsely measure multiple data transfer types and multiple tensor-shaped data under the second data type, and obtain second measured bandwidth data when performing data transfer between cache levels of the target AI chip.

[0096] S440: Construct a bandwidth equivalence calculation model between the first data type and the second data type according to the first measured bandwidth data and the second measured bandwidth data.

[0097] S450: Identify a target bandwidth sub-model in the target bandwidth model according to the target data transfer type and the two target caches.

[0098] In this embodiment, a specific implementation method for obtaining data on the target data transfer type and target tensor shape under the second data type and predicting bandwidth data when performing data transfer between two target caches of the target AI chip is specifically described.

[0099] First, it is necessary to obtain the target bandwidth sub-model corresponding to the target data transfer type and the two target buffers from all bandwidth sub-models included in the target bandwidth model.

[0100] S460. Input the target tensor shape into the target bandwidth sub-model to obtain data on the target data transfer type and target tensor shape under the first data type, and reference bandwidth data when performing data transfer between the two target caches of the target AI chip.

[0101] In an optional implementation of this embodiment, inputting the target tensor shape into the target bandwidth sub-model may specifically include:

[0102] The target tensor shape and the hardware attribute information of the matching direct access storage hardware in the target AI chip are input into the target bandwidth sub-model.

[0103] S470: Input the target data transport type, two target caches, and target tensor shape into a bandwidth equivalent calculation model, and output a target predicted bandwidth ratio relationship between the first data type and the second data type.

[0104] In an optional implementation of this embodiment, inputting the target data handling type, two target buffers, and target tensor shape into the bandwidth equivalent calculation model may specifically include:

[0105] The target data handling type, two target caches, target tensor shape, and hardware attribute information of the matching direct access storage hardware in the target AI chip are input into the bandwidth equivalent calculation model.

[0106] S480. Based on the proportional relationship between the reference bandwidth data and the target predicted bandwidth, obtain data on the target data transfer type and target tensor shape under the second data type, and obtain predicted bandwidth data when performing data transfer between the two target caches of the target AI chip.

[0107] Specifically, the quotient obtained by dividing the reference bandwidth data by the target predicted bandwidth ratio can be calculated as the target data transfer type and target tensor shape data under the second data type, and the predicted bandwidth data when performing data transfer between the two target caches of the target AI chip.

[0108] It needs to be emphasized again that in the operator performance evaluation, the performance of DMA hardware is a very important point, but the DMA performance of data with different tensor shapes and different data types when performing data transfer between different caches is different. If you need to obtain the DMA performance under a certain data type, you need to use dense tensor shapes as much as possible for this data type to perform bandwidth testing, so that the total number of bandwidth test cases is the number of tensor shapes * the number of data types * the number of cache relationships. Since the number of tensor shapes is generally in the millions, the number of overall bandwidth test cases obtained by multiplication will be very large. If all tests are completed, it will be extremely wasteful of manpower and environmental resources. However, by using the technical solutions of the various embodiments of the present invention, only the bandwidth data of the dense tensor shape corresponding to one of the data types is tested, and then a small amount of bandwidth data of the sparse tensor shape of other data types is tested. Based on the deep learning model trained with dense bandwidth data, combined with the bandwidth ratio relationship between different data types, the bandwidth data of the data type that has not been intensively tested can be accurately predicted, thereby saving a lot of manpower and material resources.

[0109] Figure 5 This is a schematic diagram of a data transfer bandwidth equivalent calculation device provided by an embodiment of the present invention. Figure 5 As shown, the apparatus includes: a first bandwidth data acquisition module 510, a target bandwidth model construction module 520, a second bandwidth data acquisition module 530, an equivalent model construction module 540, and a predicted bandwidth data acquisition module 550, wherein:

[0110] The first bandwidth data acquisition module 510 is used to run a bandwidth test program in the target AI chip, intensively measure multiple data transfer types and multiple tensor shapes under the first data type, and obtain first measured bandwidth data when performing data transfer between various cache levels of the target AI chip.

[0111] The target bandwidth model building module 520 is configured to build a target bandwidth model for the first data type according to the first measured bandwidth data.

[0112] The second bandwidth data acquisition module 530 is used to run a bandwidth test program in the target AI chip, sparsely measure multiple data transfer types and multiple tensor shapes under the second data type, and obtain the second measured bandwidth data when performing data transfer between various cache levels of the target AI chip.

[0113] The equivalent model building module 540 is used to build a bandwidth equivalent calculation model between the first data type and the second data type based on the first measured bandwidth data and the second measured bandwidth data.

[0114] The predicted bandwidth data acquisition module 550 is used to obtain data on the target data transfer type and target tensor shape under the second data type based on the target bandwidth model and the bandwidth equivalent calculation model, and to obtain the predicted bandwidth data when performing data transfer between the two target caches of the target AI chip.

[0115] The technical solution of the embodiment of the present invention is to run a bandwidth test program in the target AI chip to densely measure multiple data transfer types and multiple tensor-shaped data under a first data type, and perform first measured bandwidth data when data transfer is performed between caches at all levels of the target AI chip; based on the first measured bandwidth data, a target bandwidth model for the first data type is constructed; run a bandwidth test program in the target AI chip to sparsely measure multiple data transfer types and multiple tensor-shaped data under a second data type, and perform second measured bandwidth data when data transfer is performed between caches at all levels of the target AI chip; based on the first measured bandwidth data and the second measured bandwidth data, a bandwidth equivalent calculation model between the first data type and the second data type is constructed. type; according to the target bandwidth model and the bandwidth equivalent calculation model, the data of the target data transfer type and the target tensor shape under the second data type are obtained, and the implementation method of the predicted bandwidth data when performing data transfer between the two target caches of the target AI chip only requires intensive testing of the bandwidth data of various tensor shapes under one data type when performing data transfer, and then combining the small amount of bandwidth data of various tensor shapes under other data types obtained by sparse testing when performing data transfer. Based on the deep learning model trained based on intensive bandwidth data, combined with the bandwidth ratio relationship between different data accuracies, the corresponding predicted bandwidth data under the corresponding data accuracy can be accurately obtained, which saves a lot of manpower, material resources and chip computing power of the AI ​​chip.

[0116] Based on the above embodiments, the target bandwidth model construction module 520 can be specifically used to:

[0117] In the first measured bandwidth data, respectively obtain data of the same data transfer type under the first data type, and the measured bandwidth data when data transfer is performed between the same two caches, and construct a plurality of measured bandwidth data groups;

[0118] Adding hardware attribute information of the matching direct access storage hardware in the target AI chip to each first measured bandwidth data in each measured data group;

[0119] Using each measured bandwidth data group to train a preset machine learning model to obtain multiple bandwidth sub-models, and organizing each bandwidth sub-model to obtain a target bandwidth model for the first data type;

[0120] The bandwidth sub-model corresponds to two buffers for setting the data transfer type and executing the data transfer.

[0121] Based on the above embodiments, the equivalent model building module 540 may specifically include:

[0122] a ratio relationship acquisition unit, configured to acquire, based on the first measured bandwidth data and the second measured bandwidth data, a bandwidth ratio relationship between a first data type and a second data type when data of a set data transfer type and a set tensor shape is transferred between two set caches in the target AI chip;

[0123] A model building unit, configured to build an associated data set according to the bandwidth ratio relationship, and build a bandwidth equivalence calculation model between the first data type and the second data type according to the associated data set;

[0124] The bandwidth equivalent calculation model is used to output a predicted bandwidth ratio between a first data type and a second data type according to the input settings of two caches, a data transfer type, and a tensor shape.

[0125] Based on the above embodiments, the model building unit can be specifically used to:

[0126] After adding hardware attribute information of the matching direct access storage hardware in the target AI chip to each associated data in the associated data set, the set machine learning model is trained using the key data set to obtain a bandwidth equivalence calculation model between the first data type and the second data type.

[0127] Based on the above embodiments, the predicted bandwidth data acquisition module 550 may specifically include:

[0128] a sub-model identification unit, configured to identify a target bandwidth sub-model in the target bandwidth model according to a target data transport type and two target buffers;

[0129] a reference bandwidth data acquisition unit, configured to input a target tensor shape into a target bandwidth sub-model, obtain data on a target data transfer type and a target tensor shape under a first data type, and obtain reference bandwidth data when performing data transfer between two target caches of a target AI chip;

[0130] a predicted ratio relationship output unit, configured to input a target data handling type, two target buffers, and a target tensor shape into a bandwidth equivalent calculation model, and output a target predicted bandwidth ratio relationship between the first data type and the second data type;

[0131] A bandwidth data prediction unit is used to obtain data on the target data transfer type and target tensor shape under the second data type based on the reference bandwidth data and the target predicted bandwidth ratio, and to obtain the predicted bandwidth data when performing data transfer between the two target caches of the target AI chip.

[0132] Based on the above embodiments, the reference bandwidth data acquisition unit may be further configured to:

[0133] The target tensor shape and the hardware properties of the matching direct access storage hardware in the target AI chip are input into the target bandwidth sub-model.

[0134] Based on the above embodiments, the predicted proportional relationship output unit can be further used to:

[0135] The target data handling type, two target caches, target tensor shape, and hardware attribute information of the matching direct access storage hardware in the target AI chip are input into the bandwidth equivalent calculation model.

[0136] The device for calculating bandwidth equivalent during data transfer provided by an embodiment of the present invention can execute the method for calculating bandwidth equivalent during data transfer provided by any embodiment of the present invention, and has functional modules and beneficial effects corresponding to the execution method.

[0137] In the technical solutions disclosed herein, the collection, storage, use, processing, transmission, provision and disclosure of user personal information involved comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0138] Figure 6A schematic diagram of an electronic device 10 that can be used to implement an embodiment of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smartphones, wearable devices (such as helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or claimed herein.

[0139] like Figure 6 As shown, electronic device 10 includes at least one processor 11 and memory, such as read-only memory (ROM) 12 and random access memory (RAM) 13, communicatively connected to at least one processor 11. The memory stores computer programs executable by the at least one processor. Processor 11 can perform various appropriate actions and processes based on the computer programs stored in ROM 12 or loaded from storage unit 18 into RAM 13. RAM 13 can also store various programs and data required for the operation of electronic device 10. Processor 11, ROM 12, and RAM 13 are interconnected via bus 14. An input / output (I / O) interface 15 is also connected to bus 14.

[0140] Multiple components in the electronic device 10 are connected to the I / O interface 15, including an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a magnetic disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.

[0141] Processor 11 can be any general-purpose and / or specialized processing component with processing and computing capabilities. Some examples of processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various specialized artificial intelligence (AI) computing chips, various processors that run machine learning model algorithms, a digital signal processor (DSP), and any other suitable processor, controller, microcontroller, etc. Processor 11 executes the various methods and processes described above, such as the bandwidth equivalent calculation method for data transfer as described in any embodiment of the present invention.

[0142] In some embodiments, the bandwidth equivalent calculation method for data transport as described in any embodiment of the present invention may be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of the bandwidth equivalent calculation method for data transport as described in any embodiment of the present invention described above may be performed. Alternatively, in other embodiments, the processor 11 may be configured to execute the bandwidth equivalent calculation method for data transport as described in any embodiment of the present invention by any other appropriate means (for example, by means of firmware).

[0143] That is, a bandwidth test program is run in the target AI chip to intensively measure data of various data transfer types and multiple tensor shapes under a first data type, and first measured bandwidth data when data transfer is performed between cache levels of the target AI chip;

[0144] Constructing a target bandwidth model for the first data type according to the first measured bandwidth data;

[0145] Running a bandwidth test program in the target AI chip to sparsely measure data of multiple data transfer types and multiple tensor shapes under the second data type, and generating second measured bandwidth data when performing data transfer between cache levels of the target AI chip;

[0146] Constructing a bandwidth equivalence calculation model between the first data type and the second data type according to the first measured bandwidth data and the second measured bandwidth data;

[0147] According to the target bandwidth model and the bandwidth equivalent calculation model, data on the target data transfer type and the target tensor shape under the second data type are obtained, and the predicted bandwidth data when performing data transfer between the two target caches of the target AI chip is obtained.

[0148] Various embodiments of the systems and techniques described herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), system-on-chip systems (SOCs), programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include being implemented in one or more computer programs that are executable and / or interpreted on a programmable system comprising at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0149] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when the computer program is executed by the processor, the functions / operations specified in the flowcharts and / or block diagrams are implemented. The computer program may be executed entirely on the machine, partially on the machine, as a stand-alone software package, partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0150] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, device, or apparatus. A computer-readable storage medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or apparatus, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media may include an electrical connection based on one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0151] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device that has: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0152] The systems and techniques described herein can be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0153] A computing system may include clients and servers. The clients and servers are typically remote from each other and typically interact via a communication network. This client-server relationship arises through computer programs running on the respective computers, creating a client-server relationship. The server may be a cloud server, also known as a cloud computing server or cloud host. This server is a hosting product within the cloud computing service ecosystem that addresses the management difficulties and limited scalability of traditional physical hosting and VPS services.

[0154] It should be understood that the various forms of the processes shown above can be used to reorder, add, or delete steps. For example, the steps described in the present invention can be performed in parallel, sequentially, or in a different order, as long as the desired results of the technical solution of the present invention can be achieved. This is not limited herein.

[0155] The above specific embodiments do not limit the scope of protection of the present invention. Those skilled in the art will appreciate that various modifications, combinations, sub-combinations, and substitutions may be made based on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention are intended to be included within the scope of protection of the present invention.

Claims

1. A method for calculating bandwidth equivalence during data transport, characterized in that: include: Running a bandwidth test program in the target AI chip to intensively measure data of multiple data transfer types and multiple tensor shapes under a first data type, and first measured bandwidth data when performing data transfer between cache levels of the target AI chip; Constructing a target bandwidth model for the first data type according to the first measured bandwidth data; Running a bandwidth test program in the target AI chip to sparsely measure data of multiple data transfer types and multiple tensor shapes under the second data type, and generating second measured bandwidth data when performing data transfer between cache levels of the target AI chip; Constructing a bandwidth equivalence calculation model between the first data type and the second data type according to the first measured bandwidth data and the second measured bandwidth data; According to the target bandwidth model and the bandwidth equivalent calculation model, data on the target data transfer type and the target tensor shape under the second data type are obtained, and the predicted bandwidth data when performing data transfer between the two target caches of the target AI chip is obtained.

2. The method according to claim 1, characterized in that Constructing a target bandwidth model for the first data type according to the first measured bandwidth data includes: In the first measured bandwidth data, respectively obtain data of the same data transfer type under the first data type, and the measured bandwidth data when data transfer is performed between the same two caches, and construct a plurality of measured bandwidth data groups; Adding hardware attribute information of the matching direct access storage hardware in the target AI chip to each first measured bandwidth data in each measured data group; Using each measured bandwidth data group to train a preset machine learning model to obtain multiple bandwidth sub-models, and organizing each bandwidth sub-model to obtain a target bandwidth model for the first data type; The bandwidth sub-model corresponds to two buffers for setting the data transfer type and executing the data transfer.

3. The method according to claim 2, characterized in that Constructing a bandwidth equivalence calculation model between the first data type and the second data type according to the first measured bandwidth data and the second measured bandwidth data, including: Obtaining, based on the first measured bandwidth data and the second measured bandwidth data, a bandwidth ratio between a first data type and a second data type when data of a set data transfer type and a set tensor shape is transferred between two set caches in the target AI chip; Constructing a correlation data set according to the ratio relationship of each bandwidth, and constructing a bandwidth equivalence calculation model between the first data type and the second data type according to the correlation data set; The bandwidth equivalent calculation model is used to output a predicted bandwidth ratio between a first data type and a second data type according to the input settings of two caches, a data transfer type, and a tensor shape.

4. The method according to claim 3, characterized in that Constructing a bandwidth equivalence calculation model between the first data type and the second data type according to the associated data set, including: After adding hardware attribute information of the matching direct access storage hardware in the target AI chip to each associated data in the associated data set, the set machine learning model is trained using the key data set to obtain a bandwidth equivalence calculation model between the first data type and the second data type.

5. The method according to any one of claims 2 to 4, characterized in that: According to the target bandwidth model and the bandwidth equivalent calculation model, data on the target data transfer type and the target tensor shape under the second data type are obtained, and predicted bandwidth data when performing data transfer between two target caches of the target AI chip includes: Identifying a target bandwidth sub-model in the target bandwidth model based on the target data handling type and the two target buffers; Input the target tensor shape into the target bandwidth sub-model to obtain data on the target data transfer type and target tensor shape under the first data type, and reference bandwidth data when performing data transfer between the two target caches of the target AI chip; Inputting the target data handling type, two target buffers, and target tensor shape into a bandwidth equivalent calculation model, and outputting a target predicted bandwidth ratio relationship between the first data type and the second data type; Based on the proportional relationship between the reference bandwidth data and the target predicted bandwidth, data on the target data transfer type and target tensor shape under the second data type is obtained, and predicted bandwidth data when performing data transfer between the two target caches of the target AI chip is obtained.

6. The method according to claim 5, characterized in that Input the target tensor shape to the target bandwidth submodel, specifically: The target tensor shape and the hardware properties of the matching direct access storage hardware in the target AI chip are input into the target bandwidth sub-model. The target data handling type, two target buffers, and target tensor shape are input into the bandwidth equivalence calculation model, including: The target data handling type, two target caches, target tensor shape, and hardware attribute information of the matching direct access storage hardware in the target AI chip are input into the bandwidth equivalent calculation model.

7. A device for calculating bandwidth equivalent during data transport, characterized in that: include: A first bandwidth data acquisition module is configured to run a bandwidth test program in the target AI chip to intensively measure data of various data transfer types and multiple tensor shapes under a first data type, and obtain first measured bandwidth data when performing data transfer between cache levels of the target AI chip; a target bandwidth model building module, configured to build a target bandwidth model for the first data type based on the first measured bandwidth data; A second bandwidth data acquisition module is configured to run a bandwidth test program in the target AI chip, sparsely measure data of multiple data transfer types and multiple tensor shapes under the second data type, and generate second measured bandwidth data when performing data transfer between cache levels of the target AI chip; An equivalent model construction module, configured to construct a bandwidth equivalent calculation model between the first data type and the second data type based on the first measured bandwidth data and the second measured bandwidth data; The predicted bandwidth data acquisition module is used to obtain data on the target data transfer type and target tensor shape under the second data type based on the target bandwidth model and the bandwidth equivalent calculation model, and the predicted bandwidth data when performing data transfer between the two target caches of the target AI chip.

8. An electronic device, characterized in that: The electronic device comprises: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can perform the bandwidth equivalent calculation method for data transport according to any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the bandwidth equivalent calculation method for data transfer according to any one of claims 1 to 6 when executed.

10. A computer program product, characterized in that The computer program product comprises a computer program, which, when executed by a processor, implements the method for calculating bandwidth equivalence during data transport according to any one of claims 1 to 6.

Images