Computing engine construction method, device, equipment and storage medium

Abstract

The present application provides a method, device, equipment and storage medium for building a computing power engine. It relates to the field of computer data processing technology. The method is based on a computing chip, which includes a processing unit architecture, a first memory and a second memory. The processing unit architecture is connected to the first memory and the second memory, and the processing unit architecture includes at least two processing unit arrays; the method includes: according to the weight of each layer in the neural network, the original tensor is gradually decomposed into multiple sub-tensors; wherein the rank of each sub-tensor meets the set requirements; multiple sub-tensors are stored in the first memory; according to the sorting order of the sub-tensors and the computational dependencies between the sub-tensors, the calculation order is determined; according to the calculation order, the processing unit architecture is used to perform parallel calculations. The present application reduces computational redundancy by proposing a new tensor core construction and computational sorting mechanism, thereby significantly improving computational efficiency.

Description

Technical Field

[0001] The present application relates to the field of computer data processing technology, and in particular to a computing power engine construction method, device, equipment and storage medium. Background Art

[0002] The computing power engine refers to the provision of efficient data processing and storage services through the coordinated development of computing power, network capacity and storage capacity. It is not only a new strategic resource in the digital economy era, but also an important driving force for promoting high-quality economic and social development.

[0003] The existing tensor train accelerator is the most representative tensor train accelerator at present. Its architecture significantly improves the execution efficiency of the tensor train decomposition model through customized data flow and memory access mode. However, during the calculation process of the existing IT calculator, although the model size is significantly reduced by tensor train decomposition, the reduction in the amount of calculation is not as significant as the storage savings. This leads to insufficient computational savings when processing high compression ratio models, and the compression advantages brought by tensor train decomposition cannot be fully utilized. The insufficient computational savings limits the performance of the overall accelerator, resulting in the actual acceleration effect being far lower than the theoretical possibility. Summary of the invention

[0004] The present application provides a computing power engine construction method, apparatus, device and storage medium to solve the problem of insufficient computing savings during data compression.

[0005] In a first aspect, the present application provides a method for constructing a computing power engine, comprising: based on a computing chip, the computing chip comprises a processing unit architecture, a first memory and a second memory, the processing unit architecture is connected to the first memory and the second memory, and the processing unit architecture comprises at least two processing unit arrays; the method comprises: according to the weight of each layer in the neural network, the original tensor is gradually decomposed into multiple sub-tensors; wherein the rank of each sub-tensor meets the set requirements, and each sub-tensor corresponds to the weight of one layer; the multiple sub-tensors are stored in the first memory and the weights corresponding to each sub-tensor are stored in the second memory; the calculation order is determined according to the sorting order of the sub-tensors and the calculation dependency between the sub-tensors; according to the calculation order, parallel calculation is performed using the processing unit architecture.

[0006] In one possible design, the original tensor is gradually decomposed into multiple sub-tensors, including: based on the rank of the target tensor, a dimension is selected for splitting, the original tensor is decomposed into multiple intermediate tensors, and the decomposition algorithm is recursively applied to each intermediate tensor until the decomposed tensor is equal to the rank of the target tensor, and the decomposed tensor is used as a sub-tensor.

[0007] In one possible design, according to the calculation order, a processing unit array is used to perform parallel calculations, including:

[0008] Elimination stage: based on the calculation order, the first target sub-tensor in the first memory is input into the second memory; wherein the first target sub-tensor is the sub-tensor to be applied in the elimination stage; in response to the input original data, continuous multi-stage elimination calculations are performed; wherein, in the elimination calculation process of each stage, based on the calculation order, the corresponding first target sub-tensor is read from the second memory for elimination calculation, and the first calculation result obtained is directly passed to the next stage of elimination calculation, until the multi-stage elimination calculation is completed, and the intermediate data is obtained;

[0009] Expansion stage: based on the calculation order, the second target sub-tensor in the first memory is input into the second memory; wherein the second target sub-tensor is the sub-tensor to be applied in the expansion stage; in response to the input intermediate data, multiple consecutive multi-stage expansion calculations are performed; wherein, in each consecutive multi-stage expansion calculation, based on the calculation order, the corresponding second target sub-tensor is read from the second memory for expansion calculation, and the obtained second calculation result is directly passed to the next stage of expansion calculation until the multi-stage expansion calculation is completed to obtain intermediate compressed data; the intermediate compressed data obtained from each consecutive multi-stage expansion calculation is superimposed to obtain the target compressed data.

[0010] In one possible design, the original data includes one of image data, video data, voice data and text data.

[0011] In one possible design, in the process of performing parallel calculations using a processing unit array according to the calculation order, the method also includes: determining the reordering order of the stored data in the second memory according to the dimensional factors of the input data and output data of the second memory, and reorganizing the data storage structure based on the reordering order; adjusting the access data according to the size of the data accessed each time in the second memory; reading data from the second memory, performing matrix transformation, and obtaining transformed data to replace the corresponding read data.

[0012] In one possible design, the second memory is set to two, namely a third memory and a fourth memory; in the elimination stage, based on the calculation order, the first target sub-tensor in the first memory is input into the third memory; in the expansion stage, based on the calculation order, the second target sub-tensor in the first memory is input into the fourth memory.

[0013] In a second aspect, the present application provides a computing power engine construction device, comprising a computing chip and a processing unit; the computing chip comprises a processing unit architecture, a first memory and a second memory, the processing unit architecture is connected to the first memory and the second memory, and the processing unit architecture comprises at least two processing unit arrays; the processing unit is connected to the computing chip, and the processing unit is configured to: gradually decompose the original tensor into multiple sub-tensors; wherein the rank of each sub-tensor meets the set requirements; store the multiple sub-tensors in the first memory; determine the calculation order according to the sorting order of the sub-tensors and the calculation dependency between the sub-tensors; and perform parallel calculations using the processing unit architecture according to the calculation order.

[0014] In a third aspect, an embodiment of the present application provides an electronic device, comprising: at least one processor and a memory; the memory stores computer execution instructions; the at least one processor executes the computer execution instructions stored in the memory, so that the at least one processor executes the computing power engine construction method described in the first aspect and various possible designs of the first aspect.

[0015] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, in which computer execution instructions are stored. When a processor executes the computer execution instructions, the computing power engine construction method described in the first aspect and various possible designs of the first aspect is implemented.

[0016] In a fifth aspect, an embodiment of the present application provides a computer program product, including a computer program. When the computer program is executed by a processor, it implements the computing power engine construction method described in the first aspect and various possible designs of the first aspect.

[0017] The computing power engine construction method, device, equipment and storage medium provided in this application, by proposing a new tensor core construction and calculation sorting mechanism, reduces the computational redundancy of the tensor train decomposition model, thereby significantly improving the computational efficiency. This application can achieve higher computational savings while maintaining a high compression ratio, fully utilizing the advantages brought by tensor train decomposition, and significantly improving the performance of the overall accelerator. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0019] Figure 1 Construct a flow chart for the computing engine in the prior art;

[0020] Figure 2 A structural diagram of a computing chip provided in an embodiment of the present application;

[0021] Figure 3 A flowchart of a computing power engine construction method provided in an embodiment of the present application;

[0022] Figure 4 A parallel computing flow chart provided for an embodiment of the present application;

[0023] Figure 5 A flowchart of the elimination phase process provided in an embodiment of the present application, wherein (a) represents the overall process, and (b) represents a sub-process;

[0024] Figure 6 A flowchart of the expansion phase process provided for an embodiment of the present application, wherein (a) represents the overall process, and (b) represents a sub-process;

[0025] Figure 7 A flowchart of memory-free partition access provided by an embodiment of the present application;

[0026] Figure 8 A flowchart of read and write without memory partition for matrix transformation in the elimination and expansion phases provided in an embodiment of the present application;

[0027] Fig. 9 A schematic diagram of the structure of a computing power engine construction device provided in an embodiment of the present application;

[0028] Fig.10 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application.

[0029] The above drawings have shown clear embodiments of the present application, which will be described in more detail later. These drawings and text descriptions are not intended to limit the scope of the present application in any way, but to illustrate the concept of the present application to those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION

[0030] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application. Instead, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.

[0031] In the technical solution of this application, the collection, storage, use, processing, transmission, provision and disclosure of information such as financial data or user data involved shall comply with the provisions of relevant laws and regulations and shall not violate public order and good morals.

[0032] It should be noted that in the embodiments of the present application, certain software, components, models and other existing solutions in the industry may be mentioned, and they should be regarded as exemplary. Their purpose is only to illustrate the feasibility of implementing the technical solution of the present application, but it does not mean that the applicant has or will necessarily use the solution.

[0033] The technical solution of the present application and how the technical solution of the present application solves the above-mentioned technical problems are described in detail below with specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present application will be described below in conjunction with the accompanying drawings.

[0034] Figure 1 Construct a flow chart for the computing power engine in the prior art. Figure 1 As shown. The computing power engine construction in the prior art relies on the existing tensor train accelerator for execution. First, obtain an original tensor ,in, represents the set of real numbers, represents the first dimension, , express, express; represents the second dimension, , express, Represents. The original tensor is decomposed to obtain multiple sub-tensors , ... .

[0035] Subsequently, the original data X and the corresponding multiple sub-tensors are processed in stages to finally obtain data Y. In the staged processing process, the original data X is preprocessed to obtain preprocessed data , the preprocessed data With the first subtensor after transformation Perform multiplication calculation to obtain the data of stage 1 Phase 1 Data After preprocessing, we get the input data for stage 2 , and the second sub-tensor after transformation Multiplication calculation is performed to perform multi-stage processing, and finally in stage d, the input data of stage d is , which is the same as the last sub-tensor The multiplication calculation is performed to finally obtain data Y as output, thereby completing the entire data compression process.

[0036] The hardware that the above data compression process relies on is GPU.

[0037] Figure 1The defects of the data calculation process are as follows:

[0038] Defect 1: Insufficient calculation savings:

[0039] Although existing tensor train accelerators significantly reduce model size through tensor train decomposition, the reduction in computational effort is not as significant as the storage savings. This results in insufficient computational savings when processing high compression ratio models and inability to fully utilize the compression advantages brought by tensor train decomposition. Insufficient computational savings limits the performance of the overall accelerator, resulting in actual acceleration effects far lower than theoretical possibilities.

[0040] Defect 2: High storage overhead of intermediate results:

[0041] Existing Tensor Train accelerators divide the entire computation into multiple stages when executing a Tensor Train-decomposed DNN layer, and the size of the intermediate results in each stage is usually much larger than the size of the final output. This leads to significant on-chip storage overhead, weakening the low storage cost advantage brought by Tensor Train decomposition. High storage overhead increases the demand for hardware resources, leading to rising costs, and may limit the application of models in resource-constrained environments.

[0042] Defect 3: Extra data transmission:

[0043] Due to the multi-stage processing scheme, existing tensor train accelerators need to frequently transfer intermediate results between stages. The intermediate results of each stage need to be written and read from the memory multiple times, which increases the data transmission overhead. Frequent data transmission increases the number of memory accesses and energy consumption, limiting the energy efficiency and performance improvement of the overall system.

[0044] Defect 4: Limited scope of application:

[0045] Existing tensor train accelerators are mainly optimized for specific types of DNN models (such as convolutional neural networks), and may not achieve the same optimization effect for other types of models (such as recurrent neural networks and transformers). The limited scope of application restricts the versatility of the accelerator, making it impossible to be widely used in various DNN models, reducing the flexibility and applicability of the hardware.

[0046] In order to solve the above-mentioned defects, the embodiment of the present application provides a method for constructing a computing power engine. Figure 2 This is a structural diagram of the computing chip provided in the embodiment of the present application. The computing power engine construction method is based on Figure 2 The computing chip shown includes a processing unit architecture 210, a first memory 220 and a second memory 230. The processing unit architecture 210 is connected to the first memory 220 and the second memory 230. The processing unit architecture 210 includes at least two processing unit arrays 211.

[0047] In this embodiment, the processing unit array 211 in the processing unit architecture 210 includes 16 multipliers and 16 accumulators. Each processing unit array 211 performs input and output operations through the first memory 220 and the second memory 230. The first memory 220 stores model parameters, and the second memory 230 stores intermediate activation results.

[0048] This computing engine construction method uses computing chips as the hardware structure, and has important application prospects and wide applicability in industrial applications that require efficient reasoning and low power consumption, including image classification, video recognition, speech recognition, recommendation systems, and natural language processing. Specifically, through efficient decomposition technology, it can greatly compress the model size and significantly reduce the amount of calculation in tasks such as image classification, video recognition, speech recognition, recommendation systems, and natural language processing, thereby improving reasoning efficiency and reducing power consumption.

[0049] Figure 3 This is a flow chart of the method for constructing a computing power engine provided in an embodiment of the present application. Figure 3 As shown, based on Figure 2 The computing chip shown in the figure, the computing power engine construction method includes the following steps:

[0050] S301. According to the weight of each layer in the neural network, the original tensor is gradually decomposed into multiple sub-tensors; wherein the rank of each sub-tensor meets the set requirements, and each sub-tensor corresponds to the weight of a layer.

[0051] In this embodiment, the rank of the target tensor is first determined. Based on the original tensor, its initial state can be determined. The original tensor is gradually decomposed into multiple low-rank tensor cores through a recursive decomposition algorithm: a dimension is selected for segmentation, and the tensor is decomposed into two intermediate tensors. The decomposition algorithm is recursively applied to each intermediate tensor until the ranks of all decomposed tensors meet the requirements. For example, if the decomposed tensor is equal to the rank of the target tensor, it is determined that the rank of the decomposed tensor meets the requirements. At this time, the decomposed tensor is used as a sub-tensor to obtain multiple sub-tensors.

[0052] It should be noted that neural networks, for example, can use DNNs, use layer-by-layer importance quantification techniques to score each DNN layer, and calculate the impact of each layer's weight on the overall accuracy of the model; each layer is quantified by a proxy classifier, and the unique information contribution of the layer is evaluated using a weight quantization method. The higher the score, the greater the contribution to the overall performance of the model. Based on the scoring results, tensor ranks are dynamically assigned to each layer. High-importance layers in the model are assigned higher ranks to retain more feature information; while low-importance layers use lower ranks to reduce storage and computing requirements. This adaptive allocation mechanism avoids the loss of accuracy caused by inappropriate rank in traditional fixed-rank methods and significantly improves compression efficiency.

[0053] In one embodiment, a scaling factor can be introduced to enable flexible adjustment of rank allocation under different computing resource budgets. The scaling factor adjusts the overall level of tensor rank based on the computing budget, allowing the system to adapt to a variety of hardware conditions from high-performance servers to resource-constrained mobile devices. Through budget-aware rank allocation, the system can avoid repeated rank searches and meet compression requirements in different computing scenarios without retraining the model.

[0054] This embodiment can significantly reduce the computational redundancy of the tensor train decomposition model through a new tensor core construction and computation sorting mechanism, and achieve higher computational efficiency. While maintaining a high compression ratio, it can achieve higher computational savings, fully utilize the advantages brought by tensor train decomposition, and significantly improve the performance of the overall accelerator.

[0055] S302: Store multiple sub-tensors in a first memory and store weights corresponding to each sub-tensor in a second memory.

[0056] S303. Determine the calculation order according to the sorting order of the sub-tensors and the calculation dependencies between the sub-tensors.

[0057] In this embodiment, the sub-tensor is divided into a first sub-tensor and a second sub-tensor, wherein the first sub-tensor and the second sub-tensor are used for elimination calculation and extension calculation respectively, and the elimination calculation is usually performed first and then the extension calculation is performed, so the matching first sub-tensor combination and second sub-tensor combination are determined according to the calculation dependency relationship between the sub-tensors, wherein the first sub-tensor combination and the second sub-tensor combination respectively include multiple first sub-tensors and second sub-tensors, and the tensor sorting in the first sub-tensor combination and the second sub-tensor combination is used to determine the execution order of the elimination calculation and the execution order of the extension calculation respectively. In this way, the first sub-tensor combination is used to perform the elimination calculation in the execution order of the elimination calculation, and the intermediate result data is then subjected to the extension calculation, and the extension calculation is performed according to the execution order of the extension calculation, thereby determining the calculation order.

[0058] S304: Perform parallel computing using the processing unit architecture according to the computing sequence.

[0059] In this embodiment, the calculation order includes an elimination calculation order and an expansion calculation order performed in sequence. According to the elimination calculation order, an idle processing unit array can be selected to execute a calculation step respectively. Similarly, according to the expansion calculation order, an idle processing unit array can be selected to execute a calculation step respectively, so as to efficiently realize parallel calculation and finally obtain compressed data.

[0060] In some embodiments, Figure 4 The parallel computing flow chart provided in the embodiment of the present application. Figure 4 As shown, according to the calculation order, the processing unit array is used to perform parallel calculations, including:

[0061] S410, Elimination Phase:

[0062] S411, based on the calculation order, input the first target sub-tensor in the first memory into the second memory; wherein the first target sub-tensor is the sub-tensor to be applied in the elimination stage;

[0063] S412. In response to the input original data, continuous multi-stage elimination calculations are performed; wherein, in each stage of the elimination calculation process, based on the calculation order, the corresponding first target sub-tensor is read from the second memory for elimination calculation, and the first calculation result obtained is directly passed to the next stage of elimination calculation until the multi-stage elimination calculation is completed to obtain intermediate data.

[0064] For example, Figure 5 This is a flowchart of the elimination phase processing provided by the embodiment of the present application. Figure 5 As shown, in a calculation sequence, multiple parallel elimination processing threads are included. The figure shows two elimination processing threads, and both elimination processing threads include three stages.

[0065] Take one of the elimination processing threads as an example. The elimination processing thread includes three stages, namely stage 511, stage 512 and stage 513. Stage 511, stage 512 and stage 513 can be executed by a processing unit array. In stage 511, based on the input raw data and the corresponding first target subtensor , preprocess the original data and its corresponding first target sub-tensor (i.e. process the original data and the corresponding tensor into data with the same dimension), and obtain the preprocessed original data and the first target subtensor , and then and Multiply them together to get the processed data of stage 511 . Then the processing data of stage 511 And the corresponding first target sub-tensor is preprocessed to obtain the preprocessed data of stage 512 and the first target subtensor after preprocessing , in stage 512, and Multiply them together to get the processed data of stage 512 Similar to the previous step, in stage 513, the processed data of stage 512 is first obtained. and the corresponding first target sub-tensor and preprocessing them respectively to obtain the preprocessing data of stage 513 and the first target subtensor after preprocessing ;Will and Multiply to get the intermediate data The intermediate data is temporarily stored in the second memory as input for the subsequent step S420. It should be noted that , , , , , All represent dimension parameters. X represents the original input data. represents the first target sub-tensor corresponding to the original data X, represents the raw data after preprocessing, represents the first target sub-tensor corresponding to the original data X after preprocessing, represents the processing data of stage 511; represents the preprocessed data of stage 512, represents the first target subtensor after preprocessing in stage 512, represents the processing data of stage 512, represents the preprocessed data of stage 512, represents the first target subtensor after preprocessing in stage 513, Represents intermediate data.

[0066] S420, expansion phase:

[0067] S421, based on the calculation order, input the second target sub-tensor in the first memory into the second memory; wherein the second target sub-tensor is the sub-tensor to be applied in the expansion stage;

[0068] S422, in response to the input intermediate data, performing multiple consecutive multi-stage expansion calculations; wherein, in each consecutive multi-stage expansion calculation, based on the calculation order, reading the corresponding second target sub-tensor from the second memory for expansion calculation, and the obtained second calculation result is directly passed to the next stage of expansion calculation, until the multi-stage expansion calculation is completed, and the intermediate compressed data is obtained;

[0069] S423, superimposing the intermediate compressed data obtained from each continuous multi-stage expansion calculation to obtain target compressed data.

[0070] For example, Figure 6 The following is a flowchart of the expansion phase processing provided by the embodiment of the present application. Figure 6 As shown, in a calculation sequence, multiple parallel extended processing threads are included, and one of the extended processing threads is taken as an example. The extended processing thread includes three stages, each stage is executed by a processing unit array, and the three stages are stage 611, stage 612 and stage 613.

[0071] In stage 611, the intermediate data and the corresponding second target sub-tensor are read from the second memory, and then the pre-processed intermediate data of stage 611 is obtained by converting them. and the second target subtensor after preprocessing in stage 611 ;Will and Multiply to get the processed data of stage 611 .

[0072] In stage 612, the processed data of stage 611 is obtained And preprocessing is performed to obtain the preprocessed data of stage 612 At the same time, the corresponding second target sub-tensor is read from the second memory in the calculation order and converted to obtain the second target sub-tensor after the preprocessing in stage 612 ;Will and Multiply to get the processed data of stage 612 .

[0073] In stage 613, the processed data of stage 612 is obtained And preprocessing is performed to obtain the preprocessed data of stage 613 At the same time, the corresponding second target sub-tensor is read from the second memory in the calculation order and converted to obtain the second target sub-tensor after the preprocessing in stage 613 ;Will and Multiply to get the processed data of stage 613 .

[0074] It should be noted that the data processing principle of each stage in each extended processing thread is basically the same, that is, multiplying the corresponding second target sub-tensor with the input data, but the stages included in each extended processing thread are not the same, and it has at least one stage, such as Figure 6 The two threads 610 and 620 are shown, where thread 610 includes three stages and thread 620 includes two stages, namely stage 621 and stage 622. The processed data obtained by each thread is pre-processed to a set dimension and then superimposed to obtain the final target compressed data. .

[0075] in, , , Respectively represent the length of the corresponding data; , , Respectively represent the width of the corresponding data.

[0076] Therefore, the embodiment of the present application reduces the staged calculation and storage costs and reduces the demand for on-chip storage resources by optimizing the storage mode of the intermediate results. Through the new computing scheme and data flow optimization, the storage overhead of the intermediate results is reduced, so that the hardware resource requirements are reduced, the cost is reduced, and it is more suitable for application in resource-constrained environments. That is, the method provided by the present application can significantly reduce the storage overhead and overcome the defect 2 existing in the prior art described above.

[0077] In some embodiments, Figure 7 The non-memory partition access flow chart provided in the embodiment of the present application. Figure 3 In step S304 shown in the figure, in the process of performing parallel calculation using the processing unit array according to the calculation order, the method further includes:

[0078] S701. Determine a reordering order of stored data in the second memory according to dimension factors of input data and output data of the second memory, and reorganize a data storage structure based on the reordering order.

[0079] S702: Adjust access data according to the size of the data accessed each time in the second memory.

[0080] In this embodiment, by determining the size of the data block accessed each time, access conflicts between different data blocks are avoided, and the access sequence is designed to minimize memory access conflicts, thereby achieving efficient access.

[0081] S703: Read data from the second memory, perform matrix transformation, and obtain transformed data to replace the corresponding read data.

[0082] In this embodiment, step S703 is used to ensure the consistency and correctness of the data.

[0083] Therefore, the embodiment of the present application eliminates unnecessary stage-by-stage data movement through a forward-looking cross-stage processing solution, reduces the number of memory accesses and data transmission overhead, reduces energy consumption, and improves overall performance. That is, the method provided by the present application can significantly reduce data transmission, overcoming the defect 3 existing in the prior art described above.

[0084] Figure 8 The matrix transformation without memory partition read and write flow chart of the elimination and expansion phases provided in the embodiment of the present application. Figure 8 As shown, in the elimination stage, the intermediate data stored in the second memory is , whose length is , the width is , according to the dimension factors of the input data and the output data of the second memory, the corresponding data is extracted from the storage address of the intermediate data, and reorganized to obtain the first reorganized data , first reorganized data Length is , with a width of 1. This allows the first reorganized data to be directly read from the second memory during the expansion phase. As input, the steps of data preprocessing are reduced, thereby improving data processing efficiency.

[0085] In the expansion stage, the intermediate compressed data stored in the second memory is , whose length is , the width is 1, according to the dimension factors of the input data and the output data of the second memory, the corresponding data is extracted from the storage address of the intermediate data, and reorganized to obtain the second reorganized data , whose length is , the width is In this way, when performing data superposition processing in the expansion phase, the second reorganized data can be directly read from the second memory. The superposition is performed without the need for a pre-processing step, thereby improving data processing efficiency.

[0086] In some embodiments, the second memory is set to two, namely a third memory and a fourth memory; in the elimination stage, based on the calculation order, the first target sub-tensor in the first memory is input into the third memory; in the expansion stage, based on the calculation order, the second target sub-tensor in the first memory is input into the fourth memory.

[0087] In this embodiment, the intermediate data obtained in the elimination stage can be stored in the fourth memory, so that the expansion stage reads and writes data from the fourth memory without passing through the third memory. By setting two second memories as buffers, efficient data exchange and parallel computing can be supported.

[0088] In summary, the method provided by this application can support a wide range of DNN models, including LSTM, Transformer, DLRM, etc., by unifying the calculation formats of different types of DNNs. It demonstrates its wide applicability and versatility, and improves the flexibility and applicability of hardware. That is, this application has wide applicability and overcomes the defect 4 existing in the prior art described above.

[0089] Fig. 9 This is a schematic diagram of the structure of the computing power engine construction device provided in the embodiment of the present application. The embodiment of the present application also provides a computing power engine construction device. Fig. 9 As shown, the computing power engine construction device includes a computing chip 910 and a processing unit 920; the computing chip 910 includes a processing unit architecture, a first memory and a second memory, the processing unit architecture is connected to the first memory and the second memory, and the processing unit architecture includes at least two processing unit arrays;

[0090] The processing unit 920 is connected to the computing chip 910, and the processing unit 910 is configured to: gradually decompose the original tensor into multiple sub-tensors; wherein the rank of each sub-tensor meets the set requirements; store the multiple sub-tensors in the first memory; determine the calculation order according to the sorting order of the sub-tensors and the calculation dependency between the sub-tensors; and perform parallel calculations using the processing unit architecture according to the calculation order.

[0091] In some embodiments, the processing unit is further configured to: based on the rank of the target tensor, select a dimension for splitting, decompose the original tensor into multiple intermediate tensors, and recursively apply the decomposition algorithm to each intermediate tensor until the decomposed tensor is equal to the rank of the target tensor, and use the decomposed tensor as a sub-tensor.

[0092] In some embodiments, the processing unit is further configured to:

[0093] Based on the calculation order, inputting the first target sub-tensor in the first memory into the second memory; wherein the first target sub-tensor is the sub-tensor to be applied in the elimination stage;

[0094] In response to the input original data, a continuous multi-stage elimination calculation is performed; wherein, in each stage of the elimination calculation process, based on the calculation order, the corresponding first target sub-tensor is read from the second memory for elimination calculation, and the first calculation result obtained is directly passed to the next stage of elimination calculation until the multi-stage elimination calculation is completed to obtain intermediate data;

[0095] Based on the calculation order, input the second target sub-tensor in the first memory into the second memory; wherein the second target sub-tensor is the sub-tensor to be applied in the expansion stage;

[0096] In response to the input intermediate data, multiple consecutive multi-stage expansion calculations are performed; wherein, in each consecutive multi-stage expansion calculation, based on the calculation order, the corresponding second target sub-tensor is read from the second memory for expansion calculation, and the obtained second calculation result is directly passed to the next stage of expansion calculation until the multi-stage expansion calculation is completed to obtain the intermediate compressed data;

[0097] The intermediate compressed data obtained from each continuous multi-stage expansion calculation are superimposed to obtain the target compressed data.

[0098] In some embodiments, the original data includes one of image data, video data, voice data and text data.

[0099] In some embodiments, the processing unit is further configured to: determine the reordering order of the stored data in the second memory according to the dimension factors of the input data and the output data of the second memory, and reorganize the data storage structure based on the reordering order; adjust the access data according to the size of the data accessed each time in the second memory; read the data from the second memory, perform matrix transformation, and obtain the transformed data to replace the corresponding read data.

[0100] In some embodiments, the second memory is set to two, namely a third memory and a fourth memory; in the elimination stage, based on the calculation order, the first target sub-tensor in the first memory is input into the third memory; in the expansion stage, based on the calculation order, the second target sub-tensor in the first memory is input into the fourth memory.

[0101] Fig.10 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. Fig.10 As shown, the electronic device may include: a processor 101 and a memory 102 , wherein the processor 101 and the memory 102 may communicate with each other; illustratively, the processor 101 and the memory 102 communicate with each other via a communication bus 103 .

[0102] The processor 101 executes the computer execution instructions stored in the memory 102, so that the processor 101 executes the scheme in the above embodiment. The processor 101 can be a general-purpose processor, including a central processing unit, a network processor, etc.; it can also be a digital signal processor, an application-specific integrated circuit ASIC, a field programmable gate array FPGA or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components.

[0103] The communication bus 103 may be a peripheral component interconnection standard bus or an extended industrial standard structure bus. The system bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus. The transceiver is used to realize the communication between the database access device and other computers (such as a client, a read-write library, and a read-only library). The memory may include a random access memory and may also include a non-volatile memory.

[0104] The electronic device provided in the embodiment of the present application may be the terminal device of the above embodiment.

[0105] An embodiment of the present application also provides a computer-readable storage medium, which stores computer instructions. When the computer instructions are executed on a computer, the computer executes the technical solution of the computing power engine construction method of the above embodiment.

[0106] An embodiment of the present application also provides a computer program product, which includes a computer program stored in a computer-readable storage medium. At least one processor can read the computer program from the computer-readable storage medium. When at least one processor executes the computer program, it can implement the technical solution of the computing power engine construction method in the above embodiment.

[0107] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of modules is only a logical function division, and there may be other division methods in actual implementation, such as multiple modules can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or modules, which can be electrical, mechanical or other forms.

[0108] The modules described as separate components may or may not be physically separated, and the components shown as modules may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to implement the solution of this embodiment.

[0109] In addition, each functional module in each embodiment of the present application can be integrated into one processing unit, or each module can exist physically separately, or two or more modules can be integrated into one unit. The above-mentioned module-composed unit can be implemented in the form of hardware or in the form of hardware plus software functional units.

[0110] The above-mentioned integrated module implemented in the form of a software function module can be stored in a computer-readable storage medium. The above-mentioned software function module is stored in a storage medium, including a number of instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor to perform some steps of the methods of various embodiments of the present application.

[0111] It should be understood that the processor may be a central processing unit, or other general-purpose processors, digital signal processors, application-specific integrated circuits, etc. A general-purpose processor may be a microprocessor or the processor may be any conventional processor, etc. The steps of the method disclosed in the invention may be directly embodied as being executed by a hardware processor, or may be executed by a combination of hardware and software modules in the processor.

[0112] The memory may include a high-speed RAM memory, and may also include a non-volatile storage NVM, such as at least one disk memory, and may also be a USB flash drive, a mobile hard disk, a read-only memory, a magnetic disk or an optical disk, etc.

[0113] The bus can be an industrial standard architecture bus, a peripheral device interconnection bus, or an extended industrial standard architecture bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, the bus in the drawings of this application is not limited to only one bus or one type of bus.

[0114] The storage medium may be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory, electrically erasable programmable read-only memory, erasable programmable read-only memory, programmable read-only memory, read-only memory, magnetic memory, flash memory, magnetic disk or optical disk. The storage medium may be any available medium that can be accessed by a general or special purpose computer.

[0115] An exemplary storage medium is coupled to a processor so that the processor can read information from the storage medium and write information to the storage medium. Of course, the storage medium can also be a component of the processor. The processor and the storage medium can be located in a dedicated integrated circuit. Of course, the processor and the storage medium can also exist as discrete components in an electronic control unit or a main control device.

[0116] Those skilled in the art can understand that all or part of the steps of implementing the above-mentioned method embodiments can be completed by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, the steps of the above-mentioned method embodiments are executed; and the aforementioned storage medium includes: ROM, RAM, disk or optical disk and other media that can store program codes.

[0117] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for constructing a computing power engine, characterized in that: Based on a computing chip, the computing chip includes a processing unit architecture, a first memory and a second memory, the processing unit architecture is connected to the first memory and the second memory, and the processing unit architecture includes at least two processing unit arrays; the method includes: According to the weight of each layer in the neural network, the original tensor is gradually decomposed into multiple sub-tensors; the rank of each sub-tensor meets the set requirements, and each sub-tensor corresponds to the weight of a layer; Storing the plurality of sub-tensors in the first memory and storing the weights corresponding to the sub-tensors in the second memory; Determine the calculation order according to the sorting order of the sub-tensors and the calculation dependencies between the sub-tensors; According to the calculation sequence, performing parallel calculations using a processing unit architecture; According to the calculation sequence, the parallel calculation is performed using the processing unit array, including: Elimination Phase: Based on the calculation order, inputting the first target sub-tensor in the first memory into the second memory; wherein the first target sub-tensor is the sub-tensor to be applied in the elimination stage; In response to the input original data, a continuous multi-stage elimination calculation is performed; wherein, in each stage of the elimination calculation process, based on the calculation order, the corresponding first target sub-tensor is read from the second memory for elimination calculation, and the first calculation result obtained is directly passed to the next stage of elimination calculation until the multi-stage elimination calculation is completed to obtain intermediate data; Expansion phase: Based on the calculation order, input the second target sub-tensor in the first memory into the second memory; wherein the second target sub-tensor is the sub-tensor to be applied in the expansion stage; In response to the input intermediate data, multiple consecutive multi-stage expansion calculations are performed; wherein, in each consecutive multi-stage expansion calculation, based on the calculation order, the corresponding second target sub-tensor is read from the second memory for expansion calculation, and the obtained second calculation result is directly passed to the next stage of expansion calculation until the multi-stage expansion calculation is completed to obtain the intermediate compressed data; The intermediate compressed data obtained from each continuous multi-stage expansion calculation are superimposed to obtain the target compressed data.

2. The method according to claim 1, characterized in that The original tensor is gradually decomposed into multiple sub-tensors, including: Based on the rank of the target tensor, a dimension is selected for segmentation, the original tensor is decomposed into multiple intermediate tensors, and the decomposition algorithm is recursively applied to each intermediate tensor until the decomposed tensor is equal to the rank of the target tensor, and the decomposed tensor is used as a sub-tensor; wherein the rank of the target tensor is determined according to the weight of each layer in the neural network.

3. The method according to claim 1, characterized in that The original data includes one of image data, video data, voice data and text data.

4. The method according to claim 1, characterized in that: In the process of performing parallel calculations using a processing unit array according to the calculation sequence, the method further includes: determining a reordering order of the stored data in the second memory according to the dimension factors of the input data and the output data of the second memory, and reorganizing the data storage structure based on the reordering order; adjusting access data according to the size of the data accessed each time in the second memory; Data is read from the second memory, matrix transformation is performed, and transformed data is obtained to replace the corresponding read data.

5. The method according to claim 1, characterized in that The second memory is set to two, namely a third memory and a fourth memory; In the elimination phase, based on the calculation order, the first target subtensor in the first memory is input into the third memory; In the expansion stage, the second target sub-tensor in the first memory is input into the fourth memory based on the calculation order.

6. A computing power engine construction device, characterized in that: including computing chips and processing units; The computing chip includes a processing unit architecture, a first memory and a second memory, the processing unit architecture is connected to the first memory and the second memory, and the processing unit architecture includes at least two processing unit arrays; The processing unit is connected to the computing chip, and the processing unit is configured to: decompose the original tensor step by step into a plurality of sub-tensors; wherein the rank of each sub-tensor meets the set requirements; store the plurality of sub-tensors in the first memory; determine the calculation order according to the sorting order of the sub-tensors and the calculation dependency relationship between the sub-tensors; and perform parallel calculations using the processing unit architecture according to the calculation order; The processing unit is further configured to: Based on the calculation order, inputting the first target sub-tensor in the first memory into the second memory; wherein the first target sub-tensor is the sub-tensor to be applied in the elimination stage; In response to the input original data, a continuous multi-stage elimination calculation is performed; wherein, in each stage of the elimination calculation process, based on the calculation order, the corresponding first target sub-tensor is read from the second memory for elimination calculation, and the first calculation result obtained is directly passed to the next stage of elimination calculation until the multi-stage elimination calculation is completed to obtain intermediate data; Based on the calculation order, input the second target sub-tensor in the first memory into the second memory; wherein the second target sub-tensor is the sub-tensor to be applied in the expansion stage; In response to the input intermediate data, multiple consecutive multi-stage expansion calculations are performed; wherein, in each consecutive multi-stage expansion calculation, based on the calculation order, the corresponding second target sub-tensor is read from the second memory for expansion calculation, and the obtained second calculation result is directly passed to the next stage of expansion calculation until the multi-stage expansion calculation is completed to obtain the intermediate compressed data; The intermediate compressed data obtained from each continuous multi-stage expansion calculation are superimposed to obtain the target compressed data.

7. An electronic device, characterized in that: include: A processor, and a memory communicatively connected to the processor; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory to implement the method according to any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, which are used to implement the method according to any one of claims 1 to 5 when executed by a processor.

9. A computer program product, characterized in that The invention comprises a computer program, which implements the method according to any one of claims 1 to 5 when being executed by a processor.

Images