Tensor calculation method and device and distributed calculation system

Abstract

The invention provides a tensor calculation method and device and a distributed calculation system, and relates to the technical field of artificial intelligence, and the method comprises the steps: sending a to-be-processed tensor list in a current calculation task to a second processor corresponding to a first processor, and releasing the memory space occupied by the to-be-processed tensor list; the second processor is used for flattening the to-be-processed tensor list to obtain a first tensor; receiving a first tensor sent by the second processor, and allocating a memory space for the first tensor; performing anti-flattening processing on the first tensor to obtain a second tensor; and executing the current calculation task based on the second tensor. According to the method and the device provided by the invention, the demand of the neural network on memory resources when the neural network executes tensor calculation is reduced.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a tensor calculation method, device and distributed computing system. Background Art

[0002] In the calculation process of the neural network model, there is a scenario where the tensor list is flattened. If the tensor list is too large, it may cause out of memory (OOM) during the flattening operation, affecting the training process or reasoning process of the neural network model.

[0003] Therefore, how to reduce the demand for memory resources of neural network models during tensor calculations has become a technical problem that needs to be solved urgently in the industry. Summary of the invention

[0004] The present invention provides a tensor calculation method, device and distributed computing system for solving the technical problem of how to reduce the demand for memory resources of a neural network model during tensor calculation.

[0005] The present invention provides a tensor calculation method, applied to a first processor, comprising: Sending a list of tensors to be processed in the current computing task to a second processor corresponding to the first processor, and releasing the memory space occupied by the list of tensors to be processed; the second processor is used to flatten the list of tensors to be processed to obtain a first tensor; receiving a first tensor sent by the second processor, and allocating memory space for the first tensor; Performing a deflattening process on the first tensor to obtain a second tensor; The current computing task is performed based on the second tensor.

[0006] In some embodiments, performing a deflattening process on the first tensor to obtain a second tensor includes: Based on the multidimensional data structure in the list of tensors to be processed, the first tensor is deflattened to obtain the second tensor.

[0007] In some embodiments, executing the current computing task based on the second tensor includes: During execution of the current computing task, the to-be-processed tensor list is replaced based on the second tensor.

[0008] In some embodiments, the first processor is an artificial intelligence processor, and the second processor is a central processing unit that is communicatively connected to the artificial intelligence processor.

[0009] The present invention provides a tensor computing device, applied to a first processor, comprising: A sending module, used to send the list of tensors to be processed in the current computing task to the second processor corresponding to the first processor, and release the memory space occupied by the list of tensors to be processed; the second processor is used to flatten the list of tensors to be processed to obtain the first tensor; A receiving module, configured to receive a first tensor sent by the second processor and allocate memory space for the first tensor; A processing module, configured to perform a deflattening process on the first tensor to obtain a second tensor; A computing module is used to execute the current computing task based on the second tensor.

[0010] The present invention provides a distributed computing system, comprising a plurality of first processors and a plurality of second processors; the first processors and the second processors are connected based on a bus; the first processors are artificial intelligence processors; the second processors are central processing units; The first processor is used to execute the tensor calculation method.

[0011] In some embodiments, any second processor in the system executes the following method: Obtaining the real-time utilization of each second processor and obtaining computing task information of each first processor; Based on the computing task information of each first processor, create a tensor flattening process corresponding to each first processor; the tensor flattening process is used to flatten the list of tensors to be processed in the first processor; Determining the scheduling priority of each second processor based on the real-time utilization of each second processor; wherein the scheduling priority of the second processor with lower real-time utilization is higher; Based on the scheduling priorities of the second processors, the tensor flattening processes corresponding to the first processors are scheduled to the second processors.

[0012] In some embodiments, the system adopts a non-uniform memory access architecture to set up a local memory for each second processor; When the second processor runs the tensor flattening process, it stores the list of tensors to be processed in the tensor flattening process into the corresponding local memory.

[0013] The present invention provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the tensor calculation method is implemented when the processor executes the computer program.

[0014] The present invention provides a non-transitory computer-readable storage medium having a computer program stored thereon, wherein the computer program implements the tensor calculation method when executed by a processor.

[0015] The tensor calculation method, device and distributed computing system provided by the present invention send the list of tensors to be processed in the current computing task to the second processor corresponding to the first processor, and release the memory space occupied by the list of tensors to be processed; the second processor is used to flatten the list of tensors to be processed to obtain a first tensor; receive the first tensor sent by the second processor, and allocate memory space for the first tensor; de-flatten the first tensor to obtain a second tensor; execute the current computing task based on the second tensor; because the first processor transfers the flattening processing of the tensor list to the second processor for execution, the memory space is released after sending the tensor list, and the memory space is allocated again after receiving the flattened tensor, so that the memory space of the first processor does not need to allocate memory space for the tensor list and the flattened tensor at the same time, reducing the demand for memory resources of the neural network when performing tensor calculations; in addition, the second processor is also reasonably utilized, reducing the occupancy of the first processor, so that the first processor can more efficiently process computing tasks related to the neural network. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0017] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0018] Figure 1 It is a flowchart of the tensor calculation method provided by the present invention.

[0019] Figure 2 It is a structural schematic diagram of the tensor computing device provided by the present invention.

[0020] Figure 3 It is a schematic diagram of the structure of the distributed computing system provided by the present invention.

[0021] Figure 4 It is a schematic diagram of the tensor calculation process provided by the present invention.

[0022] Figure 5 It is a peak memory usage diagram provided by the related technology.

[0023] Figure 6 This is a schematic diagram of peak video memory usage provided by the present invention.

[0024] Figure 7 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0025] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0026] It should be noted that the terms "first", "second", etc. in the present invention are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units or modules is not necessarily limited to those steps or units or modules that are clearly listed, but may include other steps or units or modules that are not clearly listed or inherent to these processes, methods, products or devices.

[0027] In deep learning, flattening and unflattening of tensor lists are two common operations, which are usually used in collective communication and optimizers of multiple processors. Flattening tensor lists and operating them together can improve performance. Tensor is a universal representation of multi-dimensional data. Its dimensions can be zero-dimensional (scalar), one-dimensional (vector), two-dimensional (matrix) or higher-dimensional structures. A tensor list consists of multiple tensors. Flattening a tensor list is to convert multiple tensors into one tensor, which contains all the data in multiple tensors. Unflattening is to convert the flattened tensors back to their original multiple tensors. When flattening multiple tensors, if the amount of data contained in the tensors is large, it is easy to run out of memory, resulting in the calculation task being affected. In severe cases, the calculation task needs to be re-executed.

[0028] In order to solve the shortcomings of related technologies, Figure 1 It is a flowchart of the tensor calculation method provided by the present invention, such as Figure 1 As shown, the method is applied to a first processor, and includes step 110 , step 120 , step 130 and step 140 .

[0029] Step 110: Send the list of tensors to be processed in the current computing task to the second processor corresponding to the first processor, and release the memory space occupied by the list of tensors to be processed; the second processor is used to flatten the list of tensors to be processed to obtain the first tensor.

[0030] Specifically, the execution subject of the tensor calculation method provided in the embodiment of the present invention is a tensor calculation device. The device can be implemented by software, such as a tensor calculation program running in a processor; or by hardware, such as a tensor calculation device can be a specific calculation unit or core as part of the first processor.

[0031] The application scenario of the method provided in the embodiment of the present invention is to train or reason a neural network using a distributed computing system. The embodiment of the present invention does not specifically limit the model structure of the neural network.

[0032] For example, when a neural network is used to perform tasks such as image processing, speech processing, or text processing, a distributed computing system can be used to train the neural network. The current computing task can be to train the neural network using image samples, speech samples, or text samples. The list of tensors to be processed can be multiple multidimensional feature vectors obtained after feature extraction of image samples, speech samples, or text samples, such as image feature vectors (including feature vectors of multiple channels), speech feature vectors (including feature vectors of multiple frequencies), or text feature vectors (including semantic feature vectors and text position feature vectors, etc.). It is understandable that if there are many input samples and the amount of data in the tensor list is large, it may lead to risks such as memory overflow.

[0033] The current computing task refers to the computing task of training or reasoning a neural network model in a distributed computing system. In the training and reasoning of a neural network model, the model parameters and gradient parameters are generally composed of several tensors. The sequence or set of these tensors can be called a tensor list. Since the communication efficiency and computing efficiency of larger tensors are often higher than those of smaller tensors, the tensor list can be flattened to obtain a larger tensor. Or because of different data conversions in the neural network, flattening is required. For example, when the convolutional layer passes data to the fully connected layer, the multi-dimensional tensor needs to be flattened into a one-dimensional tensor. The list of tensors to be processed is a list of tensors that need to be flattened.

[0034] A plurality of first processors and a plurality of second processors may be provided in the distributed computing system. The first processor is used to perform computing tasks related to neural network training or reasoning. The second processor is used to perform other related tasks, such as hardware management, parameter configuration, and human-computer interaction. In an embodiment of the present invention, it may be determined that the execution subject of the method is the first processor.

[0035] In a distributed computing system, the first processor and the second processor may be connected by bus communication, such as a high-speed serial computer expansion bus (Peripheral Component Interconnect express, PCIe). The first processor and the second processor usually have a corresponding relationship. For example, the distributed computing system may include 8 first processors and 2 second processors. Among them, every 4 first processors are connected to 1 second processor for communication. Therefore, for any of the first processors, it can be determined that there is a unique second processor corresponding to it.

[0036] When executing the current computing task, the first processor may determine to send the list of tensors to be processed to the corresponding second processor and release the memory space occupied by the list of tensors to be processed.

[0037] Memory space refers to the space in the computer's memory used to store and process data. In the case where the first processor is a graphics processor, the memory space is also called video memory space. Video memory space refers to the memory used by the graphics processing unit to store and process data related to deep learning.

[0038] The second processor can flatten the received list of tensors to be processed to obtain a first tensor. The flattening process mainly converts multiple multi-dimensional tensors in the list of tensors to be processed into a one-dimensional tensor. The first tensor is the tensor after flattening.

[0039] Step 120: Receive the first tensor sent by the second processor, and allocate memory space for the first tensor.

[0040] Specifically, after the second processor performs the flattening process, a first tensor is generated, and the first tensor is sent to the second processor. After receiving the first tensor, the second processor allocates memory space for it.

[0041] Step 130: De-flatten the first tensor to obtain a second tensor.

[0042] Specifically, the first processor performs a deflattening process on the first tensor to obtain a second tensor. The second tensor is a deflattened tensor.

[0043] Step 140: Execute the current computing task based on the second tensor.

[0044] Specifically, when executing the current computing task, the first processor may replace the list of tensors to be processed with the second tensor.

[0045] The tensor calculation method provided by the embodiment of the present invention sends the list of tensors to be processed in the current calculation task to the second processor corresponding to the first processor, and releases the memory space occupied by the list of tensors to be processed; the second processor is used to flatten the list of tensors to be processed to obtain a first tensor; receive the first tensor sent by the second processor, and allocate memory space for the first tensor; de-flatten the first tensor to obtain a second tensor; execute the current calculation task based on the second tensor; because the first processor transfers the flattening process of the tensor list to the second processor for execution, the memory space is released after sending the tensor list, and the memory space is allocated again after receiving the flattened tensor, so that the memory space of the first processor does not need to allocate memory space for the tensor list and the flattened tensor at the same time, reducing the demand for memory resources of the neural network when performing tensor calculation; in addition, the second processor is also reasonably utilized, reducing the occupancy of the first processor, so that the first processor can more efficiently process computing tasks related to the neural network.

[0046] It should be noted that each implementation of the present invention can be freely combined, the order can be changed, or it can be executed separately, and does not need to rely on or depend on a fixed execution order.

[0047] In some embodiments, performing a deflattening process on the first tensor to obtain a second tensor includes: Based on the multidimensional data structure in the list of tensors to be processed, the first tensor is deflattened to obtain a second tensor.

[0048] Specifically, the flattening process is used to flatten a multi-dimensional tensor to obtain a one-dimensional tensor. For example, the multi-dimensional features output by the convolutional layer are flattened and sent to the fully connected layer for further processing. The unflattening process requires restoring the multi-dimensional data structure of the tensor. For example, the one-dimensional tensor output by the fully connected layer is restored to a multi-dimensional tensor that is compatible with convolution operations, etc.

[0049] The multidimensional data structure can be described by the shape, dimension or data type of the tensor. According to the multidimensional data structure in the list of tensors to be processed, the first tensor can be deflattened to obtain the second tensor. The obtained second tensor can continue to be used to execute the current computing task.

[0050] The tensor calculation method provided in an embodiment of the present invention performs deflating processing on the first tensor according to the multidimensional data structure in the list of tensors to be processed to obtain the second tensor, which is conducive to the first processor to efficiently execute the current calculation task.

[0051] In some embodiments, executing the current computing task based on the second tensor includes: During the execution of the current computing task, the list of tensors to be processed is replaced based on the second tensor.

[0052] Specifically, since the second tensor is obtained by deflating the first tensor according to the multidimensional data structure in the list of tensors to be processed, the list of tensors to be processed can be directly replaced by the second tensor in the current computing task.

[0053] The tensor calculation method provided by the embodiment of the present invention replaces the list of tensors to be processed with the second tensor, which is conducive to the first processor to efficiently execute the current calculation task.

[0054] In some embodiments, the first processor is an artificial intelligence processor, and the second processor is a central processing unit that is communicatively connected to the artificial intelligence processor.

[0055] Specifically, the artificial intelligence processor can be any one of a GPU (Graphics Processing Unit), a TPU (Tensor Processing Unit), a NPU (Neural network Processing Unit), and a GPGPU (General-Purpose computing on Graphics Processing Unit).

[0056] The following uses the GPU as an example of an AI processor. The GPU and the central processing unit (CPU) are two main processors in computer systems, and they differ significantly in function, design, and application.

[0057] Graphics processors are designed for large-scale parallel computing and are used to accelerate workloads that require large-scale parallel processing, such as deep learning and scientific computing. The central processing unit is mainly responsible for executing various instructions in computer programs and performing logical and arithmetic operations. It handles most computing tasks, especially those related to system management, task scheduling, input and output, etc.

[0058] The tensor calculation method provided in the embodiment of the present invention determines the artificial intelligence processor as the first processor and the central processing unit as the second processor, which can effectively utilize the computing resources of the central processing unit to provide support for the computing tasks of the neural network.

[0059] The following describes an apparatus provided by an embodiment of the present invention. The apparatus described below and the method described above can refer to each other.

[0060] Figure 2 is a schematic diagram of the structure of the tensor calculation device provided by the present invention, such as Figure 2 As shown, the device can be used as a computing unit or core in a first processor, including: The sending module 210 is used to send the list of tensors to be processed in the current computing task to the second processor corresponding to the first processor, and release the memory space occupied by the list of tensors to be processed; the second processor is used to flatten the list of tensors to be processed to obtain the first tensor; A receiving module 220, configured to receive a first tensor sent by a second processor and allocate memory space for the first tensor; A processing module 230 is used to perform a deflattening process on the first tensor to obtain a second tensor; The computing module 240 is configured to execute the current computing task based on the second tensor.

[0061] The tensor computing device provided by the embodiment of the present invention sends the list of tensors to be processed in the current computing task to the second processor corresponding to the first processor, and releases the memory space occupied by the list of tensors to be processed; the second processor is used to flatten the list of tensors to be processed to obtain a first tensor; receive the first tensor sent by the second processor, and allocate memory space for the first tensor; de-flatten the first tensor to obtain a second tensor; execute the current computing task based on the second tensor; because the first processor transfers the flattening processing of the tensor list to the second processor for execution, the memory space is released after sending the tensor list, and the memory space is allocated again after receiving the flattened tensor, so that the memory space of the first processor does not need to allocate memory space for the tensor list and the flattened tensor at the same time, reducing the demand for memory resources of the neural network when performing tensor calculations; in addition, the second processor is also reasonably utilized, reducing the occupancy of the first processor, so that the first processor can more efficiently process computing tasks related to the neural network.

[0062] Figure 3 is a schematic diagram of the structure of the distributed computing system provided by the present invention, such as Figure 3 As shown, the distributed computing system 300 includes a plurality of first processors 310 and a plurality of second processors 320 ; the first processors 310 and the second processors 320 are connected based on a bus 330 .

[0063] The first processor is an artificial intelligence processor. The second processor is a central processing unit. The first processor is used to execute the tensor calculation method in the above embodiment.

[0064] Specifically, Figure 4 is a schematic diagram of the tensor calculation process provided by the present invention, such as Figure 4As shown, when the artificial intelligence processor is a graphics processing unit (GPU), the graphics processor sends the list of tensors to be processed (gputensor list) in the current computing task to the corresponding central processing unit (CPU), and releases the memory space occupied by the list of tensors to be processed; the central processing unit is used to flatten the list of tensors to be processed to obtain the first tensor (cpu flattensor). The graphics processor receives the first tensor sent by the central processing unit and allocates memory space for the first tensor. The graphics processor unflattens the first tensor to obtain the second tensor (gpu flat tensor). The graphics processor updates the tensor data with the second tensor, replaces the list of tensors to be processed, and continues to execute the current computing task. Among them, the list of tensors to be processed includes n tensors, which can be expressed as tensor 1, tensor2, ..., tensor n.

[0065] Figure 5 This is a peak memory usage diagram provided by related technologies, such as Figure 5 As shown in the figure, assuming that the tensor list (tensorlist) contains 16 tensors (tensor) of size 256 megabytes (MB), the video memory occupied before flattening (flatten) is 256MB×16=4 gigabytes (GB). During flattening, 4GB of video memory is newly allocated to store the flattening results (flattensor), and the peak video memory is 8GB.

[0066] It is understandable that in the scenario of large model training, the number of model parameters is very large, and video memory will become one of the bottlenecks. When flattening, excessive peak video memory will affect video memory utilization, easily leading to memory overflow or requiring more graphics processors to accommodate the entire model.

[0067] Figure 6 Schematic diagram of peak memory usage provided by the present invention. Figure 6 As shown, the method provided by the embodiment of the present invention first unloads the tensor list to the central processing unit, flattens it, and releases the video memory on the graphics processing unit at the same time, then performs the flattening operation on the central processing unit, copies the flattened processing result (flat tensor) to the graphics processing unit, and replaces the data of the original tensor list. In this way, there will never be two copies of data on the graphics processing unit, achieving the purpose of reducing the peak video memory.

[0068] The distributed computing system provided by the embodiment of the present invention reduces the demand for memory resources of the neural network when performing tensor calculations because the first processor is used to execute the tensor calculation method in the above embodiment; in addition, the second processor is also reasonably utilized to reduce the occupancy of the first processor, so that the first processor can more efficiently process computing tasks related to the neural network.

[0069] In some embodiments, any second processor in the system performs the following method: Obtaining the real-time utilization of each second processor and obtaining computing task information of each first processor; Based on the computing task information of each first processor, a tensor flattening process corresponding to each first processor is created; the tensor flattening process is used to flatten the list of tensors to be processed in the first processor; Determine the scheduling priority of each second processor based on the real-time utilization of each second processor; the scheduling priority of the second processor with lower real-time utilization is higher; Based on the scheduling priorities of the second processors, the tensor flattening processes corresponding to the first processors are scheduled to the second processors.

[0070] Specifically, utilization refers to the percentage of time that the processor is effectively used, usually in %, indicating the degree of actual processor workload. In short, the higher the processor utilization, the heavier the processor workload; the lower the utilization, the more idle time the processor has and the lighter load. Real-time utilization indicates the resource usage and efficiency of the second processor (central processing unit) at the current moment.

[0071] The computing task information refers to information related to the computing task executed by the first processor (graphics processor), for example, the computing task information may include whether to perform tensor flattening processing, and a list of tensors that need to be flattened.

[0072] The distributed computing system provided by the embodiment of the present invention can schedule the tensor flattening process according to the real-time utilization of each second processor. The method can be executed by any second processor in the system.

[0073] The second processor can obtain the real-time utilization of each second processor and the computing task information of each first processor. According to the computing task information of each first processor, if a list of tensors that need to be flattened is included, a tensor flattening process corresponding to each first processor can be created. The tensor flattening process is used to flatten the list of tensors to be processed in the first processor. A tensor flattening process corresponds to the implementation of tensor flattening on a first processor.

[0074] The scheduling priority is used to indicate the priority level of the second processor to be scheduled to execute the tensor flattening process, which can be determined according to the real-time utilization of each second processor. The scheduling priority of the second processor with lower real-time utilization is higher; the scheduling priority of the second processor with higher real-time utilization is lower.

[0075] Then, according to the scheduling priority of each second processor, the tensor flattening process corresponding to each first processor is scheduled to each second processor in sequence.

[0076] The distributed computing system provided by the embodiment of the present invention determines the scheduling priority of each second processor according to the real-time utilization rate of each second processor, and schedules the tensor flattening process corresponding to each first processor to each second processor in turn, can dynamically perform task scheduling, reasonably utilize the second processor, and reduce the occupancy of the first processor, so that the first processor can more efficiently process the computing tasks related to the neural network, thereby improving the computing efficiency of the distributed computing system.

[0077] In some embodiments, the system uses a non-uniform memory access architecture to set up local memory for each second processor; When the second processor runs the tensor flattening process, it stores the list of tensors to be processed in the tensor flattening process into the corresponding local memory.

[0078] Specifically, the Non-Uniform Memory Access (NUMA) architecture is a computer memory design used to optimize memory access in multi-processor systems. The NUMA architecture distributes memory across multiple processor (CPU) nodes so that each processor has faster access to a portion of memory (local memory) and slower access to the memory of other processors (remote memory).

[0079] In an embodiment of the present invention, each second processor (central processing unit) can set a corresponding local memory. When running a tensor flattening process, the list of tensors to be processed in the tensor flattening process can be stored in the corresponding local memory, which can effectively reduce memory access latency and improve the computing efficiency of the distributed computing system.

[0080] Figure 7 is a schematic diagram of the structure of the electronic device provided by the present invention, such as Figure 7As shown, the electronic device may include: a processor (Processor) 710, a communication interface (Communications Interface) 720, a memory (Memory) 730 and a communication bus (Communications Bus) 740, wherein the processor 710, the communication interface 720, and the memory 730 communicate with each other through the communication bus 740. The processor 710 may call the logic command in the memory 730 to execute the method described in the above embodiment, for example: Send the list of tensors to be processed in the current computing task to the second processor corresponding to the first processor, and release the memory space occupied by the list of tensors to be processed; the second processor is used to flatten the list of tensors to be processed to obtain a first tensor; receive the first tensor sent by the second processor and allocate memory space for the first tensor; de-flatten the first tensor to obtain a second tensor; and execute the current computing task based on the second tensor.

[0081] In addition, the logic commands in the above-mentioned memory can be implemented in the form of software functional units and can be stored in a computer-readable storage medium when sold or used as an independent product. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several commands to enable a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc., which can store program code.

[0082] The processor in the electronic device provided in the embodiment of the present invention can call the logic instructions in the memory to implement the above method. Its specific implementation method is consistent with the implementation method of the aforementioned method and can achieve the same beneficial effects, which will not be repeated here.

[0083] An embodiment of the present invention further provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the method provided in the above embodiments is implemented.

[0084] Its specific implementation is consistent with the aforementioned method implementation and can achieve the same beneficial effects, so it will not be repeated here.

[0085] An embodiment of the present invention provides a computer program product, including a computer program, and when the computer program is executed by a processor, the above method is implemented.

[0086] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0087] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0088] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A tensor calculation method, applied to a first processor, characterized in that: include: Sending a list of tensors to be processed in the current computing task to a second processor corresponding to the first processor, and releasing the memory space occupied by the list of tensors to be processed; The second processor is used for flattening the list of tensors to be processed to obtain a first tensor; receiving a first tensor sent by the second processor, and allocating memory space for the first tensor; Performing a deflattening process on the first tensor to obtain a second tensor; The current computing task is performed based on the second tensor.

2. The tensor calculation method according to claim 1, characterized in that: The performing a deflattening process on the first tensor to obtain a second tensor includes: Based on the multidimensional data structure in the list of tensors to be processed, the first tensor is deflattened to obtain the second tensor.

3. The tensor calculation method according to claim 2, characterized in that: The executing the current computing task based on the second tensor includes: During execution of the current computing task, the to-be-processed tensor list is replaced based on the second tensor.

4. The tensor calculation method according to claim 1, characterized in that: The first processor is an artificial intelligence processor, and the second processor is a central processing unit that is communicatively connected to the artificial intelligence processor.

5. A tensor computing device, applied to a first processor, characterized in that: include: A sending module, used to send the list of tensors to be processed in the current computing task to the second processor corresponding to the first processor, and release the memory space occupied by the list of tensors to be processed; The second processor is used for flattening the list of tensors to be processed to obtain a first tensor; A receiving module, configured to receive a first tensor sent by the second processor and allocate memory space for the first tensor; A processing module, configured to perform a deflattening process on the first tensor to obtain a second tensor; A computing module is used to execute the current computing task based on the second tensor.

6. A distributed computing system, characterized in that: It includes a plurality of first processors and a plurality of second processors; the first processors and the second processors are connected based on a bus; the first processor is an artificial intelligence processor; the second processor is a central processing unit; The first processor is used to execute the tensor calculation method described in any one of claims 1 to 4.

7. The distributed computing system according to claim 6, characterized in that: Any second processor in the system executes the following method: Obtaining the real-time utilization of each second processor and obtaining computing task information of each first processor; Based on the computing task information of each first processor, create a tensor flattening process corresponding to each first processor; The tensor flattening process is used to flatten the list of tensors to be processed in the first processor; Determining the scheduling priority of each second processor based on the real-time utilization of each second processor; wherein the scheduling priority of the second processor with lower real-time utilization is higher; Based on the scheduling priorities of the second processors, the tensor flattening processes corresponding to the first processors are scheduled to the second processors.

8. The distributed computing system according to claim 7, characterized in that: The system adopts a non-uniform memory access architecture to set a local memory for each second processor; When the second processor runs the tensor flattening process, it stores the list of tensors to be processed in the tensor flattening process into the corresponding local memory.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the tensor calculation method according to any one of claims 1 to 4 is implemented.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the tensor calculation method according to any one of claims 1 to 4 is implemented.

Images