A tensor splitting method, device, storage medium and program product

Abstract

Embodiments of the present application provide a tensor splitting method, device, storage medium and program product, which can be applied to the field of artificial intelligence chip technology. In the method, an input tensor and a target splitting type corresponding to the input tensor are obtained, a target kernel function matched with the target splitting type is selected from a plurality of kernel functions corresponding to a task operator, the plurality of kernel functions correspond to a plurality of splitting types; when the target splitting type is a non-uniform splitting type, the input tensor is split into a plurality of sub-tensors by executing the target kernel function, wherein the shapes and values of the plurality of sub-tensors are different; the plurality of sub-tensors are respectively dispatched to a plurality of dies in an artificial intelligence chip for processing to obtain an output tensor, and the plurality of sub-tensors with different shapes and values are dispatched to the plurality of dies for processing, thereby solving the problem that tensors cannot be further split in parallel computing, the plurality of dies process different sub-tensors respectively, resource waste is reduced, and the performance of the multi-die chip is improved.

Description

TECHNICAL FIELD

[0001] Embodiments of the present application relate to the technical field of artificial intelligence chip, and particularly relate to a tensor splitting method, device, storage medium and program product. BACKGROUND

[0002] In the field of distributed computing, data parallel mode is used to split large-scale data to multiple execution devices for parallel processing to speed up the computing process. The collaborative work of multiple execution devices depends on the splitting type of tensor to regulate the operation; for example, the splitting type includes equal division type, which divides the complete tensor into different execution devices for processing in the batch dimension.

[0003] Under the related technology, the distributed computing task (such as model training task) is migrated to a double-die chip for execution. However, in some scenarios, the tensor in the distributed computing task cannot be further divided; for example, the tensor with a batch of 1; at this time, the equal division type of the splitting type is difficult to meet the computing needs of the double-die chip, thereby causing the chip performance to decline. SUMMARY

[0004] Embodiments of the present application provide a tensor splitting method, device, storage medium and program product, which are used to meet the computing needs of multi-die chip design and improve the performance of the chip.

[0005] In one aspect, the present application provides a tensor splitting method, which comprises:

[0006] obtaining an input tensor and a target splitting type corresponding to the input tensor;

[0007] selecting a target kernel function matched with the target splitting type from a plurality of kernel functions corresponding to a task operator, the plurality of kernel functions corresponding to a plurality of splitting types;

[0008] when the target splitting type is a non-equal division type, splitting the input tensor into a plurality of sub-tensors by executing the target kernel function, wherein the shapes and values of the plurality of sub-tensors are different;

[0009] scheduling the plurality of sub-tensors to a plurality of dies in an artificial intelligence chip for processing to obtain an output tensor.

[0010] In one aspect, the present application provides a tensor splitting device, which comprises:

[0011] an obtaining module configured to obtain an input tensor and a target splitting type corresponding to the input tensor;

[0012] The selecting module is configured to select a target kernel function matching the target split type from a plurality of kernel functions corresponding to a plurality of split types;

[0013] The splitting module is configured to split the input tensor into a plurality of sub-tensors by executing the target kernel function when the target split type is a non-uniform split type, wherein the plurality of sub-tensors are different in shape and value.

[0014] The scheduling module is configured to schedule the plurality of sub-tensors to a plurality of dies in an artificial intelligence chip for processing respectively to obtain an output tensor.

[0015] The scheduling module is further configured to, when the target split type is a non-uniform split type, correspond the output tensor to the non-uniform split type.

[0016] The scheduling module is specifically configured to:

[0017] schedule the plurality of sub-tensors to a plurality of dies in an artificial intelligence chip for processing respectively through an extended interface corresponding to the non-uniform split type to obtain an output tensor.

[0018] The scheduling module is specifically configured to:

[0019] schedule the plurality of sub-tensors to a plurality of dies in an artificial intelligence chip for processing respectively through an extended interface corresponding to the non-uniform split type and a communication operator corresponding to the non-uniform split type to obtain an output tensor.

[0020] The splitting module is further configured to:

[0021] split the input tensor into a plurality of sub-tensors by executing the target kernel function when the target split type is a uniform split type, wherein the plurality of sub-tensors are the same in shape.

[0022] schedule the plurality of sub-tensors to a plurality of dies in an artificial intelligence chip for processing respectively through a general interface to obtain an output tensor.

[0023] The splitting module is further configured to:

[0024] copy the input tensor to obtain a plurality of sub-tensors by executing the target kernel function when the target split type is a broadcast split type, wherein the plurality of sub-tensors are the same in shape and value.

[0025] schedule the plurality of sub-tensors to a plurality of dies in an artificial intelligence chip for processing respectively through a general interface to obtain an output tensor.

[0026] The splitting module is further configured to:

[0027] when the target split type is a partial type, splitting the input tensor into a plurality of sub-tensors by executing the target kernel function, wherein the plurality of sub-tensors have the same shape;

[0028] scheduling the plurality of sub-tensors to a plurality of dies in the artificial intelligence chip respectively through a general interface for processing to obtain an output tensor.

[0029] In one aspect, the embodiment of the present application provides a computer device, comprising:

[0030] a memory configured to store program instructions;

[0031] an artificial intelligence chip configured to run the program instructions stored in the memory and execute the steps of the tensor splitting method according to the obtained program.

[0032] In one aspect, the embodiment of the present application provides a computer readable storage medium storing a computer program executable by a computer device, and when the program runs on the computer device, the computer device executes the steps of the tensor splitting method.

[0033] In one aspect, the embodiment of the present application provides a computer program product comprising a computer program stored on a computer readable storage medium, and the computer program comprises program instructions, and when the program instructions are executed by a computer device, the computer device executes the steps of the tensor splitting method.

[0034] In the embodiment of the present application, the input tensor and the target split type corresponding to the input tensor are obtained, a target kernel function matched with the target split type is selected from a plurality of kernel functions corresponding to a task operator, the plurality of kernel functions correspond to a plurality of split types, when the target split type is a non-uniform split type, the input tensor is split into a plurality of sub-tensors by executing the target kernel function, wherein the plurality of sub-tensors have different shapes and values, and the plurality of sub-tensors are scheduled to a plurality of dies in the artificial intelligence chip respectively for processing to obtain an output tensor, so that the input tensor is non-uniformly split to obtain a plurality of sub-tensors with different shapes and values, the problem that the tensor cannot be further split in the distributed computing task is solved, and the plurality of dies can process different sub-tensors, thereby improving the performance of the multi-die chip. BRIEF DESCRIPTION OF DRAWINGS

[0035] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings needed in the embodiment description. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0036] Figure 1A structural schematic diagram of a chip architecture provided by an embodiment of the present application is shown in FIG. 1.

[0037] Figure 2 A flowchart of a tensor splitting method provided by an embodiment of the present application is shown in FIG. 2.

[0038] Figure 3 A flowchart of another tensor splitting method provided by an embodiment of the present application is shown in FIG. 3.

[0039] Figure 4 A structural schematic diagram of a tensor splitting device provided by an embodiment of the present application is shown in FIG. 4.

[0040] Figure 5 A structural schematic diagram of a computer device provided by an embodiment of the present application is shown in FIG. 5. DETAILED DESCRIPTION

[0041] In order to make the objectives, technical solutions and beneficial effects of the present application clearer, the present application will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and do not limit the present application. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative labor fall within the scope of the present application.

[0042] Reference Figure 1 A structural diagram of an artificial intelligence chip applicable to an embodiment of the present application is shown in FIG. 1, which includes at least a plurality of dies 101. Each die includes a plurality of computing units 102, wherein the computing unit 102 can be a Streaming Processing Cluster (SPC).

[0043] Each computing unit 102 includes an on-chip cache 103 and a plurality of execution units 104. The execution unit 104 is used to process computing tasks such as model training tasks.

[0044] The on-chip cache 103 is a temporary memory with small capacity but fast data exchange speed. The on-chip cache 103 can be a Gemm Main Buffer (GMB) or other types of memory.

[0045] In addition to the above structure, the artificial intelligence chip 100 in the present application can also include other structures, which are not specifically limited in the present application.

[0046] Artificial intelligence chips 100 can be: Graphics Processing Unit (GPU), General-purpose computing on graphics processing units (GPGPU), Domain Specific Architecture (DSA), etc.

[0047] This application is based on Figure 1 The diagram of the AI ​​chip architecture shown illustrates a tensor segmentation method applicable to various scenarios, such as image processing, speech processing, and text processing. The physical meaning of the input tensor used for segmentation can differ depending on the application.

[0048] For example, in a text processing scenario, the input tensor for segmentation can be text data used in tasks such as text generation and text recognition.

[0049] For example, in a speech processing scenario, the input tensor used for segmentation can be speech data used in tasks such as speech enhancement, speech recognition, and speech synthesis.

[0050] For example, in image processing scenarios, the input tensor for segmentation can be image data used in tasks such as image preprocessing, image segmentation, and object detection.

[0051] It should be noted that in this application, "first" and "second" are used for descriptive purposes only and should not be construed as indicating relative importance or implying the number of technical features indicated.

[0052] This application uses a dual-die packaged AI chip as an example to illustrate the specific process of tensor partitioning, but it should not be construed as meaning that the solution in this application is only applicable to tensor partitioning of AI chips containing two dies. In practical applications, tensors can be partitioned to AI chips containing N dies, where N is greater than or equal to 2.

[0053] In other words, this application does not limit the specific number of dies contained in the artificial intelligence chip; that is, the artificial intelligence chip can contain one or more dies.

[0054] The following is based on Figure 1 The diagram shown illustrates the architecture of an AI chip and details the process of a tensor partitioning method. (See also...) Figure 2 This method is executed by a computer device that contains an artificial intelligence chip, which can be... Figure 1The artificial intelligence chip 101 shown herein includes the following steps:

[0055] Step 201: Obtain the input tensor and the target segmentation type corresponding to the input tensor.

[0056] Specifically, in distributed model training or parallel computing tasks, input tensors are raw data passed from outside the model for computation. Input tensors have corresponding target partitioning types so that the central processing unit can partition them to adapt to data parallelism scenarios across multiple devices.

[0057] Step 202: Select the target kernel function that matches the target segmentation type from the multiple kernel functions corresponding to the task operator. Multiple kernel functions correspond to multiple segmentation types.

[0058] Specifically, any task operator has multiple kernel functions that support different segmentation types. Once the target segmentation type of the input tensor is determined, a target kernel function that matches the target segmentation type is selected from the multiple kernel functions so that the target kernel function performs the corresponding operation.

[0059] Step 203: When the target segmentation type is non-uniform (Double), the input tensor is segmented into multiple sub-tensors by executing the target kernel function, wherein the multiple sub-tensors have different shapes and values.

[0060] Specifically, the input tensor includes one or more dimensions; when the input tensor is a scalar, it corresponds to one dimension. When the input tensor is a two-dimensional array, it corresponds to two dimensions; for example, the H dimension and the W dimension. When the input tensor is a three-dimensional matrix, it corresponds to three dimensions; for example, the batch dimension, the H dimension, and the W dimension.

[0061] The input tensor must include at least one partitioning dimension. For example, when the input tensor is a three-dimensional matrix, the partitioning dimension is the batch dimension.

[0062] The input tensor is divided into multiple subtensors according to the partitioning dimension using the target kernel function. These subtensors differ in both shape and value. The shape represents the length of the subtensor in each dimension, and the value represents the value of any element within the subtensor. In practical applications, the shapes of the multiple subtensors differ along the partitioning dimension, but are identical along other dimensions.

[0063] If a subtensor is a two-dimensional array, then any two two-dimensional arrays have different shapes and contain different values ​​of their elements; the same applies to subtensors in the form of three-dimensional matrices, which will not be elaborated here.

[0064] Taking a dual-die AI chip as an example, the input tensor is... The matrix:

[0065] , including H dimension and W dimension; the matrix X is cut from the W dimension (i.e. the split dimension) into the first sub-tensor of (two rows and three columns) and the second sub-tensor of (one row and three columns), the first sub-tensor is , the second sub-tensor is , the shape and data of the first sub-tensor and the second sub-tensor are different.

[0066] For another example, the dimensions include a batch dimension, the input tensor is cut from the batch dimension, so that the dimensions of the sub-tensor on the first chip in the subsequent artificial intelligence chip are (4, 3, 224, 224), i.e. the batch dimension is 4; the dimensions of the sub-tensor on the second chip are (8, 3, 224, 224), wherein the batch dimension is 8.

[0067] In the embodiments of the present application, the target kernel function splits the input tensor into multiple sub-tensors according to the corresponding target split mode, so that the multiple sub-tensors obtained by splitting have different shapes and values, so as to solve the problem that the dimensions of the input tensor itself are not suitable for uniform scenarios, and at the same time, different dies process different shapes of sub-tensors, further improving the performance of the artificial intelligence chip.

[0068] Step 204, the multiple sub-tensors are respectively scheduled to multiple dies in the artificial intelligence chip for processing, and an output tensor is obtained.

[0069] Referring to Figure 3 , the artificial intelligence chip includes two dies, which are a first chip die0 and a second chip die1, and the model weight parameters included in the die0 and the model weight parameters included in the die1 are the same, i.e. the models deployed on the first chip die0 and the second chip die1 are the same, the input tensor is cut to obtain the first sub-tensor and the second sub-tensor, the first sub-tensor is scheduled to the first chip die0 for processing, and the second sub-tensor is scheduled to the second chip die1 for processing.

[0070] In the embodiments of the present application, the input tensor and the target split type corresponding to the input tensor are obtained, a target kernel function matching the target split type is selected from a plurality of kernel functions corresponding to the task operator, the plurality of kernel functions correspond to a plurality of split types; when the target split type is a non-uniform split type, the input tensor is split into a plurality of sub-tensors by executing the target kernel function, wherein the shapes and values of the plurality of sub-tensors are different; the plurality of sub-tensors are respectively dispatched to a plurality of dies in the artificial intelligence chip for processing to obtain an output tensor, so that the input tensor is non-uniformly split to obtain a plurality of sub-tensors with different shapes and values, solving the problem that the tensor in the distributed computing task cannot be further split, and enabling the plurality of dies to process different sub-tensors, thereby improving the performance of the multi-die chip.

[0071] In some embodiments, the plurality of sub-tensors are respectively dispatched to the plurality of dies in the artificial intelligence chip for processing to obtain the output tensor through an extension interface corresponding to the non-uniform split type.

[0072] Specifically, after the central processing unit splits the input tensor into a plurality of sub-tensors, the sub-tensors are respectively dispatched to the plurality of dies according to the pre-set communication parameters through the extension interface; wherein the communication parameters are determined by the upper logic of the extension interface.

[0073] In some embodiments, the plurality of sub-tensors are respectively dispatched to the plurality of dies in the artificial intelligence chip for processing to obtain the output tensor through the extension interface corresponding to the non-uniform split type and a communication operator corresponding to the non-uniform split type.

[0074] Specifically, the communication operator is an abstract definition of data interaction between the central processing unit and the graphics processing unit in the framework or program. When the plurality of sub-tensors need to be dispatched from the central processing unit to the plurality of dies, the communication operator is first triggered, and the communication operator performs preparation work such as accepting instructions of the upper logic, checking the legality of data, determining transmission parameters and transmission modes, etc. to determine the communication parameters; after the communication operator completes the preparation work, the bottom extension interface is actively called to convert the instructions into hardware understandable operations; the extension interface dispatches each sub-tensor to the corresponding die according to the communication parameters. Wherein, the extension interface is used to dispatch the plurality of sub-tensors (i.e. the plurality of sub-tensors obtained by splitting the input tensor according to the non-uniform split type) to the plurality of dies in the artificial intelligence chip.

[0075] For example, the artificial intelligence chip in a dual-die form includes a first chip and a second chip; the central processing unit splits the input tensor into a first sub-tensor and a second sub-tensor with different shapes and values; the communication operator first confirms that the first sub-tensor needs to be scheduled to the first chip and the second sub-tensor needs to be scheduled to the second chip, then checks the legality of the first sub-tensor and the second sub-tensor, and then determines the transmission direction (such as host to hardware device), dimension, and other basic information of the first sub-tensor and the second sub-tensor; and then determines the transmission mode as synchronous transmission (or asynchronous transmission).

[0076] Then, the communication operator calls the extension interface to transmit the communication parameters (such as the dimension of the first sub-tensor is (3, 2), the dimension of the second sub-tensor is (3, 1), and the transmission mode is synchronous transmission) determined in the previous step to the calling interface. The extension interface directly interacts with the graphics processing unit (GPU) to schedule the first sub-tensor to the first chip and the second sub-tensor to the second chip according to the communication parameters; after the scheduling is completed, the calling interface returns the scheduling success information to the communication operator, and the communication operator feeds back the scheduling result to the upper layer program.

[0077] In the embodiments of the present application, through the hierarchical abstraction principle of the communication operator and the extension interface, the upper layer user can simplify the operation on the central processing unit based on the operator, and the bottom layer can ensure the compatibility of software and hardware through the extension interface, thereby realizing efficient communication between the central processing unit and the hardware.

[0078] In some embodiments, when the target split type is a non-uniform split type, the output tensor corresponds to the non-uniform split type.

[0079] Specifically, the plurality of sub-tensors are processed to obtain an output tensor, and the split type corresponding to the output tensor (i.e., the target split type of the input tensor) is directly obtained. That is, the split type of the output tensor does not need to be derived based on the split type of the input tensor.

[0080] In the embodiments of the present application, the output tensor obtained after the die processes the sub-tensor is also a non-uniform split type, which ensures the consistency of the split types of the input data and the output data in the model training task, thereby improving the accuracy of the model training.

[0081] In some embodiments, when the target split type is a uniform split type, the input tensor is split into a plurality of sub-tensors by executing a target kernel function, wherein the plurality of sub-tensors have the same shape; and the plurality of sub-tensors are respectively scheduled to a plurality of dies in the artificial intelligence chip for processing by a general interface to obtain an output tensor.

[0082] Specifically, the uniform split type (split) refers to splitting the input tensor into a plurality of sub-tensors with the same shape, and whether the values between the plurality of sub-tensors are the same is not limited in the present application.

[0083] For matrix data, the equal division type is only applicable to an input tensor with even row dimension and even column dimension. For example, an input tensor is a matrix:

[0084] including an H dimension and a W dimension; the matrix X is divided into a first sub-tensor (two rows and two columns) and a second sub-tensor (two rows and two columns) from the W dimension (i.e., the split dimension), the first sub-tensor is , the second sub-tensor is , the shapes of the first sub-tensor and the second sub-tensor are the same (i.e., the H dimension and the W dimension of the two sub-tensors are the same), but the values of the first sub-tensor and the second sub-tensor are different due to the different values of the elements in the input tensor; if the input tensor is: , the matrix X is divided from the W dimension (i.e., the split dimension), the shapes and values of the first sub-tensor and the second sub-tensor are the same.

[0085] In some embodiments, when the target split type is the broadcast type, a plurality of sub-tensors with the same shape and value are obtained by performing the target kernel function on the input tensor to obtain a plurality of sub-tensors; the plurality of sub-tensors are respectively dispatched to a plurality of dies in the artificial intelligence chip through the general interface to obtain an output tensor.

[0086] Specifically, the broadcast type (Broadcast) refers to copying the input tensor to obtain a plurality of sub-tensors with the same shape and value. The central processing unit can first execute a communication operator to determine the communication parameters of the input tensor of the broadcast type, and call the general interface through the communication operator to enable the general interface to dispatch the plurality of sub-tensors to the plurality of dies based on the communication parameters; wherein the communication parameters include data validity, transmission parameters, transmission mode and other information.

[0087] Taking an artificial intelligence chip including a first chip and a second chip as an example, if an input tensor is a matrix (four rows and two columns) including an H dimension and a W dimension, the matrix X is copied, the first sub-tensor and the second sub-tensor are: , i.e., the first chip and the second chip subsequently process the same data.

[0088] ​In some embodiments, when the target split type is the partial type, the input tensor is split into a plurality of sub-tensors by performing the target kernel function, wherein the plurality of sub-tensors have the same shape; the plurality of sub-tensors are respectively dispatched to a plurality of dies in the artificial intelligence chip for processing through the general interface, and an output tensor is obtained.

[0089] Specifically, the partial type (Parallel) means that the plurality of sub-tensors after splitting keep the same shape as the input tensor, but the numerical values of any sub-tensor are different from the numerical values of the input tensor, and whether the numerical values of any two sub-tensors are the same is not limited in the present application.

[0090] Taking an artificial intelligence chip including a first chip and a second chip as an example, if the input tensor is a 4-row 2-column matrix , the first sub-tensor obtained by splitting can be , and the second sub-tensor is , that is, the shapes of the first sub-tensor, the second sub-tensor, and the corresponding input tensor are the same, but the numerical values of the first sub-tensor and the second sub-tensor are the same, and the numerical values of the first sub-tensor, the second sub-tensor, and the corresponding input tensor are different; the first sub-tensor can also be , and the second sub-tensor is , that is, the shapes of the first sub-tensor, the second sub-tensor, and the corresponding input tensor are the same, but the numerical values of the first sub-tensor, the second sub-tensor, and the input tensor are all different.

[0091] It should be noted that the central processing unit can first execute the communication operator to determine the communication parameters of the input tensor of the broadcast type, and call the general interface through the communication operator, so that the general interface dispatches the plurality of sub-tensors to the plurality of dies based on the communication parameters; wherein the communication parameters include data legitimacy, transmission parameters, transmission mode and the like.

[0092] For the input tensor of the split type of the equal division type, the broadcast type, and the partial type, the plurality of sub-tensors can be respectively called to the plurality of dies through the general interface; or the general interface includes sub-interfaces respectively for the equal division type, the broadcast type, and the partial type, for example, the plurality of sub-tensors of the input tensor split according to the equal division type are dispatched to the plurality of dies through the first sub-interface, the plurality of sub-tensors of the input tensor copied according to the broadcast type are dispatched to the plurality of dies through the second sub-interface, and the plurality of sub-tensors of the input tensor split according to the partial type are dispatched to the plurality of dies through the third sub-interface.

[0093] It should be noted that for the input tensors of the equal division type, the broadcast type, and the partial type, before the general interface is called to schedule multiple sub-tensors to multiple dies, the central processor executes the communication operator, so that the communication operator calls the general interface, and simplification of the upper-layer operation is realized; wherein, the scheduling operation of the input tensors of the equal division type, the broadcast type, and the partial type can be executed by the same communication operator, or can be executed by different communication operators, and the application does not make a specific limitation in this regard.

[0094] In the embodiment of the application, for any function task operator, such as a task operator for performing a splitting operation in the central processor, a communication operator, or a calculation operator for processing a sub-tensor in a double-die chip, a kernel function suitable for different input tensor splitting types is arranged in each operator, so that each task operator executes a kernel function matched with the splitting type of the input tensor or the sub-tensor when receiving the input tensor or the sub-tensor, the flexibility of processing is improved, and compared with the equal division or replication form in the traditional way, more complex scenarios can be supported, such as how to allocate an input tensor with an odd batch to a double-die chip, and the generality of tensor splitting is improved.

[0095] Based on the same technical concept, an embodiment of the application provides a structural diagram of a tensor splitting device, as shown in Figure 4 The tensor splitting device 400 comprises:

[0096] The acquisition module 401 is configured to acquire an input tensor and a target splitting type corresponding to the input tensor.

[0097] The selection module 402 is configured to select a target kernel function matched with the target splitting type from a plurality of kernel functions corresponding to the task operator, wherein the plurality of kernel functions correspond to a plurality of splitting types.

[0098] The splitting module 403 is configured to split the input tensor into a plurality of sub-tensors by executing the target kernel function when the target splitting type is a non-equal division type, wherein the plurality of sub-tensors are different in shape and value.

[0099] The scheduling module 404 is configured to schedule the plurality of sub-tensors to a plurality of dies in an artificial intelligence chip for processing, and obtain an output tensor.

[0100] The scheduling module 404 is further configured to: when the target splitting type is a non-equal division type, the output tensor corresponds to the non-equal division type.

[0101] The scheduling module 404 is specifically configured to:

[0102] The plurality of sub-tensors are scheduled to a plurality of dies in an artificial intelligence chip for processing through an extension interface corresponding to the non-equal division type, and an output tensor is obtained.

[0103] The scheduling module 404 is specifically configured to:

[0104] The multiple sub-tensors are respectively scheduled to multiple dies in the artificial intelligence chip for processing through the extension interface corresponding to the non-uniform division type and the communication operator corresponding to the non-uniform division type, and an output tensor is obtained.

[0105] The splitting module 403 is further configured to:

[0106] When the target splitting type is the uniform type, the input tensor is split into multiple sub-tensors by executing the target kernel function, wherein the multiple sub-tensors have the same shape;

[0107] The multiple sub-tensors are respectively scheduled to multiple dies in the artificial intelligence chip for processing through the general interface, and an output tensor is obtained.

[0108] The splitting module 403 is further configured to:

[0109] When the target splitting type is the broadcast type, the input tensor is replicated to obtain multiple sub-tensors by executing the target kernel function, wherein the multiple sub-tensors have the same shape and values;

[0110] The multiple sub-tensors are respectively scheduled to multiple dies in the artificial intelligence chip for processing through the general interface, and an output tensor is obtained.

[0111] The splitting module 403 is further configured to:

[0112] When the target splitting type is the partial type, the input tensor is split into multiple sub-tensors by executing the target kernel function, wherein the multiple sub-tensors have the same shape;

[0113] The multiple sub-tensors are respectively scheduled to multiple dies in the artificial intelligence chip for processing through the general interface, and an output tensor is obtained.

[0114] In this embodiment, the input tensor and its corresponding target segmentation type are obtained. From multiple kernel functions corresponding to the task operator, a target kernel function matching the target segmentation type is selected. These multiple kernel functions correspond to various segmentation types. When the target segmentation type is a non-uniform segmentation type, the input tensor is segmented into multiple sub-tensors by executing the target kernel function. These sub-tensors have different shapes and values. The multiple sub-tensors are then scheduled to multiple dies in the artificial intelligence chip for processing to obtain the output tensor. This non-uniform segmentation of the input tensor, resulting in multiple sub-tensors with different shapes and values, solves the problem of tensors being unable to be further segmented in distributed computing tasks. Furthermore, it enables multiple dies to process different sub-tensors separately, improving the performance of multi-die chips.

[0115] Based on the same technical concept, embodiments of this application provide a computer device, such as... Figure 5 As shown, it includes at least one artificial intelligence chip 100 and a memory 501 connected to at least one artificial intelligence chip 100. In this embodiment, the specific connection medium between the artificial intelligence chip 100 and the memory 501 is not limited. Figure 5 Taking the connection between the AI ​​chip 100 and the memory 501 via a bus as an example, the bus can be divided into address bus, data bus, control bus, etc.

[0116] In this embodiment of the application, the memory 501 stores instructions that can be executed by at least one artificial intelligence chip 100. By executing the instructions stored in the memory 501, at least one artificial intelligence chip 100 can perform the steps of the tensor segmentation method described above.

[0117] The artificial intelligence chip 100 serves as the control center of the computer device, connecting to various parts of the device via various interfaces and lines. It performs tensor segmentation by running or executing instructions stored in the memory 501 and accessing data stored in the memory 501. Optionally, the artificial intelligence chip 100 may include one or more processing units. It may integrate an application processor and a modem processor, where the application processor primarily handles the operating system, user interface, and applications, while the modem processor primarily handles wireless communication. It is understood that the modem processor may not be integrated into the artificial intelligence chip 100. In some embodiments, the artificial intelligence chip 100 and the memory 501 may be implemented on the same chip; in other embodiments, they may be implemented on separate chips.

[0118] The artificial intelligence chip 100 can be a general processor, such as a central processing unit (CPU), a digital signal processor, an application specific integrated circuit (ASIC), a field programmable gate array or other programmable logic device, a discrete gate or transistor logic device, a discrete hardware component, and can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of the present application. The general processor can be a microprocessor or any conventional processor. The steps of the method disclosed in combination with the embodiments of the present application can be directly embodied as execution completed by a hardware processor, or executed by a combination of hardware and software modules in the processor.

[0119] The memory 501 is a non-volatile computer readable storage medium, which can be used to store non-volatile software programs, non-volatile computer executable programs and modules. The memory 501 can include at least one type of storage medium, such as flash memory, hard disk, multimedia card, card type memory, random access memory (RAM), static random access memory (SRAM), programmable read only memory (PROM), read only memory (ROM), electrically erasable programmable read only memory (EEPROM), magnetic memory, magnetic disk, optical disk, etc. The memory 501 is any other medium capable of carrying or storing desired program code in the form of instructions or data structures and capable of being accessed by a computer device, but is not limited thereto. The memory 501 in the embodiments of the present application can also be a circuit or any other device capable of realizing a storage function, used for storing program instructions and / or data.

[0120] Based on the same inventive concept, the embodiments of the present application provide a computer readable storage medium storing a computer program executable by a computer device, which, when executed on the computer device, causes the computer device to perform the steps of the tensor splitting method described above.

[0121] Based on the same inventive concept, the embodiments of the present application provide a computer program product, which includes a computer program stored on a computer readable storage medium, and the computer program includes program instructions, which, when executed by a computer device, cause the computer device to perform the steps of the tensor splitting method described above.

[0122] Those skilled in the art will appreciate that embodiments of the application can be devised for a method, a system, or a computer program product. Accordingly, the present application can be embodied in the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present application can take the form of a computer program product on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROMs, optical storage devices, and the like) embodying computer readable program code.

[0123] The present application is described in reference to the flowchart and / or block diagrams of the method, apparatus (system) and computer program product according to this application. It will be understood that each block of the flowchart and / or block diagrams, and combinations of blocks in the flowchart and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart and / or block diagrams block or blocks. Figure 1 one or more functions specified in the flowchart and / or block diagram block or blocks. Figure 1 one or more functions specified in the flowchart and / or block diagram block or blocks.

[0124] These computer program instructions can also be stored in a computer- readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instructions which implement the flowchart and / or block diagrams block or blocks. Figure 1 one or more functions specified in the flowchart and / or block diagram block or blocks. Figure 1 one or more functions specified in the flowchart and / or block diagram block or blocks.

[0125] These computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the flowchart and / or block diagrams block or blocks. Figure 1 one or more functions specified in the flowchart and / or block diagram block or blocks. Figure 1 one or more functions specified in the flowchart and / or block diagram block or blocks.

[0126] Obviously, numerous modifications and variations of the present application are possible in light of the above teachings. It is therefore to be understood that within the scope of the apparent to those skilled in the art that various modifications and changes can be made thereto without departing from the scope and spirit of the present application. It is intended that the scope of the present application be limited only by the appended claims, and their equivalents.

Claims

1. A tensor segmentation method, characterized in that, include: Obtain the input tensor and the target segmentation type corresponding to the input tensor; From multiple kernel functions corresponding to the task operator, a target kernel function that matches the target segmentation type is selected, wherein the multiple kernel functions correspond to multiple segmentation types; When the target segmentation type is a non-uniform segmentation type, the input tensor is segmented into multiple sub-tensors by executing the target kernel function, wherein the multiple sub-tensors have different shapes and values; The multiple sub-tensors are respectively scheduled to multiple bare dies in the artificial intelligence chip for processing to obtain the output tensor.

2. The method as described in claim 1, characterized in that, Also includes: When the target segmentation type is a non-uniform segmentation type, the output tensor corresponds to the non-uniform segmentation type.

3. The method as described in claim 1, characterized in that, The step of scheduling the multiple sub-tensors to multiple dies in the artificial intelligence chip for processing to obtain the output tensor includes: Through the extended interface corresponding to the non-uniform distribution type, the multiple sub-tensors are respectively scheduled to multiple bare dies in the artificial intelligence chip for processing to obtain the output tensor.

4. The method as described in claim 3, characterized in that, The step of scheduling the multiple sub-tensors to multiple dies in the artificial intelligence chip for processing through the extended interface corresponding to the non-uniform distribution type to obtain the output tensor includes: Through the extended interface and communication operator corresponding to the non-uniform division type, the multiple sub-tensors are respectively scheduled to multiple bare dies in the artificial intelligence chip for processing to obtain the output tensor.

5. The method as described in claim 1, characterized in that, Also includes: When the target segmentation type is equal segmentation, the input tensor is segmented into multiple sub-tensors by executing the target kernel function, wherein the multiple sub-tensors have the same shape; The multiple sub-tensors are respectively scheduled to multiple dies in the artificial intelligence chip for processing through a general interface to obtain the output tensor.

6. The method as described in claim 1, characterized in that, Also includes: When the target segmentation type is broadcast, multiple sub-tensors are obtained by copying the input tensor by executing the target kernel function, wherein the multiple sub-tensors have the same shape and value; The multiple sub-tensors are respectively scheduled to multiple dies in the artificial intelligence chip for processing through a general interface to obtain the output tensor.

7. The method as described in claim 1, characterized in that, Also includes: When the target segmentation type is a partial type, the input tensor is segmented into multiple sub-tensors by executing the target kernel function, wherein the multiple sub-tensors have the same shape; The multiple sub-tensors are respectively scheduled to multiple dies in the artificial intelligence chip for processing through a general interface to obtain the output tensor.

8. A computer device comprising a memory, an artificial intelligence chip, and a computer program stored in the memory and running on the artificial intelligence chip, characterized in that, When the artificial intelligence chip executes the computer program, it implements the steps of the method as described in any one of claims 1 to 7.

9. A computer-readable storage medium, characterized in that, It stores a computer program that is executed by a computer device, which, when run on the computer device, causes the computer device to perform the steps of the method as described in any one of claims 1 to 7.

10. A computer program product, characterized in that, The computer program product includes a computer program stored on a computer-readable storage medium, the computer program including program instructions that, when executed by a computer device, cause the computer device to perform the steps of the method as described in any one of claims 1-7.

Images