Remodeling and broadcast optimization to avoid unnecessary data movement

Abstract

The invention relates to remodeling and broadcast optimization to avoid unnecessary data movement. Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for transforming a pattern of operations on tensors in a computational graph to reduce the resulting storage burden when performing remodeling operations, particularly when deploying to a hardware platform having vector instructions or vector stores requiring alignment operators.

Description

[0001] This application is a divisional application of the invention patent application with the application date of April 30, 2020 and the application number of 202080033009.6. Background Art

[0002] The TensorFlow library can be used to create machine learning models, such as, a recurrent neural network (“RNN”) model, a convolutional neural network (“CNN”) model, a feedforward neural network model, and a random forest model. (TensorFlow is described in the following literature by Abadi et al.: “TensorFlow: A System for Large-Scale Machine Learning” (OSDI’16), pp. 265-283, November 2-4, 2016. The software is available from https: / / tensorflow.org)

[0003] The TensorFlow library can be used to represent a machine learning model as a TensorFlow graph. Each node in the TensorFlow graph represents an operation. Each edge in the TensorFlow graph is directed and represents the data flow into or out of the node to which the edge is connected. The data is in the form of a zero-dimensional or multi-dimensional tensor, where each element has the same data type, e.g., 32-bit integer, double-length floating point number, or string. A tensor is represented by a vector with paired parentheses “[]” on the outside. For example, a one-dimensional (1D) tensor (also known as a vector) with 3 elements would be represented as [1, 2, 3]. A zero-dimensional tensor is a scalar. A two-dimensional (2D) tensor would be represented as [[1, 2, 3], [4, 5, 6]]. The rank of this tensor, i.e., the number of dimensions or indices required to uniquely select each element of the tensor, is 2. The shape of this tensor is [2, 3]. The number of elements in the zero-th dimension is 2, i.e., two vectors (1D tensors) [1, 2, 3] and [4, 5, 6]; and the number of elements in the first dimension is 3; that is, each of the vectors [1, 2, 3] and [4, 5, 6] has three elements. The shape of the tensor itself is a 1D tensor. As customary in many programming environments, the dimension numbers start from zero.

[0004] In this specification, the Python API for building and executing TensorFlow graphs will be used to express examples. The TensorFlow module can be loaded as follows:

[0005] import tensorflow as tf

[0006] TensorFlow operations include shape, reshape, broadcast, and reduce operations. These will be described below, omitting from the description parameters and aspects that are not important for this specification.

[0007] When executed, the shape operation returns the shape (i.e., dimensions) of the input tensor as a 1D tensor. In the following example:

[0008] X = tf.constant([[[1, 1, 1], [2, 2, 2]], [[3, 3, 3], [4, 4, 4]]])

[0009] tf.shape(X)

[0010] The shape operation returns the tensor [2, 2, 3], which represents the dimensions of tensor X.

[0011] When executed, the reshape operation returns a tensor that has the same element values in the same order as the input tensor but has the shape defined by the shape tensor input. In the following example,

[0012] X = tf.constant([[[1, 1], [2, 2]], [[3, 3], [4, 4]]])

[0013] tf.reshape(X, [2, 4])

[0014] The reshape operation takes tensor X and a 1D tensor [2, 4] representing the desired shape as input parameters. The reshape operation returns the tensor [[1, 1, 2, 2], [3, 3, 4, 4]], which has the same elements as the input tensor X and has the desired shape, i.e., [2, 4]. The desired shape input to the reshape operation can have more or fewer dimensions than the input tensor has.

[0015] Broadcast operations include broadcast_to. Broadcasting is the process of making arrays with compatible shapes for arithmetic operations. Two shapes are compatible if for each pair of corresponding dimensions of the two shapes, the dimensions are equal or one of them is one. When a tensor is broadcast to a shape, the operation starts from the trailing dimension and proceeds forward.

[0016] Thus, when executed, the broadcast_to operation returns a tensor that is the input tensor copied as many times as needed until it reaches the specified shape requested. In the following example:

[0017] V = tf.constant([7, 8])

[0018] tf.broadcast_to(V, [2, 3])

[0019] The broadcast_to operation takes the tensor V and the tensor [2, 3] specifying the desired shape as inputs. It returns the tensor [[7, 7, 7], [8, 8, 8]] with the desired shape.

[0020] Reduction operations include reduce_all, reduce_any, reduce_sum, and reduce_mean. The output tensors returned by reduction operations usually have a lower rank and fewer elements than the input tensors.

[0021] Reduction operations take an input tensor and an axis tensor. The elements of the axis tensor identify the dimensions of the shape of the input tensor. Reduction operations reduce the input tensor along the dimensions specified by the axis tensor. For example,

[0022] X = tf.constant([[1, 1, 1], [1, 1, 1]])

[0023] In the above equation, the shape of X is [2, 3], that is, X is a tensor with two rows and three columns: 1 1 1 1 1 1

[0026] Taking the specific reduction operation reduce_sum as an example, use the axis tensor [0] to identify the rows, that is, the zero-th dimension of X,

[0027] tf.reduce_sum(x, [0])

[0028] When this operation is executed, reduce the tensor along the zero-th dimension (rows) and sum the rows [1, 1, 1] + [1, 1, 1] to return [2, 2, 2]. When the operation

[0029] tf.reduce_sum(x, [1])

[0030] is executed, reduce along the first dimension (columns) and sum the columns [1, 1] + [1, 1] + [1, 1] to return [3, 3]. When the operation

[0031] tf.reduce_sum(x, [0, 1])

[0032] is executed, reduce and sum along both dimensions to return the scalar (0D tensor) 6.

[0033] The shape of the tensor returned by the reduction operation has the dimensions of the input tensor, without the indices specified by the axis tensor.

[0034] The element values of the tensors returned by other reduction operations are computed by other operations. For example, the reduce_all operation computes a logical AND, the reduce_any operation computes a logical OR, the reduce_mean operation computes a mean, and so on.

[0035] In some scenarios, a user uses a compiler, e.g., a Just-In-Time (JIT) compiler, to compile a TensorFlow graph into a graph for input into the XLA compiler. (The JIT compiler is described at https: / / www. tensorflow. org / xla / jit). The input language of XLA is called "HLO IR", or just HLO (High-Level Optimizer). The XLA compiler takes graphs (i.e., computations) defined in HLO and compiles them into machine instructions for various architectures, performing target-dependent optimizations and generating target-dependent code.

[0036] Nodes in an HLO graph represent operations. Each edge in the graph is directed and represents the data flow into or out of the node to which the edge is connected. This data is in the form of tensors. The operations represented in the HLO graph correspond to the operations in the TensorFlow flow that generated the HLO graph. Specifically, the HLO graph can include reshape, reduction, and broadcast operations.

[0037] The binaries generated by the XLA compiler are deployed to the hardware and executed by a specific processor of the hardware. Some processors implement instructions that operate on vectors. To enable the processor to execute vector instructions that operate on tensor data, the tensors must be stored such that each of the tensor vectors to be operated on by the vector instructions is aligned on a vector boundary as specified for the processor.

[0038] For example, if a reshape operation receives as input arguments a tensor [1, 2, 3, 4, 5, 6, 7, 8, 9] and a tensor [3, 3] specifying a shape, then the resulting tensor [[1, 2, 3], [4, 5, 6], [7, 8, 9]] with its three vectors [1, 2, 3], [4, 5, 6], and [7, 8, 9] must be moved to align them on a vector boundary if the vectors are not exactly aligned on the vector boundary as required by the specific processor of the hardware. Such vector boundary alignment operations can be computationally expensive. SUMMARY OF THE INVENTION

[0039] This specification describes optimization techniques that can be implemented in an XLA compiler to reduce the memory burden of a particular sequence of operations that includes a reshape operation.

[0040] These optimizations are particularly helpful in the sequence of operations implementing machine learning techniques such as group normalization (https: / / arxiv.org / pdf / 1803.08494.pdf) and ghost batch normalization (https: / / arxiv.org / pdf / 1705.08741.pdf). In a straightforward implementation of such machine learning techniques, the input tensor is reshaped to a higher dimension. Then, a reduction is performed between some of the dimensions changed by the reshape and other dimensions not changed by the reshape. The derivative of the reduction and reshape is broadcasting, and it is reshaped back to the original shape. Broadcasting is also made between some of the dimensions changed by the reshape and some of the dimensions not changed by the reshape. The reshape operation has no effect on data on platforms with a linear address space. However, on platforms with vector stores, reshape will typically change the shape alignment relative to the vector store, requiring data to be moved. This specification describes optimizations to reduce the size of the tensors that need to be moved to achieve alignment relative to the vector store. The optimizations can be used to reduce the number of reshape operations and / or the number of vectors representing intermediate tensors in the sequence of operations, thereby reducing the number of vector boundary alignment operations that need to be performed by the processor. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 is a flowchart showing an example process of a compiler transformation that implements detecting an operation mode including a reshape operation, which can be transformed to minimize the size of the reshape operation.

[0042] Figure 2 is a flowchart showing an example process of the transformation.

[0043] Like reference numerals and names in different figures indicate the same elements. DETAILED DESCRIPTION

[0044] Figure 1 is a flowchart showing an example process 100 of a compiler transformation that implements detecting an operation mode including a reshape operation, which can be transformed to minimize the size of the reshape operation. The compiler transformation will be described with reference to the XLA compiler and the patterns of reshape, reduction, and broadcast operations in the computation graph. The compiler and the process can be implemented and executed on a system of one or more computers in one or more locations.

[0045] The process detects an operation mode (102) that includes a reshape operation that can be transformed. The mode can be an operation mode in an XLA graph. One such mode is reduce(reshape(X)), where one or more dimensions are reduced and not changed in the reshape. In this mode, a reshape operation is performed on the input tensor X, and a tensor is returned as the input to the reduce operation.

[0046] The process transforms the operation into an operation mode (104) with a smaller reshape size. For example, an operation that conforms to the reduce(reshape(X)) mode above is transformed into an operation of the reduce(reshape(reduce(X))) mode. This transformation increases the computational amount by adding an additional reduce; however, it has the important advantage of reducing the overall size of the reshape, because both computations have a reshape, and the latter reshape has dimensions that have been reduced and, as a result, is strictly fewer in number of elements than the original. The process continues to detect and transform until no additional modes are detected, at which point the compiler generates code (106) specific to the target hardware and implementing the computation including the transformed mode. Then, the compiler or other elements of the TensorFlow infrastructure deploy the generated code to the target hardware for execution (108).

[0047] An example broadcast version of another mode that can be optimized is reshape(add(reshape(X),broadcast(Y))), where the input shape of tensor X is the same as the output shape of the tensor returned by the outermost reshape. This operation mode is transformed into add(X,broadcast(reshape(broadcast(Y)))), which has overall fewer reshapes and reshapes of smaller size, because the output of the reshape is broadcast to a larger shape due to having additional dimensions. In this way, the number of vector boundary alignment operations required to implement the overall operation is reduced.

[0048] This transformation can be applied to any sub-computation that matches the following pattern:

[0049]

[0050] which is transformed by the optimization into

[0051]

[0052]

[0053] The lowercase letters f, g, h, a, and b represent mathematical operations in the graph. A depth-first search of the graph can be used to find the pattern of the transformation. In some implementations, for simplicity, the search is in post order, i.e., a topological sort where producers come before consumers, and the graph is transformed in place. When the search finds a matching subtree, it duplicates the subtree and replaces the original subtree root with the new subtree root for the user. Other compilers fix the duplication and invalid code. Other search methods for searching for pattern graphs in the computation graph can also be used.

[0054] Using these two pattern transformations - from the form reduce(reshape(X)) and reshape(f(reshape(X), broadcast(Y)) - group normalization and virtual batch normalization and their derivatives can be done with smaller reshapes and the resulting smaller storage requirements.

[0055] For example, the implementation of group normalization is naturally expressed as

[0056] reduce(reshape(image, [B, H, W, C / G, G]), [1, 2, 3]).

[0057] The shape of the above image input is [B, H, W, C], which has the batch size for a batch of images in the input and the dimensions of the height, width, and channels of the image. The number of groups is G groups. If this expression is executed in this form, it creates large intermediate tensors and slow reshapes on some hardware platforms. The above transformation improves the computation by transforming it into the following form:

[0058] reduce(reshape(reduce(image, [1, 2]), [B, C / G, G]), [1])

[0059] For the purpose of describing the transformation process, it is represented as:

[0060] Y = reshape(X, [B, H, W, C / G, G])

[0061] Z = reduce(Y, [1, 2, 3])

[0062] The reduction operation reduces the tensor Y of shape [B, H, W, C / G, G] in the dimensions specified by the axis tensor [1, 2, 3] and returns the tensor Z. The axis tensor [1, 2, 3] represents the dimensions [H, W, C / G] of the tensor Y, and the reduction operation reduces along this dimension. The reduction operation returns a tensor Z of shape [B, G].

[0063] How the parameters of the original pattern are mapped to the appropriate inputs of the transformation pattern is described below.

[0064] For the purpose of discussion, the reduce(reshape(reduce(X,[1,2]),[B,C / G,G]),[1]) transformation will be represented as:

[0065] W = reduce(X,[1,2])

[0066] Y2 = reshape(W,[B,C / G,G])

[0067] Z2 = reduce(Y2,[1])

[0068] Figure 2 is a flowchart showing an example process 200 of the transformation. This will be described with reference to the example pattern just described. Process 200 is an example implementation of the transformation (104) described above with reference to Figure 1 described.

[0069] This process determines the final output dimension (202) of the tensor returned by the original pattern of the operation. The final output dimension is determined by comparing the shape of X with the final output shape. In the example, the original pattern example receives a tensor X with a shape of [B, H, W, C] and returns a tensor with a shape of [B, G].

[0070] This process reduces along the dimensions of the reshaped input tensor that are neither in the final output nor affect the reshaping operation of the original pattern (204). In the example, according to the original pattern

[0071] Y = reshape(X,[B,H,W,C / G,G])

[0072] Z = reduce(Y,[1,2,3])

[0073] The compiler determines from the axis tensor of the input to the reshaping operation that the zeroth and third indices (i.e., B and C) of the tensor X are in the final output and affect the final output respectively. Therefore, X is reduced along its first and second dimensions (i.e., the H and W dimensions):

[0074] W = reduce(X,[1,2])

[0075] This process reshapes the output tensor of the reduction operation (206). The output tensor of the reduction operation (W in the example) is reshaped into the shape of the original pattern, but without the dimensions that are not in the final output or are transformed. In the original pattern, the third dimension (i.e., C) is divided by G, and a fourth dimension (i.e., G) is added. The zeroth dimension (i.e., B) is in the final output. Therefore, the reshaping operation in the transformation reshapes the reduced tensor into [B, C / G, G]:

[0076] Y2 = reshape(W, [B, C / G, G])

[0077] This process reduces the output tensor (208) of the reshape operation along any dimension that is not in the output tensor of the original pattern. In the example, the original pattern outputs a tensor of shape [B, G]. Thus, the first index of the output of the reshape operation in the transformation is reduced, and the reduction operation returns a tensor of shape [B, G]:

[0078] Z2 = reduce(Y2, [1])

[0079] The same rule applies to transforming an original pattern of the form reshape(operator(reshape(X), broadcast(Y))) into the form operator(X, broadcast(reshape(broadcast(Y)))).

[0080] The subject matter, as well as the examples of actions and operations, described in this specification can be implemented in digital electronic circuitry, in tangibly embodied computer software or firmware, in computer hardware (including the structures disclosed in this specification and structural equivalents thereof), or in a combination of one or more of them. Embodiments of the subject matter described in this specification can be implemented as one or more computer programs, e.g., one or more modules of computer program instructions encoded on a computer program carrier for execution by, or to control the operation of, a data processing apparatus. The carrier can be a tangible non-transitory computer storage medium. Alternatively or additionally, the carrier can be an artificially generated propagated signal (e.g., a machine-generated electrical, optical, or electromagnetic signal) that is generated to encode information for transmission to a suitable receiver device for execution by the data processing apparatus. A computer storage medium can be a machine-readable storage device, a machine-readable storage substrate, a random or serial access memory device, or a combination of one or more of them, or a portion of them. A computer storage medium is not a propagated signal.

[0081] The term "data processing apparatus" includes various devices, apparatuses, and machines for processing data, including, by way of example, a programmable processor, a computer, or multiple processors or computers. A data processing apparatus can include dedicated logic circuitry, e.g., an FPGA (Field Programmable Gate Array), an ASIC (Application Specific Integrated Circuit), or a GPU (Graphics Processing Unit). In addition to the hardware, the apparatus can also include code that creates an execution environment for the computer program, e.g., code that constitutes processor firmware, a protocol stack, a database management system, an operating system, or a combination of one or more of them.

[0082] A computer program (which may also be referred to as or described as a program, software, software application, applet, module, software module, engine, script, or code) can be written in any form of programming language, including compiled or interpreted languages or declarative or procedural languages; and it can be deployed in any form, including as a stand-alone program or as a module, component, engine, subroutine, or other unit suitable for execution in a computing environment, which may include one or more computers interconnected via a data communication network in one or more locations.

[0083] A computer program can, but need not, correspond to a file in a file system. A computer program can be stored in a part of a file that holds other programs or data (e.g., one or more scripts stored in a markup language document), in a single file dedicated to the program in question, or in multiple cooperating files (e.g., files that store one or more modules, subroutines, or portions of code).

[0084] The processes and logical flows described in this specification can be performed by one or more computers executing one or more computer programs to perform operations by operating on input data and generating output. The processes and logical flows can also be performed by special-purpose logic circuitry (e.g., FPGA, ASIC, or GPU), or by a combination of special-purpose logic circuitry and one or more programmed computers.

[0085] Computers suitable for executing a computer program can be based on general or special-purpose microprocessors or both, or any other type of central processing unit. Generally, a central processing unit will receive instructions and data from a read-only memory or a random access memory or both. The basic elements of a computer are a central processing unit for executing instructions and one or more memory devices for storing instructions and data. The central processing unit and the memory can be supplemented by, or incorporated in, special-purpose logic circuitry.

[0086] Generally, a computer will also include one or more mass storage devices, or be operatively coupled to them, to receive data from or send data to them. The mass storage devices can be, for example, magnetic disks, magneto-optical disks, or optical disks, or solid-state drives. However, a computer need not have such devices. In addition, a computer can be embedded in another device, such as a mobile phone, a personal digital assistant (PDA), a mobile audio or video player, a game console, a global positioning system (GPS) receiver, or a portable storage device (e.g., a universal serial bus (USB) flash drive), to name just a few.

[0087] To provide interaction with a user, embodiments of the subject matter described in this specification may be implemented on a computer or configured to communicate with a computer that has a display device (e.g., an LCD (liquid crystal display) monitor) for displaying information to the user and an input device through which the user can provide input to the computer, such as a keyboard and a pointing device (e.g., a mouse, trackball, or touchpad). Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback, such as visual feedback, auditory feedback, or tactile feedback; and the input from the user may be received in any form, including sound, voice, or tactile input. Additionally, the computer may interact with the user by sending documents to and receiving documents from the devices used by the user; for example, by sending a web page to a web browser on the user device in response to a request received from the web browser, or by interacting with an application running on the user device (e.g., a smartphone or an electronic tablet). Further, the computer may interact with the user by sending a text message or other form of message to a personal device (e.g., a smartphone running a messaging application) and receiving a response message from the user in reply.

[0088] This specification uses the term "configured to" in connection with systems, devices, and computer program components. A system of one or more computers being configured to perform particular operations or actions means that the system has software, firmware, hardware, or a combination thereof installed on it that, in operation, causes the system to perform the operations or actions. One or more computer programs being configured to perform particular operations or actions means that the one or more programs include instructions that, when executed by a data processing apparatus, cause the apparatus to perform the operations or actions. A special-purpose logic circuit being configured to perform particular operations or actions means that the circuit has electronic logic for performing the operations or actions.

[0089] Although this specification contains many specific implementation details, these should not be construed as limitations on the scope of what is claimed, but rather as descriptions of features that may be specific to particular embodiments of a particular invention. Certain features described in the context of separate embodiments in this specification may also be implemented in combination in a single embodiment. Conversely, the various features described in the context of a single embodiment may also be implemented separately in multiple embodiments or in any suitable sub-combination. Additionally, although features may be described as acting in certain combinations and even initially claimed as such, in some cases, one or more features from the claimed combination may be excluded from the combination, and the claimed may be directed to a sub-combination or a variation of the sub-combination.

[0090] Similarly, although the operations depicted in the drawings and recited in the claims are in a particular order, this should not be construed as requiring that the operations be performed in the particular order shown or in sequential order, or that all of the illustrated operations be performed to obtain the desired result. In some cases, multitasking and parallel processing may be advantageous. Additionally, the separation of various system modules and components in the above-described embodiments should not be construed as requiring such separation in all embodiments, and it should be understood that the program components and systems described can generally be integrated in a single software product or packaged into multiple software products.

[0091] Particular embodiments of the subject matter have been described. Other embodiments are within the scope of the appended claims. For example, the acts recited in the claims can be performed in a different order and still obtain the desired result. As one example, the processes described in the figures do not necessarily need the particular order or sequential order shown to obtain the desired result. In some cases, multitasking and parallel processing may be advantageous.

Claims

1. A method performed by one or more computers, the method comprising: Detecting an original pattern of operations in a computation operation graph on a tensor, wherein the original pattern of operations returns a final output tensor and takes an input tensor as input, and wherein the original pattern includes an original reshaping operation that (i) returns the original tensor and (ii) can be transformed to use less storage; Transforming the original pattern of operations into a new pattern of operations using one or more reshaping operations that return a tensor smaller than the original tensor, including: Determining the final output dimensions of the final output tensor returned by the original pattern of operations; Reducing along dimensions of the input tensor that are neither in the final output tensor nor affect the reshaping of the original reshaping operation to return a first intermediate result tensor; Reshaping the first intermediate result tensor to return a second intermediate result tensor; and Reducing the second intermediate result tensor along any dimension that is not in the dimensions of the final output tensor from the original pattern of operations; and Generating executable code specific to a target hardware platform and implementing the computation represented by the new pattern of operations.

2. The method according to claim 1, further comprising: Deploying the generated code to the target hardware platform for execution.

3. The method according to claim 1 or 2, wherein: The original reshaping pattern of operations requires data to be moved to meet alignment requirements for vector instructions or vector stores on the target hardware platform.

4. One or more non-transitory computer-readable storage media encoded with instructions that, when executed by one or more computers, cause the one or more computers to perform actions, the actions including: Detecting an original pattern of operations in a computation operation graph on a tensor, wherein the original pattern of operations returns a final output tensor and takes an input tensor as input, and wherein the original pattern includes an original reshaping operation that (i) returns the original tensor and (ii) can be transformed to use less storage; Transforming the original pattern of operations into a new pattern of operations using one or more reshaping operations that return a tensor smaller than the original tensor, including: Determining the final output dimensions of the final output tensor returned by the original pattern of operations; Reducing along dimensions of the input tensor that are neither in the final output tensor nor affect the reshaping of the original reshaping operation to return a first intermediate result tensor; Reshaping the first intermediate result tensor to return a second intermediate result tensor; and Reducing the second intermediate result tensor along any dimension that is not in the dimensions of the final output tensor from the original pattern of operations; and Generating executable code specific to a target hardware platform and implementing the computation represented by the new pattern of operations. The actions further include:

5. The non-transitory computer-readable storage medium according to claim 4, wherein, Deploying the generated code to the target hardware platform for execution.

6. The non-transitory computer-readable storage medium according to claim 4 or 5, wherein: ​ The original reshaping pattern of the operation requires data to be moved to meet the alignment requirements for vector instructions or vector stores on the target hardware platform.

7. A system for transforming the pattern of an operation on a tensor in a computation graph, comprising: One or more computers and one or more storage devices storing instructions thereon, which when executed by the one or more computers, are operable to cause the one or more computers to perform actions, the actions including: Detecting an original pattern of an operation in a computational operation graph on a tensor, wherein the original pattern of the operation returns a final output tensor and takes an input tensor as input, wherein the original pattern includes an original reshaping operation that (i) returns an original tensor and (ii) can be transformed to use less storage; Transforming the original pattern of the operation into a new pattern of the operation using one or more reshaping operations that return a tensor smaller than the original tensor, including: Determining the final output dimensions of the final output tensor returned by the original pattern of the operation, Reducing along the dimensions of the input tensor that are neither in the final output tensor nor affect the reshaping of the original reshaping operation to return a first intermediate result tensor, Reshaping the first intermediate result tensor to return a second intermediate result tensor, and Reducing the second intermediate result tensor along any dimension not in the dimensions of the final output tensor from the original pattern of the operation; and Generating executable code specific to the target hardware platform and implementing the computation represented by the new pattern of the operation.

8. The system according to claim 7, wherein, The actions further include: Deploying the generated code to the target hardware platform for execution.

9. The system according to claim 7 or 8, wherein: The original reshaping pattern of the operation requires data to be moved to meet the alignment requirements for vector instructions or vector stores on the target hardware platform.

Images