Padding input data for artificial intelligence accelerators

By using SAMV instructions in the AI ​​accelerator to fill in invalid elements in mask mode, the problem of low data transmission efficiency in AI accelerator is solved, real-time performance improvement and data effectiveness maintenance are achieved.

CN119013653APending Publication Date: 2024-11-22INTERNATIONAL BUSINESS MACHINE CORPORATION
View PDF 0 Cites 1 Cited by

Patent Information

Application Number
CN202380021115.6
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Priority Date
2022-02-23
Filing Date
2023-02-20
Publication Date
2024-11-22

AI Technical Summary

Technical Problem

When existing AI accelerators process input data, they cannot efficiently fill in invalid elements, resulting in low data transmission efficiency and difficult for the existing technology to achieve immediate performance improvement.

Method used

Use fill instruction (SAMV instruction) to replace invalid elements in mask mode, export the mask mode through the fill value in the register file, and apply this mode in subsequent loading instructions to realize instant filling of the data stick and avoid preprocessing steps.

Benefits of technology

Improves data transmission efficiency, reduces instruction imprinting, achieves instant performance improvement, and supports various element sizes and layered rod layouts, reducing complexity and area occupation.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119013653A_ABST
    Figure CN119013653A_ABST
Patent Text Reader

Abstract

Processing the input data for delivery to a data consumer, such as an artificial intelligence engine, is performed by arranging the input data into a unified structure consisting of data bars that are combined to form bar pages. The rod is any set of input data elements of appropriate size, whereby the size of the rod is fixed. A mask pattern is established for data bars having certain ranges of invalid data for consuming a portion of the bars while maintaining validity of the transmitted input data. A mask pattern is derived based on a Setup Effective Mask and Value (SAMV) instruction. The derived mask pattern is to be forwarded for sending subsequent load instructions to the data consumer.
Need to check novelty before this filing date? Find Prior Art

Description

Background Art

[0001] The present invention relates generally to the field of hardware accelerators, and more particularly to accelerating artificial intelligence (AI) and machine learning.

[0002] AI accelerators are a class of specialized hardware accelerators designed to accelerate AI and machine learning applications through high-performance parallel computing machinery that is specifically designed to efficiently process AI workloads such as artificial neural networks and machine learning. Typical applications include algorithms for robotics, the Internet of Things, and other data-intensive or sensor-driven tasks.

[0003] Artificial intelligence accelerators are usually present in computing systems together with processors and memory subsystems. They can be integrated on the same chip or attached to the processor system through external I / O interfaces or remotely attached in cloud computing environments. Summary of the invention

[0004] In one aspect of the present invention, a method, computer program product, and system for filling data when loading data from a memory to a data consumer include: the load instruction includes a fill instruction, the load instruction includes a read instruction, the read instruction is used to read input data arranged in a stick layout from the memory to the data consumer, the fill instruction includes a replacement value for a mask element and a fill parameter; derive a mask pattern from the parameters specified in the fill instruction; fill the input data by masking invalid elements of the stick layout; and generate a set of data sticks including fill data for the data consumer. Immediate application of fill from the load instruction via the fill instruction has a large performance benefit by eliminating any pre-processing steps. The parameters specified in the fill instruction include layout parameters and fill range parameters. The stick layout can be divided into a predefined number of slices, and masking invalid elements includes: identifying invalid slices in the stick, the invalid slices including invalid elements, and determining a starting slice having a portion of valid elements and a portion of invalid elements. The fill instruction provides a specified data format in a 6-bit instruction, including a data format, a cross-slice dimension type, and a key dimension of the stick.

[0005] In another aspect of the invention, a method, computer program product, and system for populating data while loading data from a memory to a data consumer includes transmitting a set of data sticks to the data consumer in response to a load instruction.

[0006] In yet another aspect of the present invention, a method, computer program product, and system for padding data when loading data from a memory to a data consumer includes decoding the padding instruction to determine the replacement value and the mask pattern.

[0007] In yet another aspect of the invention, a method, computer program product, and system include: decoding a fill instruction to determine a replacement fill value and fill parameters for a mask pattern for a set of sticked input data, the fill instruction being embedded in a first load instruction for the sticked input data; deriving the mask pattern from the fill parameters; generating a set of data sticks for a requesting consumer from the set of sticked input data, the set of data sticks being filled according to the derived mask pattern including the replacement fill value; and applying the derived mask pattern and associated replacement fill value to a subsequent load instruction to load subsequent input data, the subsequent load instruction being equivalent to the first load instruction, the subsequent input data being filled during transmission to the requesting consumer. The set of sticked input data is arranged as sticks, each stick being separable into a predetermined number of slices. The replacement fill value is determined with reference to a fill value register file.

[0008] In yet another aspect of the invention, a method, computer program product, and system include: filling the set of sticked input data by masking invalid elements of each stick to generate the set of data sticks, masking the invalid elements of the sticks includes: identifying invalid slices in the stick, the invalid slices including invalid elements, and determining a starting slice having a portion of valid elements and a portion of invalid elements. The set of sticked input data is arranged as sticks, each stick separable into a predetermined number of slices. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] Figure 1 is a schematic diagram of a first embodiment of a system according to the present invention;

[0010] Figure 2 is a flow chart illustrating a first method performed at least in part by the first embodiment system;

[0011] Figure 3 is a schematic diagram of the machine logic (eg, software) portion of the system of the first embodiment;

[0012] Figure 4 is a 4D tensor view showing information generated by and / or useful for understanding embodiments of the present invention;

[0013] Figure 5 is a hierarchical diagram of a 4D tensor showing information generated by and / or useful for understanding embodiments of the present invention;

[0014] Figure 6 is a schematic diagram of a second embodiment of a system according to the present invention; and

[0015] Figure 7 is a flow chart illustrating a second method performed at least in part by the second embodiment system. DETAILED DESCRIPTION

[0016] Processing input data for transmission to a data consumer such as an artificial intelligence engine is performed by arranging the input data into a unified structure that is combined by data sticks to form a data stick page. A mask pattern is established for data sticks with certain invalid data ranges to consume part of the stick while maintaining the validity of the transmitted input data. The mask pattern is derived based on a Set Valid Mask and Value (SAMV) instruction. The derived mask pattern is forwarded to the data consumer for use in a subsequent load instruction.

[0017] The present invention may be a system, method and / or computer program product. The computer program product may include a computer-readable storage medium (or multiple media) having computer-readable program instructions thereon, the computer-readable program instructions being used to cause a processor to perform various aspects of the present invention.

[0018] A computer-readable storage medium may be a tangible device capable of retaining and storing instructions used by an instruction execution device. A computer-readable storage medium may be, for example, but not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. A non-exhaustive list of more specific examples of computer-readable storage media includes the following: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a portable compact disc read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanical encoding device such as a punch card or a raised structure in a groove on which instructions are recorded, and any suitable combination of the foregoing. As used herein, a computer-readable storage medium should not be interpreted as a temporary signal itself, such as a radio wave or other freely propagating electromagnetic wave, an electromagnetic wave propagated by a waveguide or other transmission medium (e.g., a light pulse by an optical fiber cable), or an electrical signal transmitted by a wire.

[0019] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to a corresponding computing / processing device, or downloaded to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network can include copper transmission cables, optical transmission fibers, wireless transmissions, routers, firewalls, switches, gateway computers, and / or edge servers. The network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions to be stored in a computer-readable storage medium within the corresponding computing / processing device.

[0020] The computer-readable program instructions for performing the operation of the present invention can be assembly instructions, instruction set architecture (ISA) instructions, machine-related instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​(such as Smalltalk, C++, etc.) and conventional procedural programming languages ​​(such as "C" programming language or similar programming languages). The computer-readable program instructions can be executed completely on the user's computer, partially on the user's computer, executed as an independent software package, partially on the user's computer and partially on a remote computer, or completely on a remote computer or server. In the latter case, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computer (for example, using an Internet service provider through the Internet). In certain embodiments, in order to perform various aspects of the present invention, the electronic circuit including, for example, a programmable logic circuit, a field programmable gate array (FPGA) or a programmable logic array (PLA) can execute the computer-readable program instructions to personalize the electronic circuit by utilizing the state information of the computer-readable program instructions.

[0021] Various aspects of the present invention are described herein with reference to the flowchart and / or block diagram of the method, device (system) and computer program product according to embodiments of the present invention. It will be understood that each frame of the flowchart and / or block diagram and the combination of frames in the flowchart and / or block diagram can be implemented by computer-readable program instructions.

[0022] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device create a device for implementing the functions / actions specified in one or more boxes of the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium, which can guide the computer, programmable data processing device, and / or other equipment to work in a specific manner, so that the computer-readable storage medium having the instructions stored therein includes an article of manufacture, which includes instructions for implementing various aspects of the functions / actions specified in one or more boxes of the flowchart and / or block diagram.

[0023] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device, so that a series of operational steps are performed on the computer, other programmable apparatus, or other device to produce a computer-implemented process, so that the instructions executed on the computer, other programmable apparatus, or other device implement the functions / actions specified in one or more boxes of the flowchart and / or block diagram.

[0024] Flowcharts and block diagrams in the accompanying drawings show the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present invention.In this regard, each frame in the flow chart or block diagram can represent a module, segment or part of an instruction, which includes one or more executable instructions for realizing the specified logical function.In some alternative embodiments, the function noted in the frame may not occur in the order noted in the figure.For example, two frames shown continuously can actually be implemented as a step, and are performed simultaneously, substantially simultaneously, in a partially or entirely time-overlapping manner, or these frames can sometimes be performed in reverse order, depending on the function involved.It will also be noted that the combination of the frames in each frame of the block diagram and / or flow chart illustration and the block diagram and / or flow chart illustration can be implemented by a dedicated hardware-based system that performs a specified function or action or performs a combination of special-purpose hardware and computer instructions.

[0025] The present invention will now be described in detail with reference to the accompanying drawings. Figure 1 1 is a functional block diagram showing various parts of a networked computer system 100 according to one embodiment of the present invention, including: an accelerator subsystem 102; a neural network subsystem 104; an artificial intelligence engine 105; data consumer subsystems 106, 108; workload subsystems 110, 112; image recording 111; text recording 113; a communication network 114; a mask computer 200; a communication unit 202; a processor set 204; an input / output (I / O) interface set 206; a memory device 208; a persistent storage device 210; a display device 212; an external device set 214; a random access memory (RAM) device 230; a cache memory device 232; a fill engine 240; a data mover program 300; and a fill register 302.

[0026] Subsystem 102 is representative in many respects of various computer subsystems in the present invention. Accordingly, several portions of subsystem 102 will now be discussed in the following paragraphs.

[0027] Subsystem 102 may be a laptop computer, a tablet computer, a netbook computer, a personal computer (PC), a desktop computer, a personal digital assistant (PDA), a smart phone, or any programmable electronic device capable of communicating with a client subsystem via network 114. Program 300 is a collection of machine-readable instructions and / or data for creating, managing, and controlling certain software functions that will be discussed in detail below.

[0028] Subsystem 102 can communicate with other computer subsystems via network 114. Network 114 can be, for example, a local area network (LAN), a wide area network (WAN) such as the Internet, or a combination of both, and can include wired, wireless, or fiber optic connections. In general, network 114 can be any combination of connections and protocols that support communication between server and client subsystems.

[0029] Subsystem 102 is shown as a block diagram with many double arrows. These double arrows (without separate reference numerals) represent a communication structure that provides communication between the various components of subsystem 102. The communication structure can be implemented with any architecture designed to transfer data and / or control information between processors (such as microprocessors, communication and network processors, etc.), system memory, peripheral devices, and any other hardware components within the system. For example, the communication structure can be implemented at least in part with one or more buses.

[0030] Memory 208 and persistent storage 210 are computer-readable storage media. In general, memory 208 may include any suitable volatile or non-volatile computer-readable storage media. It should also be noted that now and / or in the near future: (i) (one or more) external devices 214 can provide some or all of the memory for subsystem 102; and / or (ii) devices external to subsystem 102 can provide memory for subsystem 102.

[0031] Program 300 is stored in persistent storage 210 for access and / or execution by one or more corresponding computer processors 204, typically through one or more memories of memory 208. Persistent storage 210: (i) is at least more permanent than a signal in transit; (ii) stores the program (including its soft logic and / or data) on a tangible medium (such as a magnetic or optical domain); and (iii) is much less permanent than persistent storage. Alternatively, data storage may be more permanent and / or permanent than the type of storage provided by persistent storage 210.

[0032] Program 300 may include machine-readable and executable instructions and / or substantive data (i.e., the type of data stored in a database). In this particular embodiment, persistent storage 210 includes a magnetic hard drive. To name some possible variations, persistent storage 210 may include a solid-state hard drive, a semiconductor memory device, a read-only memory (ROM), an erasable programmable read-only memory (EPROM), flash memory, or any other computer-readable storage medium capable of storing program instructions or digital information.

[0033] The media used by persistent storage 210 may also be removable. For example, a removable hard drive may be used for persistent storage 210. Other examples include optical and magnetic disks, thumb drives, and smart cards that are inserted into a drive for transfer to another computer-readable storage medium that is also part of persistent storage 210.

[0034] In these examples, communications unit 202 provides communications with other data processing systems or devices external to subsystem 102. In these examples, communications unit 202 includes one or more network interface cards. Communications unit 202 can provide communications using one or both of physical and wireless communications links. Any software modules discussed herein can be downloaded to a persistent storage device such as persistent storage device 210 via a communications unit such as communications unit 202.

[0035] The I / O interface set 206 allows data to be input and output with other devices that can be locally connected for data communication with the computer 200. For example, the I / O interface set 206 provides a connection to an external device set 214. The external device set 214 typically includes devices such as a keyboard, a keypad, a touch screen, and / or some other suitable input device. The external device set 214 can also include a portable computer-readable storage medium, such as a thumb drive, a portable optical or magnetic disk, and a memory card. Software and data (e.g., program 300) for implementing embodiments of the present invention can be stored on such a portable computer-readable storage medium. In these embodiments, the relevant software may (or may not) be loaded into the persistent storage device 210 in whole or in part via the I / O interface set 206. The I / O interface set 206 is also connected to the display device 212 in data communication.

[0036] Display device 212 provides a mechanism for displaying data to a user and may be, for example, a computer monitor or a smartphone display screen.

[0037] The programs described herein are identified based on the applications in which they are implemented in specific embodiments of the invention. However, it should be understood that any specific program terminology herein is used only for convenience, and thus the present invention should not be limited to use only in any specific application identified and / or implied by such terminology.

[0038] The data mover program 300 operates to process input data for transmission to a data consumer such as an artificial intelligence engine. The input data is arranged into a uniform structure for easy consumption, which is composed of rods that are combined to form pages. The data mover program 300 establishes mask patterns for data rods that have certain invalid data ranges. The padding process provides for the consumption of partial rods while maintaining the validity of the input data being transmitted. The data mover program derives the mask pattern for the input data during transmission to the data consumer with a specified load instruction or loader, which may include a SAMV instruction to set the valid mask and padding values. The derived mask pattern is forwarded to the data consumer for use in subsequent load instructions.

[0039] Some embodiments of the present invention recognize the following facts, potential problems and / or potential areas for improvement over the prior art state: (i).

[0040] An example of standard length input data is found in high bandwidth memory (HBM), which can only be addressed at a granularity of 128 bytes. Therefore, a 128-byte sequence of input data will be referred to as a "stick", as the term is used herein. When defining the data layout of each data structure to be configured, certain (multiple) specific dimensions make up the stick. Dividing a tensor into pages of data element sticks is referred to herein as "sticking" or, simply, arranging a tensor into data element sticks. The term "stick" as used herein can be defined as any set of input data elements of a suitable size, whereby the size of the stick is fixed. According to some embodiments of the present invention, the size of the stick is the SIMD capacity of the accelerator.

[0041] If the workload parameter (N) in a given dimension is not an ideal multiple of the stick size (S) in that dimension, a portion of the last stick needs to be padded to fill empty or invalid elements created by input data that is less than 128 bytes. The last input data stick is of interest here because the other sticks in the arrangement of input data are the full set of valid input data. Some embodiments of the present invention relate to fast filling of invalid elements in an AI accelerator. As described above, the invalid element of each input data can be found in the last stick of the sticks that include the input data.

[0042] For example, for the digital format FP16, the stick dimension for the input activations can be chosen to be the channel dimension "in". For a stick size of 128 bytes, the stick channel dimension is 64 elements, where each element is 16 bits, for a total of 128 bytes per stick. If the workload parameter N is not a multiple of the size of the stick size (128 bytes), there will be a remainder when the workload parameter is divided by 128. The remainder indicates how many elements of the last stick should be padded, since they would be invalid if not padded. For the given case, where Nin %S in >0, value 64-(N in %S in )=PE,where N in is the workload value, S in is the rod size and PE is the number of filling elements that make up the final rod of the workload.

[0043] Some embodiments of the present invention relate to an instruction format for specifying a mask pattern and selecting a fill value from a register file to replace masked elements within a stick without changing unmasked elements. An example of a suitable instruction format is referred to herein as a SAMV (SetActiveMask&Value) instruction. The fill value itself may vary depending on the specific requirements of the application, but in many cases a zero fill value is appropriate.

[0044] Figure 2 A flow chart 250 describing a first method according to the present invention is shown. Figure 3 A procedure 300 is shown for performing at least some of the method steps of flowchart 250. In the course of the following paragraphs, reference will now be made extensively to Figure 2 (for method step blocks) and Figure 3 (for software blocks) to discuss the method and the associated software.

[0045] Processing begins at step S255, where the stick module ("mod") 355 generates input data to the data sticks as a set of stickified input data. The term "stick" as discussed herein refers to a uniform portion of input data that is organized for accelerated consumption by a target application. As discussed in more detail below, an exemplary stick size is 128 bytes or 64 elements within each stick. Arranging the input data into data sticks may be referred to herein as "stickification." A specific process for stickification of input data is provided herein. When forming data sticks, the input data oftentimes does not fit exactly into a 128-byte portion, but for a given set of data, there are some remaining unused bits in a stick.

[0046] Processing proceeds to step S260, where the loading module 360 ​​receives a loading instruction or program for loading input data from the memory to the data consumer. In this example, the loading from the workload subsystem 110 includes the image record storage 111 ( Figure 1) of an image. In addition, in this example, the load instruction directs the transfer of the input data to the AI ​​engine 105 of the neural network subsystem 104. The load instruction may include: (i) an instruction to read data from the memory to the consumer; (ii) an instruction to copy an intermediate value to the register file; (iii) an instruction to "set valid mask and value" for filling (SAMV instruction); and (iv) an instruction to loop or repeat a series of instructions for a given number of cycles.

[0047] Processing proceeds to step S265, where the input module 365 identifies the bar-ized input data and fill values. Upon receiving the load instruction, the data mover program 300 continues to locate the input data and assign fill values. The input data location can be stored at a remote location or on a computer system running the fill program. For the purposes of this disclosure, the input data will be consumed by a target application such as an AI engine of a neural network or a machine learning application. The fill value is application-dependent and is provided for masking invalid elements of the length when the input data does not meet the required sequence length. In this example, the fill value is recorded to the fill register 302 for filling the identified input data. The stored fill value is applied to the mask element later.

[0048] Processing proceeds to step S270, where the decode module 370 decodes the SAMV instruction. The decoded SAMV instruction starts reading fill values ​​from registers and keeps the fill values ​​for use. The SAMV instruction also provides mask mode operations and generates a valid mask for input data read from memory.

[0049] Processing proceeds to step S275, where the pattern module 375 derives a mask pattern for the rodized input data. The mask pattern is derived from the layout and fill range parameters encoded in the bit fields of the SAMV instruction. The SAMV instruction provides controlled replacement of fill elements along the inner and outer dimensions within a rod, which can be divided into slices. A detailed discussion of mask pattern derivation is provided below.

[0050] Processing proceeds to step S280, where the mask module 380 generates a mask for filling the input data based on the mask pattern. The mask module 380 operates to generate a mask based on parameters derived from the SAMV instruction. In this example, the image records from the workload 110 will be sent to the AI ​​engine 105. The image records are arranged in a stick format and divided into slices for identifying and filling invalid elements. The mask module establishes a collection of fill data sticks in the form of input data pages for accelerated consumption by the target consumer.

[0051] Processing proceeds to step S285, where the loop module 385 applies the mask pattern to subsequent input data. In this example, a loop instruction is received at step S255, where the loop instruction indicates how to process a set of data according to the SAMV instruction to fill in invalid elements of the input data arranged into a unified data structure. In this way, a single SAMV instruction is applied to multiple mask generation cycles outside the set of first input data.

[0052] Other embodiments of the invention are discussed in the following paragraphs and referenced herein. Figure 4-7 .

[0053] Some embodiments of the invention relate to implementations of a load unit for filling input data when loading a data stick from a memory or a scratch pad. Some embodiments of the invention relate to stick layouts that support various element sizes and hierarchies.

[0054] According to some embodiments of the invention, during initialization or during run-time, values ​​to replace fill elements are stored in a register file of a load unit.

[0055] Some embodiments of the present invention relate to deriving mask patterns from layout and fill range parameters encoded in bit fields of instructions such as SAMV instructions. The operation of obtaining mask patterns provides real-time generation of mask patterns during transfer to a consumer. Cross-slice and intra-slice mask operations are used to reduce the complexity of mask decoding by using coarse-grained (cross-slice) and fine-grained (intra-slice) masks. In addition, the compact nature of the 6-bit stick data layout scheme ensures a small instruction footprint. Hierarchical layouts with inner and outer dimensions within a slice are supported by the SAMV instruction, so that replacement of fill elements can be controlled to occur along the inner and outer dimensions within a slice.

[0056] Some embodiments of the present invention are directed to a 6-bit instruction format that uses two bits for element size indication, three bits for length indication or innermost dimension, and one bit to indicate whether the innermost dimension is a cross-slice type or an intra-slice type.

[0057] According to some embodiments of the present invention, when a SAMV instruction is executed, the derived mask and replacement value are saved and applied accordingly to the fill stick by all subsequent load instructions. In this way, some embodiments of the present invention allow for on-the-fly execution of in-stick fill without any software overhead other than one "SAMV instruction" for setting the fill rules for all subsequent loads. Furthermore, because the fill region is defined in elements rather than bytes, the precision of the workload can be changed without changing the fill region definition.

[0058] Figure 4A 4D tensor 400 is shown representing input data elements having certain characteristics or dimensions, including a height, H; a width, W; a channel, channel C; and a batch size, N. In this embodiment, the values ​​shown in Table 1 are assigned to the dimensions:

[0059] Batch size, N Height, H Width, W Channel, C Element size, expired 2 3 150 120 2

[0060] Table 1. Original 4D tensor dimensions

[0061] The original tensor described in Table 1 is reconstructed into the number of pages per dimension according to the hierarchical partitioning of the data elements. Each page consists of a memory structure referred to herein as a "stick". In this example, each stick block is quantized to 128 bytes, and each page is quantized to 32 stick blocks. Therefore, most pages are full pages 532 because they are fully utilized with valid elements. The stick may correspond to the width of the data bus and / or the effective SIMD (single instruction, multiple data) width. For example: 64 elements × 2 bytes / element = 128 bytes.

[0062] Due to the fractional number, the new page structure may include partial pages 533 at the edges that are not fully utilized. In this case, only the used sticks should be loaded. This results in address gaps 534 at the edges, which need to be compensated for the data flow.

[0063] Figure 5 Show that Figure 4 The data elements of are hierarchically divided into data components including pages 531, 532, 533 and bars 536 within each page. The component dimensions QH, QW, QC and QN are derived from the 4D tensor dimensions.

[0064] The calculation of the component dimensions is shown in Table 2.

[0065]

[0066]

[0067] Table 2. Component dimensions of the segmented 4D tensor.

[0068] If the original tensor size in the stick dimension is not 128 bytes that evenly divide the stick, the new data structure resulting in the hierarchical partitioning may include empty elements at the edges. In this example, where the dimension of interest is channel C, the value of C must be evenly divided into 128 bytes to avoid having empty elements. If there are 128 elements of 2 bytes each in the example 4D tensor (C=128), there will be no empty elements in the hierarchical partitioning of the 4D tensor. The data residing at the address of the apparently empty element is invalid and should not contribute to the result. Because SIMD operations operate on the entire joystick, any invalid area needs to be filled with a suitable value from a register, such as "0", so as not to change the result value or cause exception 531. Referring again to the example, because the value of C is 120 elements, there are 16 bytes of empty elements, or 8 empty elements, and should be filled before processing.

[0069] like Figure 5 As shown, many pages are full pages 532 that are completely used for valid elements. Other pages include partial bars 531 or partial pages 533 with address gaps 534 at the edges. As discussed herein, pages with partial bars 531 are dynamically filled according to an algorithm performed during a masking operation before sending the data volume to the consumer.

[0070] Note here that the dimension order of a given input tensor can be changed to match the processing order of the accelerator. For example, the original tensor is shown in the order of NHWC in Table 1. When the tensor is reconstructed with reference to the page, the order shown in Table 2 is NCHW, which is changed from the order of Table 1.

[0071] Figure 6 A system for performing on-the-fly padding on a data path to consumer logic for single instruction multiple data (SIMD) parallel processing or use by an arithmetic logic unit (ALU) is shown. The term "on-the-fly padding" refers to performing padding or masking operations when the appropriate amount of input data is delivered to the data consumer. Figure 6 The system includes: a load unit subsystem 602; an instruction fetch module 610; an instruction buffer 612; an instruction decode module 620; a register file module 630; a fill value register 640; a mask decode module 650; a mask register 660; a memory subsystem 604; a mask operation subsystem 606; and a consumer subsystem 608.

[0072] Some embodiments of the present invention relate to a load unit subsystem including instruction fetch logic 610 and instruction buffer 612 to execute procedures associated with memory load operations. Alternatively, the instruction fetch logic is located outside the load unit subsystem and may be shared with other load unit subsystems (not shown). Decode logic 620 decodes the fetched instructions and generates control signals to register file 630, mask decode 650, and actual memory block 604.

[0073] The set of instructions to perform some embodiments of the present invention may include: (i) Load - read data from memory to a consumer; (ii) Regcopy - copy an immediate value to register file 630; (iii) SAMV (Set Active Mask Value) instruction - set an active mask and a value to use for fill; and (iv) Loop - repeat a series of instructions a given number of times.

[0074] As mentioned above, for some embodiments of the present invention, the SAMV instruction is read as a part of the instruction. The SAMV instruction specifies the mask mode and selects the fill value from the register file 630 to replace the mask element in the stick and keep the unmasked element unchanged. In addition, the fill value in the register file is pre-stored or pre-defined, and is maintained in the fill value register 640. The SAMV instruction parameterizes the mask decoding logic to generate an effective mask. The effective mask is used by the mask operation logic 606 for applying the selected fill value to the mask element of the subsequent data read from the memory 604. According to some embodiments of the present invention, the instruction decoding module and the mask decoding module are combined into a single module to decode. In addition, the memory module 604 can reside in the load unit 602.

[0075] A masking operation 606 is applied to all memory reads on the data path to the consumer module 608. The mask determines whether any elements are replaced with fill values ​​and which identified elements are to be replaced.

[0076] Figure 7 A flowchart 700 describing a second method according to an embodiment of the present invention is shown. Figure 6 A system 600 is shown for performing at least some of the method steps of flowchart 700. In the course of the following paragraphs, reference will now be made extensively to Figure 7 (for method step blocks) and FIG8 (for system blocks) to discuss the method and associated software.

[0077] Processing begins at step S702, where the data organization within the stick is obtained from the 6-bit layout field of the SAMV instruction. The stick organization specified by the SAMV instruction includes slice-specific data. Each 128-byte stick can be subdivided into 16-byte slices, where each slice consists of 2 dimensions A and B. Eight slices, X xsl= 8, forming a rod where the slices are only along a single dimension [L A ] or [L B ] Copy. Dimensions that exist across slices are cross-slice dimensions, marked as "xsl", and dimensions that exist only within a slice are intra-slice dimensions "wsl". Therefore, the data layout within a bar is limited to the following format:

[0078] {[X xsl ][L wsl ][L xsl ]}or

[0079] {[X xsl ][L xsl ][L wsl ]}.

[0080] It should be noted that in all data or number formats, whether FP16, FP8, INT4 or INT2, X xsl is always equal to 8. In addition, L wsl *L xsl =16 bytes / element length (element length is expressed in bytes / element), i.e., 8 elements in FP16, 16 elements in FP8, 32 elements in INT4, or 64 elements in INT2.

[0081] To specify the data layout in a stick form, only 6 bits are used. Two bits indicate the data format, one bit indicates whether the cross-slice dimension is innermost, [L wsl ][L xsl ] or [L xsl ][L wsl ], three bits represent the length of the cross-slice dimension or the dimension within the slice. Note that only three bits are used for the length of the slice dimension, because [L xsl ] and [L wsl ] is a power of 2, where the choices are {64, 32, 16, 8, 4, 2, 1}. Therefore, entry 7 is invalid for the set {6, 5, 4, 3, 2, 1, 0} which represents the set {64, 32, 16, 8, 4, 2, 1}.

[0082] This will be referenced when developing subsequent mask generation algorithms. Figure 5 The necessary values ​​are provided by the SAMV instruction as input values ​​for filling.

[0083] X xsl =sticksz / 128=8,

[0084] elsz=2 1-layout[5:4] ,

[0085] inner_dim=(layout[3]==0)? xsl:wsl

[0086] L wsl =2 layout[2:0] ,

[0087] L xsl =128 / (elsz*L wsl ),

[0088] Where: Sticsz is the specified stick size, for example 128 bytes, and layout refers to the 6-bit layout field in the SAMV instruction (see Table 3 below).

[0089] Parameter SliceID xsl and Valid xsl is derived from the layout parameters and the number of valid elements n in the stick. The layout parameters and the number of valid elements are provided in the SAMV instruction. The derivation is as follows:

[0090]

[0091] and

[0092] Valid xsl =(sliceID xsl ==Ceil_SliceID xsl )? L wsl ∶n%L wsl

[0093] Processing proceeds to step S704, where the slice count "s" is set to zero. The process for filling invalid elements of a stick begins by "slicing" the selected stick into a set of slices, such as eight slices in this example. Processing begins with slice 0 and ends with slice 7 of the 8-slice stick. As will be noted here, in the event that the input data is not equally divided into sticks, the last stick of input data arranged into sticks will include invalid data and should be filled. Fill locations are automatically identified according to the process described herein, so that a portion of the designated slice is marked to begin masking, which will proceed to the end of slice number 7, i.e., the 8th slice. Processing iterates through s=X over the total number of slices. xsl –1.

[0094] Processing proceeds to decision step S706, where the slice count "s" is compared to X xsl The value is compared so that when the slice count value is equal to X xsl When the value of , the mask mode for the rod in question is completed, and the mask processing follows the "no" branch, where, for example, the mask operation module 606 ( Figure 6 ) performs mask generation for subsequent masking operations.

[0095] After the "yes" branch, processing proceeds to decision step S708, where the current value of the slice count is compared with SliceID xsl The value of SliceID is compared. xsl The value of is the slice number where the mask starts. xsl When less than the slice count value, processing proceeds along the "yes" branch for the slice that needs to be fully filled or masked. xsl The value is set to all "1" (S710), indicating that the corresponding slice is masked, and the slice count is incremented by 1 (S712).

[0096] After the "No" branch, the process proceeds to decision step S714, in which Valid xsl The value of L wsl The value to be compared. xsl The value of Mask is the element number within a given slice where the mask begins. If the two values ​​are equal, processing follows the "yes" branch for the slice or elements of the slice that should be fully filled or masked. xsl The value is set to all "1" (S716), and the process proceeds to step S720.

[0097] After the "No" branch, the process proceeds to step S718, where Mask xsl The value is set to all "0"s, indicating that the slice or element in question should not be masked.

[0098] The process proceeds to decision step S720, where Valid xsl The value of is compared to zero. If Valid xsl is zero, the indicated element should not be masked, so processing follows the "yes" branch to set Mask wsl The value is set to all "0" (S722), and the slice count is incremented by 1 (S724).

[0099] After the "No" branch, processing proceeds to decision step S725, in which a decision is made based on the value of s. If s is equal to SliceID xsl , the indicated element should not be masked, so processing follows the "No" branch to set Mask wsl The value is set to all "0" (S722), and the slice count is incremented by 1 (S724).

[0100] After the "yes" branch, the process proceeds to decision step S726, where it is determined whether the internal dimension is an intra-slice dimension (wsl) or an inter-slice dimension (xsl). If the internal dimension is an intra-slice dimension, the process follows the "wsl" branch to mask the elements in the specified range in the bar. wslSet to all "1" (S728), and the remaining range of elements in the bar are Masked wsl All "0"s are set (S730). The process proceeds to step S732.

[0101] Following the "xsl" branch, processing proceeds to step S734 where the term "l" is set to a zero value.

[0102] The process proceeds to decision step S736, where the value of "1" is compared with L xsl If L xsl is not greater than zero, i.e., is set to the first value of "1", the process proceeds along the "No" branch to step S732. The term "1" is an internal dimension counter, L xsl Indicates the number of inner dimensions, where inner dimensions are cross-slicing dimensions.

[0103] In L xsl After the "yes" branch of the internal dimension count being greater than zero, the process proceeds to step S738, where for the element range that should be masked, Mask wsl is set to all "1", and processing proceeds to step S740, where for the element range that should not be masked in the slice, Mask wsl Set to all "0".

[0104] Processing proceeds to step S742, where the value of L is increased by 1.

[0105] The process returns to decision step S736, where L xsl The value of is compared with the revised value of l (from step S742), which in the first iteration will be 0+1=1. xsl If it is greater than 1, the process proceeds to step S732 along the "No" branch, otherwise the process returns to the specified element for slicing and Mask wsl The value is set to 1 (S738) and 0 (S740), and the value of l is increased by 1 each time until the value of l is greater than L xsl value, which triggers processing to proceed along the "No" branch to step S732.

[0106] Processing continues along the "No" branch to step S732, where the slice counter is incremented by 1, and processing returns to decision step S706. Processing continues following the "Yes" branch until each slice of the selected stick has been processed and the value of the slice counter is equal to X. xsl , which triggers the process to follow the “no” branch to decision step S744.

[0107] Processing continues along the "no" branch to decision step S744, where the mask all (MA) value is compared to l, and if the MA value is set to 1, processing continues along the "yes" branch to set the final mask value to all "1" (S746), which causes every element marked as 1 during mask patterning to be masked, and processing ends.

[0108] After the "No" branch goes to step S748, when the xsl and Mask wsl The processing ends when the related values ​​are logically ORed to obtain the final mask pattern.

[0109] To summarize the above process, for all 8 slices of each wand from slice 0 to slice 7, a cross-slice mask is generated in the following way. For those slices that should be fully filled, the cross-slice mask Mask xsl Set to "1" to mask SliceID across slices for all other slices. xsl Set to "0" for the slice where the mask starts, SliceID xsl , the intra-slice mask is generated as follows. If the innermost dimension is of cross-slice type, the least significant bit of the corresponding intra-slice mask portion is set to "0" and the other bits are set to "1". The least significant bit can be identified by the following formula: Bit Validity Measure = elsz*Valid xsl In the case where the innermost dimension is intra-slice, then for all cross-slice dimension sub-slices, the least significant bit of the corresponding intra-slice mask portion is set to "0" and the other bits are set to "1". For all other slices, the corresponding intra-slice mask portion is set to "0".

[0110] Some embodiments of the present invention relate to mask instruction formats under 32 bits, such as the format shown in Table 3, which provides for deriving the mask format from the information provided. Example instruction format operates to mask the most significant element of a slice starting with element N. SliceID xsl It is the slice number where the mask starts from n. Valid xsl is the SliceID where the mask starts xsl The element number within. A value of zero indicates that masking of all 8 elements is not performed. The term MA (Mask All) is set to one to mask all elements of the bar. REGILE[msk_src] is used as a 16-bit value to replace the masked element.

[0111] reserve layout msk_src Valid-Xsl SliceID-Xsl MA OPCODE 31:25 24:19 18:16 15:10 9:7 6 5:0

[0112] Table 3. SAMV mask instruction examples.

[0113] For example, referring to the digital format FP16, Xxsl =8,L xsl = 1, and the cross slice (xsl) is not the innermost slice, that is, the slice layout is {[X xsl ][L xsl ][L wsl ]}, where "xsl" refers to the cross-slice direction and "wsl" refers to the intra-slice direction. The calculation is as follows:

[0114] L xsl *L wsl =8,

[0115] L wsl =8

[0116] For a width of 43 elements:

[0117] SliceID_Xsl=43 / 8=5,

[0118] Valid_Xsl = 43%8 = 3 (zero will be interpreted as eight - no mask)

[0119] An exemplary pseudo program is as follows for loading a NCHW=1 x 1 x 64 x 43 tensor using SAMV instructions, where W=43 is the stick dimension, arranged as 64 sticks and 64 elements, with invalid elements filled with 0x3e00.

[0120] 0:regcopy(imm=0x3e00,index=0)

[0121] 1:SAMV(layout=0v110011,msk_src=0,Valid_Xsl=0b000011,SliceID_Xsl=0b101,MA=0)

[0122] 2: loop(i=0;i<64;i++)

[0123] {

[0124] 3:load(mem[i])

[0125] }

[0126] Example mask patterns generated according to the above pseudo program and an example 1x 1x 64x 43 tensor are shown in Table 4.

[0127]

[0128] Table 4. Example mask patterns.

[0129] Some embodiments of the present invention relate to using a computer system to just-in-time populate data from a set of first data into a first data volume (format, referred to as a stick) for performance and code path improvements, the computer system having at least one central processing unit (CPU), having at least one storage device associated with the at least one CPU; having at least one artificial intelligence (AI) accelerator unit (AU) associated with the at least one CPU; and having a load unit (LU) that populates the set of first data when loading the set of first data from at least one storage into the first data volume (format) (referred to as a stick). The process may include: (i) storing a set value of the first data in a register file (RF) of the LU during initialization or during runtime based on parameters associated with the set of first data; (ii) specifying a mask mode for the first data volume (format) using an instruction called SAMV (SetActiveMask&Value); (iii) selecting a set of values ​​from the register file to replace mask elements within the first data volume (format / stick); (iv) leaving the unmasked elements of the first data volume (format / stick) unchanged; (v) saving the derived mask and associated replacement set of values ​​after the SAMV instruction appears; and (vi) applying the derived mask and related replacement values ​​to fill the data in the set of first data into the first data format / stick accordingly through all subsequent equivalent load instructions.

[0130] According to some embodiments of the present invention, the mask pattern is not specified directly, but is derived from the layout and fill range parameters encoded within the bit fields of the SAMV instruction.

[0131] According to some embodiments of the present invention, combined cross-slice (XSL) and within-slice (WSL) masks are used to reduce mask decoding complexity through coarse-grained and fine-grained masks, respectively.

[0132] Some embodiments of the present invention relate to a compact 6-bit stick-shaped data layout encoding that utilizes 2 bits for element size indication, 3 bits for length indication of the innermost dimension, and 1 bit for indicating whether the innermost dimension is across slices or within a slice. This supports hierarchical layouts with inner and outer dimensions within a slice. Thus, replacement of padding elements can be controlled to occur along either the inner or outer dimension within a slice.

[0133] Some embodiments of the present invention specify mask patterns based on compact placement and range parameters in the SAMV instruction.

[0134] Some embodiments of the invention involve selecting a set of values ​​from a register file for replacing masked elements while leaving unmasked elements unchanged.

[0135] Some embodiments of the present invention involve saving a derivation mask after a SAMV instruction occurs, and applying the derivation mask and associated replacement values ​​by all subsequent equivalent load instructions to dynamically populate data.

[0136] According to some embodiments of the present invention, the input data may be padded with different values ​​based on different processing steps, e.g., for logSoftmax, in the softmax step, the padded value will be the smallest representable number, while in the log step, the padded value will be 0. (Note: The terms "LOGSOFTMAX" and / or "SOFTMAX" may have trademark rights in various jurisdictions around the world, and are used herein only with reference to products or services appropriately named by the mark, the scope of which is the scope to which such trademark rights may exist)

[0137] Some embodiments of the present invention relate to storing fill values ​​in a register file (RF).

[0138] Some embodiments of the present invention provide increased performance benefits by performing dynamic padding without the need to store padding data.

[0139] Some embodiments of the present invention may include one or more of the following features, characteristics and / or advantages: (i) adding greater flexibility in data formats and also supporting 2D padding patterns; (ii) when providing a hybrid software and hardware product, embodiments of the present invention are likely to be more area-efficient than conventional solutions; (iii) the padding process does not take additional time, improving performance; (iv) the compact layout of the encoding eliminates the need to store explicit data structures for masks in memory; (v) common mask patterns can be implicitly defined with very few bits suitable for a short instruction format; (vi) more than one padding is added in a preprocessing step and the padded vector is stored in a second memory; (vii) dynamic padding is applied when data is loaded to the consumer; (viii) more than padding based on padding replacement tables and mask tables is done; (ix) padding masks are synthesized online based on a compact layout suitable for short instructions and range parameters; (x) the application of on-the-fly padding is not limited to convolutions, but has a wider range of uses; and / or (xi) the complexity and area used to provide instructions are reduced.

[0140] Some useful definitions are:

[0141] The present invention: should not be taken as an absolute indication that the subject matter described by the term "the present invention" is covered by the claims as filed, or by claims that may ultimately issue following the patent prosecution process; while the term "the present invention" is used to help the reader gain a general sense as to what is believed to be new, as indicated by the use of the term "the present invention", that understanding is tentative and provisional and is subject to change during the patent prosecution process as relevant information develops and the claims are potentially amended.

[0142] Example: See - Similar caveats to the definition of "the present invention" above apply to the term "example."

[0143] And / or: Including the terminal values ​​or; for example, A, B and / or C means that at least one of A or B or C is true and applicable.

[0144] User / Subscriber: includes but is not necessarily limited to: (i) a single individual person; (ii) an artificial intelligence entity with sufficient intelligence to act as a user or subscriber; and / or (iii) a group of related users or subscribers.

[0145] Module / Sub-module: Any collection of hardware, firmware, and / or software that works operatively to perform a function, regardless of whether the module is: (i) in a single local proximity; (ii) distributed over a wide area; (iii) in a single proximity within a larger software code segment; (iv) located within a single software code segment; (v) located in a single storage device, memory, or medium; (vi) mechanically connected; (vii) electrically connected; and / or (viii) connected in a data communication manner.

[0146] Computer: Any device with significant data processing and / or machine-readable instruction reading capabilities, including but not limited to: desktop computers, mainframe computers, laptop computers, field programmable gate array (FPGA) based devices, smart phones, personal digital assistants (PDAs), body mounted or embedded computers, embedded device type computers, and application specific integrated circuit (ASIC) based devices.

Claims

1. A method for filling data when loading data from a memory to a data consumer, the method comprising: receiving a load instruction, the load instruction comprising a fill instruction and a read instruction for reading input data arranged in a stick layout from a memory to a data consumer, the fill instruction comprising a replacement value for a mask element and a fill parameter; deriving a mask pattern from parameters specified in said fill instruction; filling the input data by masking invalid elements of the stick layout; as well as A set of data sticks including the populated data is generated for the data consumer.

2. The method according to claim 1, wherein: The parameters specified in the fill instruction include layout parameters and fill range parameters.

3. The method according to claim 1, further comprising: In response to the load instruction, the set of data sticks is transferred to the data consumer.

4. The method according to claim 1, further comprising: The fill instruction is decoded to determine the replacement value and the mask pattern.

5. The method according to claim 1, wherein: The stick layout can be divided into a predefined number of slices, and The invalid elements of the mask include: identifying invalid slices in a stick, the invalid slices comprising invalid elements, and Determine a starting slice that has a portion of valid elements and a portion of invalid elements.

6. The method according to claim 1, wherein: The fill instruction provides the specified data format in a 6-bit instruction that includes the data format, the cross-slice dimension type, and the key dimension of the stick.

7. The method according to claim 1, wherein: The stick layout in which the input data is arranged is of a predetermined size that matches a SIMD (Single Instruction Multiple Data) capacity of an accelerator that performs the deriving and filling steps.

8. A computer program product comprising a computer readable storage medium having a set of instructions stored therein, the set of instructions, when executed by a processor, causing the processor to perform a method comprising: receiving a load instruction, the load instruction comprising a fill instruction, the load instruction comprising a read instruction for reading input data arranged in a stick layout from a memory to a data consumer, the fill instruction comprising a replacement value for a mask element and a fill parameter; deriving a mask pattern from parameters specified in said fill instruction; filling the input data by masking invalid elements of the stick layout; as well as A set of data sticks including the populated data is generated for the data consumer.

9. The computer program product according to claim 8, wherein: The parameters specified in the fill instruction include layout parameters and fill range parameters.

10. The computer program product of claim 8, further causing the processor to perform a method comprising: In response to the load instruction, the set of data sticks is transferred to the data consumer.

11. The computer program product of claim 8, further causing the processor to perform a method comprising: The fill instruction is decoded to determine the replacement value and the mask pattern.

12. The computer program product of claim 8, wherein: The stick layout can be divided into a predefined number of slices, and The invalid elements of the mask include: identifying invalid slices in a stick, the invalid slices comprising invalid elements, and Determine a starting slice that has a portion of valid elements and a portion of invalid elements.

13. The computer program product of claim 8, wherein: The fill instruction provides the specified data format in a 6-bit instruction that includes the data format, the cross-slice dimension type, and the key dimension of the stick.

14. A computer system for populating data when loading data from a memory to a data consumer, the computer system comprising: Processor collection; as well as computer readable storage medium; in: The processor set is constructed, located, connected and / or programmed to execute program instructions stored on the computer-readable storage medium; and The program instructions, when executed by the set of processors, cause the set of processors to perform a method comprising the following steps: receiving a load instruction, the load instruction comprising a fill instruction, the load instruction comprising a read instruction for reading input data arranged in a stick layout from a memory to a data consumer, the fill instruction comprising a replacement value for a mask element and a fill parameter; deriving a mask pattern from parameters specified in said fill instruction; filling the input data by masking invalid elements of the stick layout; and A set of data sticks including the populated data is generated for the data consumer.

15. The computer system of claim 14, wherein: The parameters specified in the fill instruction include layout parameters and fill range parameters.

16. The computer system of claim 14, further causing the set of processors to perform a method comprising: In response to the load instruction, the set of data sticks is transferred to the data consumer.

17. The computer system of claim 14, further causing the set of processors to perform a method comprising: The fill instruction is decoded to determine the replacement value and the mask pattern.

18. The computer system of claim 14, wherein: The stick layout can be divided into a predefined number of slices, and The invalid elements of the mask include: identifying invalid slices in a stick, the invalid slices comprising invalid elements, and Determine a starting slice that has a portion of valid elements and a portion of invalid elements.

19. The computer system of claim 14, wherein: The fill instruction provides the specified data format in a 6-bit instruction that includes the data format, the cross-slice dimension type, and the key dimension of the stick.

20. A computer-implemented method comprising: decoding a fill instruction embedded in a first load instruction for the sticked input data to determine a replacement fill value and a fill parameter for a mask pattern for a set of sticked input data; deriving a mask pattern from the padding parameters; as well as The derived mask pattern and associated replacement fill value are applied to a subsequent load instruction, which is equivalent to the first load instruction, to load subsequent input data, the subsequent input data being filled during transmission to a request consumer.

21. The method according to claim 20, wherein: The set of stick-ified input data is arranged into a plurality of sticks, each stick being divisible into a predefined number of slices.

22. The method according to claim 21, further comprising: The set of the sticked input data is filled by masking the invalid elements of each stick to generate a set of data sticks, the invalid elements of the masked sticks comprising: identifying an invalid slice in the stick, the invalid slice comprising an invalid element, and Determine a starting slice that has a portion of valid elements and a portion of invalid elements.

23. The method of claim 20, wherein: The replacement fill value is determined by reference to a fill value register file.

24. The method of claim 20, further comprising: A set of data sticks is generated for a requesting consumer from the set of sticked input data, the set of data sticks being populated according to the derived mask pattern including the replacement fill value.

Citation Information

Cited By

  • Padding input data for artificial intelligence accelerators

    US12399743B2