A method for storing the output data of a hardware accelerator in a memory, a method for reading the input data of a hardware accelerator from a memory, and a hardware accelerator thereof.

Abstract

The present invention is a data processing method that, when it is determined that a preferred order should be set for the first dimension of the input array before the second dimension, and the elements are sequentially input into the data processing unit, sets a preferred order for the first dimension before the second dimension and sequentially reads the plurality of uncompressed data groups or the plurality of compressed data groups corresponding to each of the uncompressed data groups from the memory.

Description

Technical Field

[0001] This invention relates to computer technology, and specifically to techniques for effectively managing the input and output data of hardware accelerators. Background Technology

[0002] The present invention will be described starting with an example of the structure of a neural network accelerator, which is one type of hardware accelerator to which the present invention is subject.

[0003] <Neural Networks>

[0004] Neural networks are a well-known technology that embodies artificial intelligence.

[0005] Figure 1 This is a conceptual diagram illustrating a portion of the neural network structure presented to aid in understanding the present invention.

[0006] A neural network (600) according to one embodiment may include multiple layers. Conceptually, a first layer (610) of the multiple layers may output output data (611) referred to as a feature map or activation. Moreover, the output data (611) output from the first layer (610) may be provided as input data to a second layer (620) that is lower than the first layer (610).

[0007] Each of the aforementioned layers can be viewed as a data conversion functional module or data processing unit for converting input data into predetermined output data. For example, the first layer (610) can be viewed as a data conversion functional module for converting input data (609) input to the first layer (610) into output data (611). To implement such a data conversion functional module, the structure of the first layer (610) needs to be defined. Based on the structure of the first layer (610), input variables for storing the input data (609) input to the first layer (610) need to be defined, and output variables representing the output data (611) output from the first layer (610) need to be defined. The first layer (610) can use a set of weights (612) to perform its function. The set of weights (612) can be the value multiplied by the input variable in order to calculate the output variable from the input variable. The set of weights (612) can be one of various parameters of the neural network (600).

[0008] In a neural network (600), for example, the process of calculating the output data (611) from the input data (609) to the first layer (610) can be implemented by software or by hardware.

[0009] <Hardware Implementation of Neural Network Operations>

[0010] Figure 2 The diagram illustrates some key structures in a neural network computing device, including a neural network accelerator that implements neural network functionality as hardware, and a computer device that includes the same.

[0011] The computer device (1) may include DRAM (Dynamic Random Access Memory) (10), neural network computing device (100), bus (700) connecting DRAM (10) and neural network computing device (100), and other hardware (99) connected to bus (700).

[0012] In addition, the computer device (1) may also include a power supply unit, a communication unit, a main processor, a user interface, a memory unit, and a peripheral device unit (not shown). The bus (700) may be shared by the neural network computing device (100) and other hardware (99).

[0013] The neural network computing device (100) may include a DMA unit (20), a control unit (40), an internal memory (30), a compression unit (620), a decoding unit (630), and a neural network acceleration unit (60).

[0014] In this specification, decoding may be referred to as decompression. Therefore, decoding can be expressed as decoding or decompress.

[0015] Furthermore, in this specification, compression may be referred to as encoding. Therefore, compression can be translated as either compress or encoding.

[0016] In order for the neural network accelerator (60) to operate, the input array (310) should be provided as the input data for the neural network accelerator (60).

[0017] The input array (310) can be a collection of data in the form of a multidimensional array. For example, the input array (310) may include... Figure 1 The input data (609) and a set of weights (612) are shown in the diagram. In this specification, the input array may be referred to as input data.

[0018] The input array (310) provided to the neural network acceleration unit (60) can be output from the internal memory (30).

[0019] The internal memory (30) can receive at least a portion or all of the input array (310) from the DRAM (10) via the bus (700). At this time, in order to move the data stored in the DRAM (10) to the internal memory (30), the control unit (40) and the DMA unit (20) can control the internal memory (30) and the DRAM (10).

[0020] If the neural network acceleration unit (60) operates, the output array (330) can be generated based on the input array (310).

[0021] The output array (330) can be a collection of data in the form of a multidimensional array. In this specification, the output array may be referred to as the output data.

[0022] The generated output array (330) can be preferentially stored in the internal memory (30).

[0023] Under the control of the control unit (40) and the DMA unit (20), the output array (330) stored in the internal memory (30) can be stored in the DRAM (10).

[0024] The control unit (40) can control the operation of the DMA unit (20), the internal memory (30), and the neural network acceleration unit (60).

[0025] In one embodiment, the neural network acceleration unit (60) can be executed during a first time interval. Figure 1 The first layer (610) shown has the function and can be executed during the second time interval. Figure 1 The function of the second layer (620) shown.

[0026] In one embodiment, Figure 2 The neural network acceleration unit (60) shown can be provided in multiple ways, each performing the operations requested by the control unit (40) in parallel.

[0027] In one embodiment, the neural network acceleration unit (60) can output all the data of the output array (330) sequentially over time in a given order, instead of outputting them all at once.

[0028] The compression unit (620) can compress the output array (330) and provide it to the internal memory (30) to reduce the amount of data in the output array (330). As a result, the output array (330) can be stored in the DRAM (10) in a compressed state.

[0029] The input array (310) input to the neural network acceleration unit (60) can be read from the DRAM (10). The data read from the DRAM (10) can be compressed, and the compressed data can be decoded by the decoding unit (630) and converted into the input array (310) before being provided to the neural network acceleration unit (60).

[0030] <Features of neural network operations using hardware>

[0031] Preferably, in Figure 2During the first time interval, when the neural network acceleration unit (60) performs a computational operation to generate an output array (330) from the input array (310), the internal memory (30) retrieves new data from the DRAM (10).

[0032] That is, for example, the neural network acceleration unit (60) can receive during the first time interval. Figure 1 The input array (609) and the weighted values ​​(612) of the first group are used to perform the function of the first layer (610).

[0033] Furthermore, the neural network acceleration unit (60) can receive signals during the second time interval. Figure 1 The second layer (620) functions by using the input array (611) and the weighted values ​​(622) of the second group.

[0034] At this time, it is preferable that during the period when the neural network acceleration unit (60) performs the function of the first layer (610), the internal memory (30) obtains the input array (611) and the weighted value (622) of the second group from the DRAM (10).

[0035] Figure 3 It shows Figure 2 The output array is structured as 330.

[0036] The output array (330) can be a collection of data with a multidimensional structure. Figure 3 For ease of explanation, the following description uses data with a two-dimensional structure as an example, but the concept of the present invention described below can also be applied to cases where the output array (330) has a three-dimensional or higher structure.

[0037] In one embodiment, the output array (330) is divided into and defined multiple uncompressed data groups (NCGs). The first uncompressed data group can be initially recorded in the internal memory (30), then moved to the DRAM (10), and subsequently deleted from the internal memory (30). Similarly, the second uncompressed data group in the output array (330) can be initially recorded in the internal memory (30), then moved to the DRAM (10), and subsequently deleted from the internal memory (30). This method can be used, for example, when the size of the internal memory (30) is not large enough to store all sets of output arrays (330).

[0038] Furthermore, when any uncompressed data group (NCG) of the output array (330) is recorded in the internal memory (30), the resulting compressed data group (CG) can be recorded in the internal memory (30) by first compressing the arbitrary uncompressed data group (NCG), instead of directly recording the arbitrary uncompressed data group (NCG). Then, the compressed data group (CG) recorded in the internal memory (30) can be moved to the DRAM (10).

[0039] In order to generate the compressed data groups by compressing the uncompressed data groups, it is possible to provide Figure 2 A separate data buffer is not shown in the diagram.

[0040] Figure 4 This is a schematic diagram illustrating some of the constraints considered in this invention as constraints that may occur in some embodiments.

[0041] Reference Figure 4 In (a), during the first time interval (T1), the neural network acceleration unit (60) can execute the function of layer k (610). At this time, the neural network acceleration unit (60) can output an output array (330), which can be arranged according to... Figure 4 The order of indices 1, 2, 3, 4, ..., 15, 16 shown (refer to...) Figure 4 The Z-shaped arrow of (a) outputs the elements of the output data (330) at a time, rather than outputting the elements of the output data (330) all at once. The output array (330) output in the first time interval (T1) can be stored in DRAM (10).

[0042] Moreover, refer to Figure 4 (b) In the second time interval (T2), the neural network acceleration unit (60) can execute the function of layer k+1 (610). For this purpose, the neural network acceleration unit (60) can request the output array (330) recorded in DRAM (10) as input data. At this time, the neural network acceleration unit (60) may need to follow the order of indices 1, 5, 9, 13, 2, 6, 10, 14, 3, 7, 11, 15, 4, 8, 12, 16 (refer to...) Figure 4 The Z-shaped arrow of (b) receives the constraints of the constituent elements of the output array (330).

[0043] Under these constraints, if the data processing and input / output are not properly designed, problems such as increased data processing time and reduced utilization efficiency of internal memory (30) may occur. (Refer to...) Figure 4 This problem is illustrated in Figures 5A to 5C, which will be described later.

[0044] Figures 5A to 5C are schematic diagrams illustrating a problem with a data processing method according to one embodiment.

[0045] Referring to FIG5A, the neural network acceleration unit (60) can execute the function of layer k (610) during the first time interval (T1). At this time, the neural network acceleration unit (60) can sequentially output the constituent elements corresponding to indices 1 to 8 of the first uncompressed data group (NCG1) in the output array (330) to complete the output of the first uncompressed data group (NCG1). Then, the compression unit (620) can generate a first compressed data group (CG1) from the compressed first uncompressed data group (NCG1). The first compressed data group (CG1) can be temporarily stored in the internal memory (30) and then moved to the DRAM (10).

[0046] The compression unit (620) may include a data buffer.

[0047] Next, referring to FIG5B, during the first time interval (T1), the neural network acceleration unit (60) can sequentially output the constituent elements corresponding to indices 9 to 16 of the output array (330) belonging to the second uncompressed data group (NCG2) to complete the output of the second uncompressed data group (NCG2). Then, the compression unit (620) can generate a second compressed data group (CG2) from the compressed second uncompressed data group (NCG2). The second compressed data group (CG2) can be temporarily stored in the internal memory (30) and moved to the DRAM (10).

[0048] Referring to Figure 5C, the neural network acceleration unit (60) can execute the function of layer k+1 (620) in the second time interval (T2). At this time, as described above, the neural network acceleration unit (60) executing the function of layer k+1 (620) may be subject to the constraint that it needs to be input in the order of indices 1, 5, 9, 13, 2, 6, 10, 14, 3, 7, 11, 15, 4, 8, 12, 16.

[0049] At this time, if only a portion of the first compressed data group (CG1) and a portion of the second compressed data group (CG2) are read from the DRAM (10), the data cannot be input to the neural network acceleration unit (60) using only the read data. This is because the entire first compressed data group (CG1) is needed to recover the first uncompressed data group (NCG1), and the entire second compressed data group (CG2) is needed to recover the second uncompressed data group (NCG2).

[0050] Therefore, all first compressed data groups (CG1) can first be read from DRAM (10) and stored in internal memory (30), and the first compressed data groups (CG1) stored in internal memory (30) can be restored using the decoding unit (630) to prepare the first uncompressed data group (NCG1). However, there are no elements corresponding to indices 9 and 13 in the prepared first uncompressed data group (NCG1). Therefore, after inputting the elements corresponding to indices 1 and 5 into the neural network acceleration unit (60), the elements corresponding to indices 9 and 13 cannot be input into the neural network acceleration unit (60). Therefore, in order to solve this problem, the second compressed data group (CG2) must be restored. As a result, there is a problem that after all the uncompressed data groups constituting the output array (330) are restored, data cannot be continuously input into the neural network acceleration unit (60).

[0051] In this situation, since a separate buffer is needed to store the data output from the decoding unit (630), or a predetermined space in the internal memory (30) needs to be borrowed, the efficiency of computer resource utilization is greatly reduced. In addition, there is also the problem that data read from DRAM (10) cannot be used in real time.

[0052] The foregoing description represents background knowledge known to the inventors of this invention in the course of completing this invention, and should not be considered as having been known to unspecified persons at the time of filing this patent application. Furthermore, at least a portion of the foregoing description may constitute embodiments of this invention. Summary of the Invention

[0053] The problem that the invention aims to solve

[0054] To address the aforementioned problems, this invention provides a compression method for grouping (fragmenting) elements of the output array output by the data processing unit of a hardware accelerator, and a scheduling technique for reading the grouped (fragmented) data and storing it in DRAM.

[0055] Methods for solving problems

[0056] According to one aspect of the present invention, a data processing method can be provided for a hardware accelerator (110) including a data processing unit (610) to process an input array consisting of multiple uncompressed data groups, wherein the input array is an input array (310) to be input to the data processing unit. The processing method includes: when it is determined that a preferred order needs to be set for the first dimension direction (91) of the input array before the second dimension direction (92) and the elements need to be input sequentially to the data processing unit, the hardware accelerator performs the steps of setting the preferred order for the first dimension direction before the second dimension direction and sequentially reading the multiple uncompressed data groups or multiple compressed data groups corresponding to the various uncompressed data groups from the memory (11); and if a series of elements of the input array arranged along the first dimension direction are all ready, the hardware accelerator performs the step of inputting the series of elements to the data processing unit.

[0057] At this time, the input array can be a matrix with two dimensions or an array with three or more dimensions.

[0058] In this case, each element can be information that constitutes the smallest unit of the input array. For example, when the input array is a two-dimensional matrix, each element can be data at the intersection of specific rows and specific columns of the matrix.

[0059] At this time, the reading step can be performed by reading the plurality of uncompressed data groups or the plurality of compressed data groups sequentially.

[0060] At this point, more than two data groups are defined in the input array along the second dimension.

[0061] At this point, the hardware accelerator according to claim 1 is characterized in that the plurality of uncompressed data groups or the plurality of compressed data groups constituting the input array stored in the memory can be stored in the memory in a preferred order prior to the first dimension direction of the input array, with the second dimension direction being set in the preferred order.

[0062] At this time, the input array (310) can be the output data output by the data processing unit before the acquisition step. The output data can be output by the data processing unit setting a preferred order for the second dimension direction of the output array before the first dimension direction.

[0063] At this point, the hardware accelerator according to claim 2 is characterized in that the plurality of uncompressed data groups or the plurality of compressed data groups constituting the input array stored in the memory can be stored in the memory in a preferred order prior to the second dimension direction of the input array.

[0064] At this time, the input array (310) can be the output data output by the data processing unit before the acquisition step. The output data can be output by the data processing unit setting a preferred order for the first dimension direction of the output array before the second dimension direction.

[0065] At this time, the hardware accelerator can be configured such that, when it is determined that a preferred order needs to be set for the first dimension direction (91) of the input array before the second dimension direction (92) and the elements need to be input into the data processing unit sequentially, a preferred order needs to be set for the first dimension direction before the second dimension direction, and multiple compressed data groups corresponding to the multiple uncompressed data groups constituting the input array are sequentially read from the memory (11). Moreover, the input step may include: decoding the read compressed data groups to generate the respective uncompressed data groups (NCGs) corresponding to the respective compressed data groups; and after all the generated uncompressed data groups have been prepared to form a series of elements of the input array arranged along the first dimension direction, the series of elements are input into the data processing unit.

[0066] According to another aspect of the present invention, a data processing method can be provided, comprising: in a first time interval, a data processing unit (610) of a hardware accelerator (110) outputs an output array, and sets a preferred order for the second dimension direction of the output array prior to the first dimension direction and sequentially outputs the elements of the output array; the hardware accelerator divides the output array into multiple data groups and stores them in a memory, and sets a preferred order for the second dimension direction prior to the first dimension direction and sequentially stores the multiple data groups in the memory; and in a second time interval, the hardware accelerator reads the multiple data groups stored in the memory as an input array for inputting into the data processing unit and inputs them into the data processing unit. In this case, the input step includes: the hardware accelerator setting a preferred order for the first dimension direction prior to the second dimension direction and sequentially reading the multiple groups stored in the memory; and the hardware accelerator inputting the series of elements into the data processing unit after all the elements of the input array arranged along the first dimension direction are prepared.

[0067] At this time, a group of first numbers is defined in the output array along the first dimension, and a group of second numbers is defined in the output array along the second dimension. At least some or all of the multiple groups may have the same group size. The total size of the data groups included in one column extending along the second dimension may be less than or equal to the size of the output buffer. The output buffer accommodates a portion of the output array output by the data processing unit in the first time interval. The total size of the data groups included in one column extending along the first dimension may be less than or equal to the size of the output buffer. The input buffer accommodates a portion of the input array received by the data processing unit in the second time interval.

[0068] At this time, the data processing unit can be configured to, during the second time interval, set a preferred order for the first dimension of the input array prior to the second dimension and receive the elements of the input array sequentially.

[0069] According to one aspect of the present invention, a hardware accelerator may be provided, comprising: a control unit (40); and a data processing unit (610), the data processing unit receiving and processing an input array (310) consisting of a plurality of uncompressed data groups (NCGs). In this case, the control unit is configured to, when it is determined that a preferred order needs to be set for the first dimension direction (91) of the input array prior to the second dimension direction (92) and the elements need to be sequentially input into the data processing unit, set a preferred order for the first dimension direction prior to the second dimension direction and sequentially read the plurality of uncompressed data groups or the plurality of compressed data groups corresponding to the plurality of uncompressed data groups from the memory (11); and, if all elements of the input array arranged along the first dimension direction are ready, input the series of elements into the data processing unit.

[0070] At this time, the hardware accelerator may also include a decoding unit (630). Moreover, the control unit may be configured to, when it is determined that a preferred order should be set for the first dimension direction (91) of the input array before the second dimension direction (92) and the elements should be input into the data processing unit in sequence, set a preferred order for the first dimension direction before the second dimension direction and read from the memory (11) a plurality of compressed data groups corresponding to the plurality of uncompressed data groups constituting the input array in sequence. It may be configured to decode the read compressed data groups respectively to generate the respective uncompressed data groups (NCG) corresponding to the respective compressed data groups. Moreover, it may be configured to, after all the generated uncompressed data groups have been prepared to form a series of elements of the input array arranged along the first dimension direction, input the series of elements into the data processing unit.

[0071] At this point, more than two data groups are defined in the input array along the second dimension.

[0072] At this point, the hardware accelerator according to claim 1 is characterized in that the plurality of uncompressed data groups or the plurality of compressed data groups constituting the input array stored in the memory can be stored in the memory in a preferred order prior to the first dimension direction of the input array, with the second dimension direction being set in the preferred order.

[0073] At this time, the input array (310) can be the output data output by the data processing unit before the acquisition step. The output data can be output by the data processing unit setting a preferred order for the second dimension direction of the output array before the first dimension direction.

[0074] According to another aspect of the present invention, a hardware accelerator may be provided, comprising: a control unit (40); and a data processing unit (610), the data processing unit receiving and processing an input array (310) consisting of a plurality of uncompressed data groups. In this case, the data processing unit is configured to output an output array in a first time interval, and to set a preferred order for the second dimension direction of the output array prior to the first dimension direction and sequentially output the elements of the output array; the control unit is configured to divide the output array into a plurality of groups and store them in a memory, and to set a preferred order for the second dimension direction prior to the first dimension direction and sequentially store the plurality of groups in the memory; the control unit is configured to execute, in a second time interval, a step of reading the plurality of groups stored in the memory as an input array for inputting into the data processing unit and inputting them into the data processing unit; in the input step, the control unit is configured to set a preferred order for the first dimension direction prior to the second dimension direction and sequentially read the plurality of groups stored in the memory from the memory, and after all the elements of the input array configured along the first dimension direction are prepared, input the series of elements into the data processing unit.

[0075] At this time, the hardware accelerator may further include: an output buffer that accommodates a portion of the output array output by the data processing unit in the first time interval; and an input buffer that accommodates a portion of the input array received by the data processing unit in the second time interval. Furthermore, a first group of numbers is defined in the output array along the first dimension, and a second group of numbers is defined in the output array along the second dimension. At least some or all of the plurality of groups may have the same group size. The total size of the data groups included in one column extending along the second dimension may be less than or equal to the size of the output buffer, and the total size of the data groups included in one column extending along the first dimension may be less than or equal to the size of the output buffer.

[0076] According to another aspect of the present invention, a hardware accelerator (110) can be provided, comprising: a data processing unit (610); a compression unit (620); a control unit (40); and a decoding unit (630). The data processing unit (610) is configured to output a first output array (331) based on a first input array (311) input to the data processing unit (610) during a first processing time interval (T1). The first output array (331) includes N1*N2 uncompressed data groups (NCGs) having N1 and N2 segments respectively in a first dimension direction (91) and a second dimension direction (92) (where N1 is a natural number greater than 1 and N2 is a natural number greater than 2). The compression unit (620) is... The configuration is as follows: each of the uncompressed data groups (NCGs) is compressed to generate N1*N2 compressed data groups (CGs). After the N2 uncompressed data groups (NCGs) belonging to the pth entry in the first dimension direction (91) are sequentially compressed along the second dimension direction (92), the N2 uncompressed data groups (NCGs) belonging to the p+1th entry (p+1th row) in the first dimension direction (91) are sequentially compressed along the second dimension direction (92) (where p is a natural number N1 or smaller). The control unit (40) is configured to store the N1*N2 compressed data groups (CGs) in the memory unit (30, 11), and the control unit (40) is configured to, after sequentially retrieving N1 compressed data groups (CGs) belonging to the qth entry of the second dimension direction (92) from the memory unit (30, 11 along the first dimension direction (91) and providing them to the decoding unit (630), sequentially retrieve N1 compressed data groups (CGs) belonging to the qth entry of the second dimension direction (92) from the memory unit (30, 11 along the first dimension direction (91) and providing them to the decoding unit (630), sequentially retrieve N1 compressed data groups (CGs) belonging to the qth entry of the second dimension direction (92) from the memory unit (30, 11 along the first dimension direction (91) and providing them to the decoding unit (630). The N1 compressed data groups (CG) of the q+1th entry in the degree direction (92) are provided to the decoding unit (630), which is configured to decode the provided N1*N2 compressed data groups respectively to recover the N1*N2 uncompressed data groups. The data processing unit (610) is configured to output a second output array (332) based on the N1*N2 uncompressed data groups recovered by the decoding unit (630) during the second processing time interval (T2).

[0077] At this time, the data processing unit can be configured to process the first input group during the first processing time interval to output the first output array with two or more dimensions. It can be configured to output the elements of the kth entry (kth row) belonging to the first dimension direction (91) in the first output array sequentially along the second dimension direction (92), and then output the elements of the k+1th entry (k+1th row) belonging to the first dimension direction (91) sequentially along the second dimension direction (92).

[0078] The data processing unit can be configured to, during the second time interval, set a preferred order for the first dimension of the input array prior to the second dimension and receive the elements of the input array sequentially.

[0079] According to another aspect of the present invention, a hardware accelerator may be provided, comprising: a control unit (40); and a data processing unit (610). The data processing unit may be configured to output an output array in a first time interval, and to set a preferred order for the second dimension direction of the output array prior to the first dimension direction and sequentially output the elements of the output array. The control unit may be configured to divide the output array into multiple data groups and store them in a memory, and to set a preferred order for the second dimension direction prior to the first dimension direction and sequentially store the multiple data groups in the memory. The control unit may be configured to execute, in a second time interval, a step of reading the multiple data groups stored in the memory as an input array for inputting into the data processing unit and inputting them into the data processing unit. In the input step, the control unit may be configured to set a preferred order for the first dimension direction prior to the second dimension direction and sequentially read the multiple data groups stored in the memory from the memory, and after all the elements of the input array configured along the first dimension direction are prepared, input the series of elements into the data processing unit.

[0080] At this time, the hardware accelerator may further include: an output buffer that accommodates a portion of the output array output by the data processing unit in the first time interval; and an input buffer that accommodates a portion of the input array received by the data processing unit in the second time interval. In this case, the total size of the data groups included in a column extending along the second dimension can be less than or equal to the size of the output buffer, and the total size of the data groups included in a column extending along the first dimension can be less than or equal to the size of the output buffer.

[0081] According to one aspect of the present invention, a computer device including the aforementioned hardware accelerator can be provided.

[0082] Invention Effects

[0083] According to the present invention, a compression method for grouping (fragmenting) elements of the output array output by the data processing unit of a hardware accelerator is provided, as well as a scheduling technique for reading the grouped (fragmented) data and storing it in DRAM. Attached Figure Description

[0084] Figure 1This is a conceptual diagram illustrating a portion of the neural network structure presented to aid in understanding the present invention.

[0085] Figure 2 The diagram illustrates some key structures in a neural network computing device, including a neural network accelerator that implements neural network functionality as hardware, and a computer device that includes the same.

[0086] Figure 3 It shows Figure 2 The output array is structured as 330.

[0087] Figure 4 This is a schematic diagram illustrating some of the constraints considered in this invention as constraints that may occur in some embodiments.

[0088] Figures 5A to 5C are schematic diagrams illustrating a problem with a data processing method according to one embodiment.

[0089] Figure 6 A hardware accelerator according to an embodiment of the present invention and some key structures in a computer device including the accelerator are shown.

[0090] Figure 7 This is a schematic diagram illustrating how the data processing unit operates according to time, according to an embodiment of the present invention.

[0091] Figures 8A and 8B illustrate a method for grouping and compressing the output array of the data processing unit according to an embodiment of the present invention. Figure 8C is a schematic diagram illustrating the order of reading compressed data groups from the memory unit and the order of decoding compressed data groups according to an embodiment of the present invention.

[0092] Figures 9A and 9B show the order in which the output array output by the data processing unit is grouped, compressed, and stored in the memory unit according to another embodiment of the present invention, and Figure 9C shows the order in which the groups stored in the memory unit are read and decompressed (decoded).

[0093] Figures 10A, 10B, and 10C are schematic diagrams illustrating the relationship between the size of each data group and the size of the output buffer and the input buffer according to an embodiment of the invention.

[0094] Figure 11 This is a conceptual diagram illustrating the shape of the input array or output array of a neural network accelerator according to an embodiment of the present invention. Detailed Implementation

[0095] Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings. However, the present invention is not limited to the embodiments described herein, and the present invention may be embodied in many different forms. The terminology used in this specification is for the purpose of aiding in the understanding of the embodiments and is not intended to limit the present invention. Furthermore, unless otherwise specified, the singular form of the following statements includes the plural form.

[0096] Figure 6 A hardware accelerator according to an embodiment of the present invention and some key structures in a computer device including the accelerator are shown.

[0097] The computer device (1) may include a memory (11), a hardware accelerator (110), a bus (700) connecting the memory (11) and the hardware accelerator (110), and other hardware (99) connected to the bus (700).

[0098] In addition, the computer device (1) may also include a power supply unit, a communication unit, a main processor, a user interface, a memory unit, and a peripheral device unit (not shown). The bus (700) may be shared by the hardware accelerator (110) and other hardware (99).

[0099] The hardware accelerator (110) may include a DMA unit (20), a control unit (40), an internal memory (30), a compression unit (620), a decoding unit (630), a data processing unit (610), an output buffer (640), and an input buffer (650).

[0100] Although Figure 6 In this embodiment, the decoding unit (630), compression unit (620), output buffer (640), input buffer (650), and internal memory (30) are each different constituent elements. However, in the modified embodiment, the decoding unit (630), compression unit (620), output buffer (640), input buffer (650), and internal memory (30) can be configured as a single functional unit.

[0101] For example, the memory (11), the hardware accelerator (110), and the data processing unit (610) can be Figure 2 The DRAM (10), neural network computing device (100), and neural network acceleration unit (60) shown are illustrated. However, the present invention is not limited thereto.

[0102] In order for the data processing unit (610) to operate, an input array (310) should be provided to the data processing unit (610). The input array (310) may be a collection of data in the form of a multidimensional array.

[0103] The input array (310) provided to the data processing unit (610) can be output from the internal memory (30).

[0104] The internal memory (30) can receive at least a portion or all of the input array (310) from the memory (11) via the bus (700). At this time, in order to move the data stored in the memory (11) to the internal memory (30), the control unit (40) and the DMA unit (20) can control the internal memory (30) and the memory (11).

[0105] If the data processing unit (610) operates, an output array (330) can be generated based on the input array (310). The output array (330) can be a collection of data in the form of a multidimensional array.

[0106] The generated output array (330) can be preferentially stored in the internal memory (30).

[0107] Under the control of the control unit (40) and the DMA unit (20), the output array (330) stored in the internal memory (30) can be stored in the memory (11).

[0108] The control unit (40) can control the operation of the DMA unit (20), the internal memory (30), and the data processing unit (610).

[0109] In one embodiment, the data processing unit (610) may perform a first function during a first time interval and a second function during a second time interval. The second function may be different from the first function.

[0110] For example, the data processing unit (610) can perform operations during the first time interval. Figure 1 The first layer (610) shown has the function and can be executed during the second time interval. Figure 1 The function of the second layer (620) shown.

[0111] In one embodiment, Figure 6 The data processing unit (610) shown can provide multiple units, each performing the calculations requested by the control unit (40) in parallel.

[0112] In one embodiment, the data processing unit (610) can output all the data of the output array (330) sequentially over time, instead of outputting it all at once.

[0113] The compression unit (620) can compress the output array (330) and provide it to the internal memory (30) to reduce the amount of data in the output array (330). As a result, the output array (330) can be stored in the memory (11) as a compressed array (340).

[0114] The output buffer (640) can have a storage space larger than that of the output array (330). The data constituting the output array (330) can be output sequentially over time. First, only the first sub-data that is the first part of the output array (330) to be output can be stored in the output buffer (640). The first sub-data stored in the output buffer (640) can be compressed by the compression unit (620) and transferred to the memory (11). Then, the second sub-data that is the other part of the output array (330) to be output later can be transferred to the memory (11) through the same process.

[0115] The input array (310) input to the data processing unit (610) can be read from the memory (11). The data read from the memory (11) can be compressed and can be decoded by the decoding unit (630) and converted into the input array (310) before being provided to the data processing unit (610).

[0116] The input buffer (650) can have a storage space larger than that of the input array (uncompressed) (310). The data constituting the input array (compressed) (320) can be provided sequentially over time. First, only the first sub-data, which is the first part of the input array (compressed) (320), can be stored in the input buffer (650). The first sub-data stored in the input buffer (650) can be decoded by the decoding unit (630) and input to the arithmetic unit (610). Then, the second sub-data, which is another part of the input array (compressed) (320) provided later, can be input to the data arithmetic unit (610) through the same process.

[0117] Figure 7 This is a schematic diagram illustrating how the data processing unit operates according to time, according to an embodiment of the present invention.

[0118] Figures 8A and 8B illustrate a method for grouping and compressing the output array of the data processing unit according to an embodiment of the present invention.

[0119] The following will refer to Figure 6 , Figure 7 Figures 8A and 8B will be used for explanation.

[0120] The data processing unit (610) can be configured to output a first output array (331) based on a first input array (311) input to the data processing unit (610) during a first processing time interval (T1). Figure 7 ).

[0121] The first output array (331) may include N1*N2 uncompressed data groups with N1 segments in the first dimension direction (91) and N2 segments in the second dimension direction (92). However, N1 may be a natural number greater than 1 and N2 may be a natural number greater than 2 (Figure 8).

[0122] In the example shown in Figure 8A, N1 = 10 and N2 = 2. In Figure 8A, each region represented by a dashed line corresponds to an uncompressed data set.

[0123] Figure 8B illustrates the concept of uncompressed data groups more clearly by showing only the dashed lines of Figure 8A. Figure 8B shows a total of 20 uncompressed data groups (NCG101 to NCG120). Compared to Figure 8B, in Figure 8A, due to limitations in the accompanying drawings, only uncompressed data groups (NCG101 to NCG104) are given reference numerals.

[0124] In this specification, the various uncompressed data groups represented by different figure numbers may be collectively referred to as NCG (Non-Compressed data Group).

[0125] The compression unit (620) can compress each uncompressed data group (NCG) to generate N1*N2 compressed data groups (CG).

[0126] In Figure 8B, compressed data groups (CGs) are shown using reference numerals such as CG (Compressed Group), which correspond to the 20 uncompressed data groups (NCGs) shown in Figure 8B.

[0127] In this specification, the various compressed data groups represented by different figure numbers may be collectively referred to as CG (Compressed data Group).

[0128] In this specification, the NCG and CG mentioned above can be collectively referred to as data group (G).

[0129] The amount of data in any k-th uncompressed data group is greater than the amount of data in the k-th compressed data group corresponding to the k-th uncompressed data group.

[0130] In one embodiment, to begin generating the k-th compressed data group, all data from the k-th uncompressed data group can be prepared. Furthermore, to recover any k-th uncompressed data group from the k-th compressed data group, all data belonging to the k-th compressed data group may be required.

[0131] In one embodiment of the invention, after the N2 uncompressed data groups (NCGs) belonging to the pth entry (pth row of FIG8B) in the first dimension direction (91) are sequentially compressed along the second dimension direction (92), the N2 uncompressed data groups (NCGs) belonging to the p+1th entry (p+1th row of FIG8B) in the first dimension direction (91) are sequentially compressed along the second dimension direction (92). Here, p can be N1 or a smaller natural number.

[0132] That is, in one embodiment of the present invention, it can be said that the compression order of the uncompressed data groups defined in the output array output by the data processing unit (610) is preferred over the second dimension direction (92) compared to the first dimension direction (91). In FIG8B, the first processing order of compressing the uncompressed data groups is indicated by the reference numeral CO1.

[0133] In this regard, in the example shown in Figure 8B, one pth entry comprises a total of two uncompressed data groups. In Figure 8B, if we assume that the pth entry is, for example, the row to which the uncompressed data group (NCG101) and the uncompressed data group (NCG102) belong, then the compression order of the pth entry and the uncompressed data group belonging to the p+1th entry is G101, G102, G103, and G104.

[0134] Refer again Figure 6 The control unit (40) can store N1*N2 compressed data groups (CGs) in the memory unit (13). The internal memory (30) inside the hardware accelerator (110) and the external memory (11) can be collectively referred to as the memory unit (13). In this configuration, each compressed data group (CG) can actually be first stored in the memory (11) via the internal memory (30). The capacity of the internal memory (30) can be significantly smaller than that of the memory (11).

[0135] Refer again Figure 7 The data processing unit (610) can be configured to output a second output array (332) based on the second input array (312) input to the data processing unit (610) during the second processing time interval (T2).

[0136] At this time, the second input array (312) can be obtained from the N1*N2 compressed data groups (CG) stored in the memory unit (13).

[0137] Figure 8C is a schematic diagram illustrating the order of reading compressed data groups from the memory unit and the order of decoding the compressed data groups according to an embodiment of the present invention.

[0138] The control unit 40 can access the N1*N2 compressed data groups (CG) obtained from the memory unit (13) according to a second processing order (CO2) that is different from the first processing order (CO1).

[0139] The control unit (40) can be configured to sequentially obtain N1 compressed data groups (CGs) belonging to the qth entry (qth column of FIG. 8A) in the second dimension direction (92) from the memory unit (30, 11) along the first dimension direction (91) and provide them to the decoding unit (630), and then sequentially obtain N1 compressed data groups (CGs) belonging to the q+1th entry (q+1th column of FIG. 8A) in the second dimension direction (92) from the memory unit (30, 11) along the first dimension direction (91) and provide them to the decoding unit (630).

[0140] That is, regarding the second processing order, in one embodiment of the present invention, it can be said that the decoding order of the compressed data group that needs to be input to the data processing unit (610) in the second operation time interval (T2) is earlier than that in the first dimension direction (91) than in the second dimension direction (92). In other words, for the decoding order of the compressed data group, the first dimension direction (91) has a preferred order relative to the second dimension direction (92).

[0141] As described above, in one embodiment of the present invention, the compression order for uncompressed data groups / compressed data groups defined in the output array generated in the first operation time interval (T1) may be different from the decoding order for reconstruction in order to reuse the output array.

[0142] Furthermore, in one embodiment of the invention, the order in which uncompressed data groups / compressed data groups are recorded in the memory in the output array generated in the first operation time interval (T1) may be different from the order in which the uncompressed data groups / compressed data groups are read from the memory in order to reuse the output array.

[0143] The second processing order (CO2) can be related to the order in which the data processing unit (610) receives input data during the second processing time interval (T2). For example, if the data processing unit (610) is configured to set a preferred order for the first dimension direction (91) before the second dimension direction (92) and receive elements of the input array that need to be input to the data processing unit (610) during the second processing time interval (T2), then the decoding order of the compressed data group that needs to be input to the data processing unit (610) from the second processing time interval (T2) can also be a preferred order for the first dimension direction (91) before the second dimension direction (92).

[0144] Conversely, if the data processing unit (610) is configured to set a preferred order for the second dimension direction (92) before the first dimension direction (91) and receive elements of the input array that need to be input to the data processing unit (610) during the second processing time interval (T2), then the decoding order of the compressed data group that needs to be input to the data processing unit (610) from the second processing time interval (T2) can also be a preferred order for the second dimension direction (92) before the first dimension direction (91).

[0145] Therefore, the control unit 40 needs to know in advance the order in which the data processing unit (610) receives the elements of the input array that need to be input to the data processing unit (610) during the second processing time interval (T2), and can read the compressed data group from the memory unit (13) according to this order.

[0146] According to the above-described configuration of the present invention, the following effects can be obtained. That is, referring to FIG8C, if it is assumed that the first dimension direction (91) is set in a preferred order before the second dimension direction (92) and the elements of the input array to be input to the data processing unit (610) are received, then when only the uncompressed data groups (NCG101, NCG103, NCG105, NCG107, NCG109, NCG111, NCG113, NCG115, NCG117 and NCG119) are prepared, the input data required by the data processing unit (610) can be continuously and easily input. Moreover, during the input of the data to the data processing unit (610), other uncompressed data groups (NCG102, NCG104, NCG106, NCG108, NCG110, NCG112, NCG114, NCG116, NCG118 and NCG120) can be read from the memory unit (13) and prepared.

[0147] For example, as shown in FIG8C, if the compressed data group (CG101) and the compressed data group (CG102) are not divided into more than two and stored, but are compressed into a single group, then reading all the compressed data groups stored in the memory unit (13) related to the output array (331) may cause problems with the input data required by the continuous input data processing unit (610) before the decoding ends.

[0148] One of the main ideas of this invention is to compress and store data after dividing it into two or more groups in a specific dimensional direction, so that the data can be optimized according to the input requirements of the data processing unit that needs to receive the data and the data can be compressed and stored in the memory unit 13 in groups.

[0149] Furthermore, another key aspect of the present invention is that, before reading the data compressed and stored in groups from the memory unit (13), the control unit (40) that controls the data to be received obtains the input data input order of the data processing unit that needs to receive the data in advance, reads each compressed data group according to the obtained input order, and decodes each compressed data group according to the order.

[0150] The decoding unit 630 can decode the provided N1*N2 compressed data groups respectively to recover the N1*N2 uncompressed data groups. During the second operation time interval (T2), the data operation unit (610) can output a second output array (332) based on the N1*N2 uncompressed data groups recovered by the decoding unit (630).

[0151] Figures 9A and 9B show the order in which the output array output by the data processing unit is grouped, compressed, and stored in the memory unit according to another embodiment of the present invention, and Figure 9C shows the order in which the groups stored in the memory unit are read and decompressed (decoded).

[0152] Figures 9A, 9B, and 9C correspond to Figures 8A, 8B, and 8C, respectively. Therefore, the descriptions given above regarding Figures 8A, 8B, and 8C can be applied to Figures 9A, 9B, and 9C as is.

[0153] Figures 9A, 9B, and 9C are schematic diagrams illustrating various methods for grouping elements of an input array or output array according to an embodiment of the present invention, and the dimension of a data group can be freely set. However, it is preferred that multiple groups be divided along a dimension in a specific direction.

[0154] Figures 10A, 10B, and 10C are schematic diagrams illustrating the relationship between the size of each data group and the size of the output buffer and the input buffer according to an embodiment of the invention.

[0155] Referring to FIG10A, the data processing unit (610) can output an output array (330). The output array (330) can be divided into 5 data groups (=N1) along the first dimension direction (91) and into 5 data groups (=N2) along the second dimension direction (92).

[0156] Each data group (NCG) may include multiple elements along the first dimension (91) and multiple elements along the second dimension (92).

[0157] The output buffer (640) can be smaller than the total size of the output array (330).

[0158] At this time, the data processing unit (610) can sequentially output the constituent elements of the output array (330) by giving a preferred order to the second dimension direction (92). As shown in the column of data groups (OI1) in FIG10C, each constituent element of the output array (330) can be output sequentially in the order of (1) to (2). Referring to FIG10A, when the data processing unit (610) sequentially outputs each element of the output array (330), each data group can be completed starting from reference number 101 in ascending order of reference number.

[0159] In Figure 10A, when the data group (OI1) of the first row is stored in the output buffer (640), the data stored in the output buffer (640) can be transferred to the memory in data groups, or it can be compressed and transferred to the memory in data groups. Then, the data stored in the output buffer (640) can be deleted, and the above process can be repeated for the data group (OI2) of the second row.

[0160] At this point, in order to compress the elements output sequentially in the order of (1) to (2) according to the above data group, the size of the output buffer (640) must be greater than or equal to the size of the data group (OI1) of the first row (row).

[0161] Referring to FIG10B, the data processing unit (610) can receive an input array (310). The input array (310) can have the same data as the output array (330) shown in FIG10A.

[0162] The input buffer (650) can be smaller than the total size of the input array (310).

[0163] At this time, the data processing unit (610) can sequentially receive the constituent elements of the input array (330) by giving a preferred order in the first dimension direction (91). As shown in a column of data groups (II1) in FIG10C, each constituent element of the input array (310) can be input sequentially in the order of (3) to (4). Referring to FIG10B, the preferred order of each group of the input array (310) in the first dimension direction (91) is set and read from the memory, so that the data groups can be secured in the order of data group (NCG101), data group (NCG106), data group (NCG111), data group (NCG116), data group (NCG121), ...

[0164] In Figure 10B, when the data group (II1) of the first column is stored in the input buffer (650), the elements stored in the input buffer (650) can be input to the data processing unit (610). Then, the data stored in the input buffer (650) can be deleted, and the above process can be repeated for the data group (II2) of the second column.

[0165] At this point, in order to prepare the elements that need to be entered sequentially in the order of (3) to (4), the size of the input buffer (640) must be greater than or equal to the size of the data group (II1) of the first column.

[0166] Figure 10A shows an example where the data groups (NCGs) are of the same size. In a preferred embodiment, all data groups may be the same size. However, in another embodiment, data groups at the edges may have a smaller size because situations may occur where the data array is not an exact integer multiple of the input / output buffer. For example, NCGs (105), (110), (115), (120), and (125) which are the rightmost edge data groups in Figure 10A may have a smaller size than NCG (101) which is the first generated data group. Alternatively, for example, NCGs (121), (122), (123), (124), and (125) which are the bottommost edge data groups in Figure 10A may have a smaller size than NCG (101) which is the first generated data group. It is understood that the same applies to the data groups (NCGs) shown in Figure 10B.

[0167] In one embodiment of the present invention, a data group of a first number (N1) is defined in the output array along the first dimension direction, and a data group of a second number (N2) is defined in the output array along the second dimension direction.

[0168] In a preferred embodiment of the present invention, a second data group is defined within the output array along the second dimension. All of these data groups can have the same group size, and the value of the second number multiplied by the group size can be less than or equal to the size of the output buffer. That is, the total size of the data groups extending along one column of the second dimension can be less than or equal to the size of the output buffer.

[0169] Alternatively, in another embodiment of the invention, a second set of data groups is defined within the output array along the second dimension. A portion of these data groups (e.g., N²-1 data groups) may have the same group size, while the remaining data groups (e.g., the last data group provided along the second dimension) may have a group size larger than the same group size. In this case, the total size of the data groups extending along one column of the second dimension may be less than or equal to the size of the output buffer.

[0170] Furthermore, in a preferred embodiment of the present invention, a data group with a first number is defined within the output array along a first dimension. All of these data groups can have the same group size, and the value of the first number multiplied by the group size can be less than or equal to the size of the input buffer. That is, the total size of the data groups extending along one column of the first dimension can be less than or equal to the size of the input buffer.

[0171] Alternatively, in another embodiment of the invention, a first data group is defined within the output array along a first dimension. At least a portion of these data groups (e.g., N1-1 data groups) may have the same group size, while the remaining data groups (e.g., the last data group provided along the first dimension) may have a group size larger than the same group size. In this case, the total size of the data groups extending along one column of the first dimension may be less than or equal to the size of the input buffer.

[0172] Figure 11 This is a conceptual diagram illustrating the shape of the input array or output array of a neural network accelerator according to an embodiment of the present invention.

[0173] exist Figure 1 In Figure 9, the output and input arrays are shown as having two-dimensional array shapes, but the output or input array can also have a three-dimensional array shape as shown in Figure 10. Furthermore, it can also have array shapes with four or more dimensions (not shown). However, this invention can be understood to be applicable to cases where the output or input array is three-dimensional or more.

[0174] In a modified embodiment of the present invention, the following can be omitted: Figure 6 The compression unit (620) and decoding unit (630) are used. That is, uncompressed data groups can be directly stored in the memory unit 13 without undergoing a compression process.

[0175] Using the embodiments of the present invention described above, those skilled in the art can easily make various changes and modifications without departing from the essential characteristics of the present invention. The content of each claim can be incorporated into other claims not referenced, to the extent that it can be understood from this specification.

[0176] This invention was developed by OPENEDGES Technology Co., Ltd. (the project implementing agency) during a research project supported by the Ministry of Science and ICT and the Korea Research Foundation's Information and Communication Planning and Evaluation Institute, namely, the development (design) of next-generation intelligent semiconductor technology - artificial intelligence processor project, specifically the development of a multi-sensory situation prediction mobile artificial intelligence processor (project inherent number 2020-0-01310, project number 202-0-0-01310, research period 2020.04.01~2024.12.31).

Claims

1. A hardware accelerator, comprising: Control unit (40); and The data processing unit (610) receives and processes an input array (310) consisting of multiple uncompressed data groups. The control unit is configured to, when determining that it is necessary to input each element sequentially into the data processing unit by setting a preferred order for the first dimension direction (91) of the input array before the second dimension direction (92), execute the step of sequentially reading the plurality of uncompressed data groups or the plurality of compressed data groups corresponding to the plurality of uncompressed data groups from the memory (11) by setting a preferred order for the first dimension direction before the second dimension direction. If all elements of the input array configured along the first dimension are ready, then the step of inputting the series of elements into the data processing unit is performed. The plurality of uncompressed data groups or the plurality of compressed data groups constituting the input array, stored in the memory, are sequentially stored in the memory by setting a preferred order in which the second dimension of the input array takes precedence over the first dimension. The input array (310) is output data output by the data processing unit before the reception. This output data is generated by the data processing unit through setting a preferred order for the second dimension of the output array before the first dimension. In the input array, two or more data groups are defined along the second dimension.

2. The hardware accelerator according to claim 1, wherein, The hardware accelerator is configured such that, when it is determined that the elements need to be input into the data processing unit sequentially by setting a preferred order for the first dimension direction (91) of the input array before the second dimension direction (92), it sequentially reads from the memory (11) multiple compressed data groups corresponding to the multiple uncompressed data groups constituting the input array by setting a preferred order for the first dimension direction before the second dimension direction. The step of inputting the series of elements into the data processing unit includes: The steps of decoding the read compressed data groups to generate corresponding uncompressed data groups; and The step of inputting the series of elements into the data processing unit after all the elements of the input array configured along the first dimension direction from the generated uncompressed data set are prepared.

3. A hardware accelerator, comprising: Control unit (40); and Data processing unit (610); The data processing unit is configured to output an output array during a first time interval, and to set a preferred order for the second dimension of the output array before the first dimension, and to output the elements of the output array sequentially. The control unit is configured to divide the output array into multiple data groups and store them in a memory, and to set a preferred order for the second dimension direction over the first dimension direction, and to store the multiple data groups sequentially in the memory. The control unit is configured to perform the step of reading the plurality of data groups stored in the memory as an input array for inputting into the data processing unit and inputting them into the data processing unit during a second time interval; In the step of inputting the plurality of data groups stored in the memory into the data processing unit, the control unit is configured to set a preferred order for the first dimension direction before the second dimension direction, sequentially read the plurality of data groups stored in the memory, and input the series of elements into the data processing unit after all the elements of the input array arranged along the first dimension direction are prepared. The data processing unit is configured to, during the second time interval, sequentially receive elements of the input array by setting a preferred order for the first dimension of the input array before the second dimension. Two or more data groups are defined in the input array along the second dimension.

4. The hardware accelerator according to claim 3, characterized in that, The total size of the data groups included in one column extending along the second dimension is less than or equal to the size of the output buffer, wherein the output buffer accommodates a portion of the output array output by the data processing unit during the first time interval. The total size of the data groups included in a column extending along the first dimension is less than or equal to the size of the output buffer, wherein the input buffer accommodates a portion of the input array received by the data processing unit in the second time interval.

5. The hardware accelerator according to claim 3, characterized in that, Also includes: An output buffer that accommodates a portion of the output array output by the data processing unit during the first time interval; and An input buffer that accommodates a portion of the input array received by the data processing unit during the second time interval; Wherein, the total size of the data groups included in one column extending along the second dimension is less than or equal to the size of the output buffer. The total size of the data groups included in one column extending along the first dimension is less than or equal to the size of the input buffer.

6. A computer device comprising the hardware accelerator according to any one of claims 1-5.

Images