Memory for Performing Deep Neural Network Operations and Method of Operating the Same

Abstract

The present invention provides a memory for performing deep neural network operations and an operation method thereof. The memory includes: a processing unit and a weight unit. The processing unit has a data input end and a data output end. The weight unit is configured to be coupled to the data input end of the processing unit. The weight unit includes an index memory and a mapping table. The index memory is configured to store a plurality of weight indexes. The mapping table is configured to respectively correspond the plurality of weight indexes to a plurality of representative weight data.

Description

Technical Field

[0001] The present invention relates to a memory for performing deep neural network operations and an operation method thereof. Background Art

[0002] With the evolution of artificial intelligence (AI) operations, the application scope of AI operations has become increasingly extensive. For example, neural network operations such as image analysis, speech analysis, and natural language processing are performed via a neural network model. Therefore, various technical fields continue to invest in the research and development and application of AI, and various algorithms applicable to deep neural networks (DNNs), convolutional neural networks (CNNs), and so on are constantly emerging.

[0003] However, regardless of the algorithm used in any neural network operation, a very large amount of data is used in the hidden layer to achieve the function of machine learning. Specifically, the operation basis of a deep neural network actually comes from the matrix operation between neurons and weights. In this case, when performing deep neural network operations, a large amount of memory space is required to store weights. If the memory for storing weights exhibits a stuck-at-faults phenomenon, the operations of the deep neural network will be incorrect. Therefore, how to provide a memory and an operation method thereof that can reduce the stuck-at-faults phenomenon and improve the correct rate of deep neural network operations will become an important topic. Summary of the Invention

[0004] The present invention provides a memory applicable to performing deep neural network operations and an operation method thereof, which can find out the encoded data with the fewest stuck-at-faults to represent the mapping relationship between the weight index and the representative weight data, thereby reducing the stuck-at-faults of the index memory.

[0005] The present invention provides a memory applicable to performing deep neural network operations. The above-mentioned memory includes: a processing unit and a weight unit. The processing unit has a data input end and a data output end. The weight unit is configured to be coupled to the data input end of the processing unit. The weight unit includes an index memory and a mapping table. The index memory is configured to store a plurality of weight indexes. The mapping table is configured to respectively correspond a plurality of weight indexes to a plurality of representative weight data.

[0006] The present invention provides an operation method for a memory, which is applicable to performing deep neural network operations. The above-mentioned operation method for the memory includes a mapping method. The above-mentioned mapping method includes: coupling a weight unit to a data input end of a processing unit, where the weight unit includes an index memory storing a plurality of weight indexes and a mapping table that respectively corresponds the plurality of weight indexes to a plurality of representative weight data; detecting the index memory to generate a fault map, where the fault map includes a plurality of stuck faults; counting the number of stuck faults of the encoded data between each representative weight data and its corresponding weight index according to the fault map; and sequentially selecting the encoded data with the fewest stuck faults to establish a mapping table between the plurality of representative weight data and the plurality of weight indexes.

[0007] Based on the above, embodiments of the present invention can group a plurality of weight values into a plurality of representative weight data, and respectively correspond the plurality of weight indexes to the plurality of representative weight data through a mapping table, so as to greatly reduce the space for storing the plurality of weight values in the memory. In addition, embodiments of the present invention can generate a fault map by detecting the index memory, count the number of stuck faults of the encoded data between each representative weight data and its corresponding weight index according to the fault map, and sequentially select the encoded data with the fewest stuck faults to establish the above-mentioned mapping table. In this way, embodiments of the present invention can effectively reduce the stuck faults of the index memory, thereby improving the correct rate of deep neural network operations.

[0008] To make the above features and advantages of the present invention more obvious and understandable, the following specific embodiments are given and described in detail in conjunction with the accompanying drawings as follows. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] Figure 1 is a schematic diagram of the architecture of a memory shown according to an embodiment of the present invention;

[0010] Figure 2 is a relationship diagram between an index memory and a mapping table shown according to an embodiment of the present invention;

[0011] Figure 3 is a mapping table shown according to an embodiment of the present invention;

[0012] Figure 4 is a flowchart of an operation method for a memory shown according to an embodiment of the present invention;

[0013] Figure 5 is a fault map shown according to an embodiment of the present invention;

[0014] Figures 6A to 6C is Figure 4 a flowchart of step 404 of

[0015] Figure 7 It is a relationship table representing weight data and encoded data shown in an embodiment of the present invention. Detailed implementation manners

[0016] In order to make the content of the present invention easier to understand, the following specific embodiments are given as examples that the present invention can actually be implemented based on. Additionally, wherever possible, components / elements / steps with the same reference numerals in the drawings and implementation manners represent the same or similar components.

[0017] Please refer to Figure 1 , an embodiment of the present invention provides a memory 100 including a processing unit 110, a data input unit 120, a weight unit 130, a feedback unit 140, and a data output unit 150. Specifically, the processing unit 110 has a data input terminal 112 and a data output terminal 114. In some embodiments, the processing unit 110 may be an artificial intelligence engine, for example, a Processing In Memory (PIM) architecture or a Near Memory Processing (NMP) architecture constructed by circuit components such as control logic, arithmetic logic, and cache memory. In this embodiment, the processing unit 110 is designed to have the function of performing deep neural network operations. In this case, the memory 100 of this embodiment may be a Dynamic Random Access Memory (DRAM) chip, a resistive random access memory (RRAM), a phase-change random access memory (PCRAM), a Magnetoresistive random-access memory (MRAM), etc., but the present invention is not limited thereto.

[0018] In some embodiments, the data input unit 120 and the weight unit 130 are configured to be respectively coupled to the data input terminal 112 of the processing unit 110, and the feedback unit 140 is configured to be coupled to the data input terminal 112 and the data output terminal 114 of the processing unit 110. For example, when the processing unit 110 performs deep neural network operations, the processing unit 110 can access the operation input data (or operation input value) D1 in the data input unit 120 and the weight data 136 in the weight unit 130, and perform deep neural network operations based on the input data D1 and the weight data 136. In this embodiment, the processing unit 110 can be regarded as a hidden layer in the deep neural network, which is composed of a plurality of layers 116 connected in series, and each layer 116 has a plurality of neurons 118. When the input data D1 and the weight data 136 are operated by the processing unit 110 to obtain an operation result value R1, this operation result value R1 will be re-input into the processing unit 110 through the feedback unit 140 as a new operation input data (or operation input value) D2, so as to complete an operation of the hidden layer. And so on, until all the hidden layer calculations are completed, and the final operation result R2 of the output layer is transmitted to the data output unit 150.

[0019] It should be noted that in the prior art, the weight data is usually represented by floating point numbers and stored in the weight memory. In this case, a large amount of memory space is required to store the weight data when performing deep neural network operations. Based on this, the embodiment of the present invention uses the weight unit 130 to replace the existing weight memory, thereby reducing the storage space of the memory. Specifically, the weight unit 130 includes an index memory 132 and a mapping table 134. As Figure 2 shown, the index memory 132 is configured to store a plurality of weight indexes I0, I1, I2... I n (collectively referred to as weight index I hereinafter). The number of weight indexes I is equivalent to the number of existing weight data, which is related to the number of layers connected in series in the hidden layer and the number of neurons in each layer, and should be well known to those skilled in the art of neural networks, so it will not be elaborated here. In addition, the mapping table 134 is configured to respectively correspond a plurality of weight indexes I to a plurality of representative weight data RW0, RW1, RW2... RW k-1 (collectively referred to as representative weight data RW hereinafter). In some embodiments, a plurality of weight values (such as known weight data) can be grouped into representative weight data RW, thereby reducing the number of representative weight data RW. In this case, the weight change of the representative weight data RW can be smaller than the weight change of the weight value, so as to reduce the error rate of deep neural network operations. In addition, the number of weight indexes I can be more than the number of representative weight data RW. As Figure 2As shown, one or more weight indices I can simultaneously correspond to the same representative weight data RW.

[0020] In some embodiments, as Figure 3 shown, the mapping table 134 has a plurality of encoded data E to represent the mapping relationship between the plurality of weight indices I and the plurality of representative weight data RW. For example, as Figure 2 and Figure 3 shown, I0 in the weight index I can correspond to the representative weight value W "-0.7602" in the representative weight data RW0 through "0000" in the encoded data E. However, when a stuck-at-fault phenomenon occurs in the index memory 132 that stores the weight index I, it will still cause errors in the operations of the deep neural network. In this case, the following embodiments provide a mapping method that can find the encoded data E with the fewest stuck-at-faults to represent the mapping relationship between the weight index I and the representative weight data RW, thereby reducing the stuck-at-faults of the index memory 132.

[0021] Please refer to Figure 4 , an embodiment of the present invention provides an operation method 400 of a memory applicable to performing deep neural network operations. The operation method 400 of the memory includes a mapping method as follows. First, step 402 is performed to detect the index memory to generate an error mapping graph 500, as Figure 5 shown. In some embodiments, the error mapping graph 500 includes a plurality of stuck-at-faults 502. Herein, the so-called stuck-at-faults refer to the status level of a storage unit always being 0 or always being 1. For example, as Figure 5 shown, the status level of each storage unit storing the weight index I can be represented by four bits. Each bit position is a power of two. The status level of the storage unit storing the weight index I1 can be "X1XX", that is, the second bit position of this storage unit is always 1, and the other bit positions can be 1 or 0 (represented by X). In this case, a stuck-at-fault will occur if the encoded data "X0XX" is used to correspond to the weight index I1. Similarly, the status level of the storage unit storing the weight index I2 can be "XX11"; and the status level of the storage unit storing the weight index I3 can be "0XXX". In addition, the status level of the storage unit storing the weight index I0 can be "XXXX", that is, any encoded data can be used to correspond to the weight index I0. It should be understood that the above storage units can also use two bits to represent four status levels, or more bits to represent more status levels.

[0022] Next, perform step 404 to count the number of stuck errors of the encoded data between each representative weight data and its corresponding weight index according to the error mapping diagram. For example, as Figure 5 shown, when the weight index I1 corresponds to the representative weight data RW3, the state level of the storage unit storing the weight index I1 is "X1XX". That is to say, the encoded data with "X0XX" will have a stuck error, which is represented by a symbol of +1, as Figure 6A shown. Similarly, as Figure 5 shown, when the weight index I2 corresponds to the representative weight data RW1, the state level of the storage unit storing the weight index I2 is "XX11". That is to say, the encoded data with "XX00" will have a stuck error, which is represented by a symbol of +1, as Figure 6B shown. Next, as Figure 5 shown, when the weight index I3 corresponds to the representative weight data RW3, the state level of the storage unit storing the weight index I3 is "0XXX". That is to say, the encoded data with "1XXX" will have a stuck error, which is represented by a symbol of +1, as Figure 6C shown. And so on until the number of stuck errors of the encoded data E between each representative weight data RW and its corresponding weight index I is counted.

[0023] Then, perform step 406 to sequentially select the encoded data with the fewest stuck errors to establish a mapping table between multiple representative weight data and multiple weight indexes. Figure 7 Table 700 showing the relationship between the representative weight data RW and the encoded data E is shown. Although the encoded data in the above embodiments represents sixteen state levels with four bits, for the sake of easy explanation, Figure 7 it is changed to represent four state levels with two bits.

[0024] Specifically, when the representative weight data RW is arranged in the order of representative weight data RW0, RW1, RW2, RW3, the corresponding encoded data E can be selected in this order. For example, as Figure 7As shown, since in the column representing the weight data RW0, the encoded data "01" has the fewest stuck errors (i.e., 0), multiple encoded data "01" in the encoded data E can be selected to correspond to the weight data RW0. That is, the number of stuck errors of the encoded data "01" is less than that of the other encoded data "11", "10", and "00". Next, in the column representing the weight data RW1, the encoded data "10" has the fewest stuck errors (i.e., 0), so multiple encoded data "10" in the encoded data E can be selected to correspond to the weight data RW1. It should be noted that although in the column representing the weight data RW2, the encoded data "01" or "10" has fewer stuck errors (i.e., 1 or 2), since the encoded data "01" or "10" has been selected to correspond to the weight data RW0 or RW1, the encoded data "11" in the encoded data E can be selected instead to correspond to the weight data RW2. That is, each weight data RW can correspond to different encoded data E. Finally, in the column representing the weight data RW3, the encoded data "00" has the fewest stuck errors (i.e., 2), so multiple encoded data "00" in the encoded data E can be selected to correspond to the weight data RW3. After performing steps 402, 404, and 406 of the above-described method 400 for operating the memory, the encoded data E with the fewest stuck errors can be found to represent the mapping relationship between the weight index I and the representative weight data RW, so as to effectively reduce the stuck errors of the index memory 132 (as Figure 1 shown), thereby improving the accuracy of the deep neural network operation.

[0025] In some embodiments, when performing deep neural network operations, as Figure 1 shown, the required weight index can be read from the index memory 132 and the corresponding representative weight data (or representative weight value) can be mapped through the above mapping table. Then, the corresponding representative weight data is input to the processing unit 110 to perform deep neural network operations.

[0026] In summary, the embodiments of the present invention can group multiple weight values into multiple representative weight data, and map multiple weight indexes to multiple representative weight data through a mapping table, so as to greatly reduce the space for storing multiple weight values in the memory. In addition, the embodiments of the present invention can generate an error mapping graph by detecting the index memory, count the number of stuck errors of the encoded data between each representative weight data and its corresponding weight index according to the error mapping graph, and sequentially select the encoded data with the fewest stuck errors to establish the above mapping table. In this way, the embodiments of the present invention can effectively reduce the stuck errors of the index memory, thereby improving the accuracy of the deep neural network operation.

[0027] Although the present invention has been disclosed above by way of examples, it is not intended to limit the present invention. Any person skilled in the relevant art can make some modifications and refinements without departing from the spirit and scope of the present invention. Therefore, the protection scope of the present invention shall be subject to what is defined by the claims.

Claims

1. A memory, applicable to performing deep neural network operations, characterized in that, The memory includes: A processing unit having a data input terminal and a data output terminal; and A weight unit configured to be coupled to the data input terminal of the processing unit, wherein the weight unit includes: An index memory configured to store a plurality of weight indexes; and A mapping table configured to respectively correspond the plurality of weight indexes to a plurality of representative weight data, wherein the mapping table is established by detecting the index memory to generate an error mapping diagram, counting the number of stuck errors of the coded data between each representative weight data and its corresponding weight index according to the error mapping diagram, and sequentially selecting the coded data with the least stuck errors.

2. The memory according to claim 1, wherein Wherein the mapping table has a plurality of coded data to represent the mapping relationship between the plurality of weight indexes and the plurality of representative weight data.

3. The memory according to claim 1, characterized in that, Wherein the plurality of representative weight data is obtained by clustering a plurality of weight values, and the weight change of the plurality of representative weight data is less than the weight change of the plurality of weight values.

4. The memory according to claim 1, wherein It further includes: A data input unit configured to be coupled to the data input terminal of the processing unit and used to input an arithmetic input value to the processing unit; And A feedback unit configured to be coupled to the data input terminal and the data output terminal, wherein the feedback unit re-inputs the arithmetic result value output by the processing unit to the processing unit as a new arithmetic input value.

5. A method for operating a memory, applicable to performing deep neural network operations, characterized in that, The operation method of the memory includes a mapping method, and the mapping method includes: Coupling the weight unit to the data input terminal of the processing unit, wherein the weight unit includes an index memory storing a plurality of weight indexes and a mapping table that respectively corresponds the plurality of weight indexes to a plurality of representative weight data; Detecting the index memory to generate an error mapping diagram, wherein the error mapping diagram includes a plurality of stuck errors; Counting the number of stuck errors of the coded data between each representative weight data and its corresponding weight index according to the error mapping diagram; and Sequentially selecting the coded data with the least stuck errors to establish the mapping table between the plurality of representative weight data and the plurality of weight indexes.

6. The method for operating a memory according to claim 5, characterized in that, Wherein the step of sequentially selecting the coded data with the least stuck errors includes: Selecting a first coded data among the plurality of coded data to correspond to a first representative weight data among the plurality of representative weight data, Wherein the number of stuck errors of corresponding the first representative weight data with the first coded data is less than the number of stuck errors of corresponding the first representative weight data with other coded data among the plurality of coded data.

7. The method for operating a memory according to claim 6, wherein, It further includes: Selecting a second coded data among the plurality of coded data to correspond to a second representative weight data among the plurality of representative weight data; Selecting a third coded data among the plurality of coded data to correspond to a third representative weight data among the plurality of representative weight data; And Selecting a fourth coded data among the plurality of coded data to correspond to a fourth representative weight data among the plurality of representative weight data, wherein the first coded data, the second coded data, the third coded data, and the fourth coded data have different coded data.

8. The method for operating a memory according to claim 5, wherein, It further includes a reading method, where the reading method includes: Reading the required weight index from the index memory and mapping out the corresponding representative weight data through the mapping table; and Inputting the corresponding representative weight data into the processing unit to perform the deep neural network operation.

9. The method for operating a memory according to claim 5, wherein, Where the mapping method further includes: clustering a plurality of weight values into the plurality of representative weight data, and the weight change of the plurality of representative weight data is less than the weight change of the plurality of weight values.

Images