Intelligence processing unit and method of finding extreme value

Abstract

The intelligent processor includes a first memory, a second memory, and a vector core circuit. The first memory stores a batch of data. The second memory stores masking data. The vector core circuit is used to: find a first extreme value among a plurality of data values ​​in the batch of data, and store the first extreme value and a first position index value of the first extreme value in the first memory; adjust a corresponding bit in the masking data according to the first position index value; and find a second extreme value among the plurality of data values ​​according to the corresponding bit, and store the second extreme value and a second position index value of the second extreme value in the first memory, wherein the second extreme value is different from the first extreme value.

Description

Technical Field

[0001] This case concerns intelligent processors, particularly intelligent processors capable of processing several batches of data in parallel to find several extreme values ​​in those batches of data, and their methods for finding these extreme values. Prior Technology

[0002] The TopK operator is a common operation in machine learning and deep learning. Its main function is to select the top K largest (or smallest) values ​​from a dataset or tensor. Therefore, it is frequently used in scenarios requiring sorting, filtering data, or selecting important features. In current technology, the TopK operator is typically executed on the data to be processed by the system's central processing unit (CPU). Since the CPU can only process data sequentially, if there are many data points to be processed, the CPU can only execute the TopK operator on each of those data points individually, resulting in relatively low overall processing efficiency. Summary of the Invention

[0003] In some embodiments, one of the objectives of this invention is to provide an intelligent processor capable of processing several batches of data in parallel to find several extreme values ​​among the batches of data, and a method for finding extreme values, in order to improve the shortcomings of prior art.

[0004] In some embodiments, the intelligent processor includes a first memory, a second memory, and a vector core circuit. The first memory stores a batch of data. The second memory stores masking data. The vector core circuit is used to: find a first extreme value among a plurality of data values ​​in the batch of data, and store the first extreme value and a first location index value of the first extreme value in the first memory; adjust a corresponding bit in the masking data according to the first location index value; and find a second extreme value among the plurality of data values ​​according to the corresponding bit, and store the second extreme value and a second location index value of the second extreme value in the first memory, wherein the second extreme value is different from the first extreme value.

[0005] In some embodiments, the method for finding an extreme value executed by a smart processor includes the following operations: finding a first extreme value among a plurality of data values, and storing the first extreme value and a first position index value of the first extreme value in a first memory of the smart processor; adjusting a corresponding bit in a masking data according to the first position index value; and finding a second extreme value among the plurality of data values ​​according to the corresponding bit, and storing the second extreme value and a second position index value of the second extreme value in the first memory, wherein the second extreme value is different from the first extreme value.

[0006] Regarding the features, implementation, and effects of this case, the preferred embodiments are described in detail below with reference to the drawings. Simple Explanation of the Diagram

[0007] [Figure 1] is a schematic diagram illustrating a smart processor according to some embodiments of this case; [Figure 2] is a flowchart illustrating the execution of the TopK operator by the intelligent processor of Figure 1 according to some embodiments of this case; [Figure 3A] is a schematic diagram illustrating multiple batches of data in Figure 1 according to some embodiments of this case; [Figure 3B] is a schematic diagram illustrating multiple masking data in Figure 1 according to some embodiments of this case; [Figure 3C] is a schematic diagram illustrating the operation of finding the first maximum value of multiple batches of data in Figure 3A according to some embodiments of this case; [Figure 3D] is a schematic diagram illustrating the operation of adjusting corresponding bits of multiple masking data in Figure 3B according to some embodiments of this case; [Figure 3E] is a schematic diagram illustrating the operation of finding the second maximum value of multiple batches of data in Figure 3C according to some embodiments of this case; [Figure 4] is a flowchart illustrating the operation of the intelligent processor in Figure 1 searching for the second maximum value according to some embodiments of this case; and [Figure 5] is a flowchart of a method for finding extreme values ​​according to some embodiments of this case. Implementation

[0008] All terms used herein have their common meanings. The definitions of the terms mentioned above in commonly used dictionaries, and any examples of the use of any term discussed herein, are merely illustrative and should not limit the scope or meaning of this application. Similarly, this application is not limited to the various embodiments shown in this specification.

[0009] As used herein, "coupling" or "connection" can refer to two or more components making direct physical or electrical contact with each other, or indirectly making direct physical or electrical contact with each other, or to two or more components operating or moving together. As used herein, the term "circuit" can refer to a device in which at least one transistor and / or at least one active or passive component are connected in a certain manner to process signals.

[0010] Figure 1 is a schematic diagram of an intelligent processor 100 according to some embodiments of this invention. In some embodiments, the intelligent processor 100 can be applied to machine learning or deep learning and can execute the TopK operator in the field of machine learning (or deep learning). In some embodiments, the TopK operator can be used to find the top K maximum values ​​and the index values ​​of the top K maximum values ​​in the data set (e.g., tensor data, but not limited to). The above function can be used to sort data, filter data, or select important features. It should be understood that in some embodiments, the TopK operator can also be modified to find the top K minimum values ​​and the index values ​​of the K minimum values. Therefore, in different embodiments, the intelligent processor 100 can find the extreme values ​​(which can be maximum or minimum values) in a data set by executing the TopK operator. For ease of explanation, the embodiments described below will be illustrated by finding the maximum value, but this invention is not limited thereto.

[0011] The intelligent processor 100 includes a vector core circuit 110, memory 120, memory 130, a direct memory access (DMA) controller 140, and a controller circuit 150. The controller circuit 150 is configured to control the vector core circuit 110, memory 120, and DMA controller 140. The DMA controller 140 can read multiple batches of data (BDs) from main memory 101 and sequentially store these batches of data BDs into memory 120. In some embodiments, the DMA controller 140 can be coupled to main memory 101 via an external memory interface (EMI). In some embodiments, main memory 101 can obtain multiple batches of data BDs from a central processing unit (not shown) in the system, wherein the central processing unit can generate these batches of data BDs by partitioning the tensor data according to the innermost dimension of the tensor data or several consecutive dimensions including the innermost dimension.

[0012] The controller circuit 150 stores a preset instruction CMD, which can be executed to control the vector core circuit 110, memory 120, and direct memory controller 140 to begin performing the operations corresponding to the TopK operator, in order to find K extreme values ​​and their positions in the batch data BD. For example, the controller circuit 150 can execute the preset instruction CMD to initialize the vector core circuit 110, thereby configuring the relevant calculation parameters, operating modes, and / or types of operations performed in the vector core circuit 110, so that the vector core circuit 110 is configured to perform extreme value search. Similarly, the controller circuit 150 can execute the preset instruction CMD to initialize memory 120 and memory 130, thereby establishing a corresponding mapping relationship between the data in memory 120 and memory 130, which can be used to assist in performing extreme value search. In some embodiments, the preset instruction CMD can be, but is not limited to, a jump instruction. Since the data amounts of these batch data BDs are the same, the controller circuit 150 can repeatedly execute the jump instruction to sequentially read these batch data from the independent memory 101 via the direct memory controller 140 according to a fixed address offset, thereby performing the TopK operator operation on these batch data BDs. In this way, the instruction size required to execute the TopK operator can be significantly reduced.

[0013] The vector core circuit 110 may include, but is not limited to, comparators, registers, multipliers, adders, multiply-accumulate circuits, and other arithmetic circuits to perform the relevant calculations required for machine learning and / or deep learning. In some embodiments, the vector core circuit 110 may use circuits such as comparators and registers to perform the relevant operations corresponding to the TopK operator. This will be explained later with reference to Figures 2 and 3A through 3E.

[0014] Memory 120 stores multiple batch data BDs and the operation results (i.e., the first K extreme values ​​N1~NK and the position index values ​​M1~MK of the first K extreme values ​​N1~NK) of the TopK operator executed by the vector core circuit 110, where K can be a positive integer greater than or equal to two. In some embodiments, memory 120 can be, but is not limited to, L2 memory. Memory 130 stores multiple mask data MDs corresponding to the multiple batch data BDs, where each mask data MD corresponds to a specific batch data BD, and the correspondence will be explained later with reference to Figures 3A and 3B. In some embodiments, memory 130 can be, but is not limited to, static random access memory.

[0015] In some embodiments, after finding the first K extreme values ​​N1~NK of each of all batch data BD stored in memory 120, the controller circuit 150 can control the direct memory access controller 140 to store the first K extreme values ​​N1~NK stored in memory 120 and the position index values ​​M1~MK of the first K extreme values ​​into main memory 101, so as to release the storage space of memory 120 to continue processing the subsequent TopK operator operation.

[0016] Figure 2 is a flowchart illustrating the execution of the TopK operator by the intelligent processor 100 of Figure 1 according to some embodiments of this invention. In Figure 2, the operations that the intelligent processor 100 can perform include operations S210 to S280. For ease of explanation, the flowchart in Figure 2 will be illustrated using the search for the maximum value as an example, but this invention is not limited to this. It should be understood that the flowchart in Figure 2 can also be modified to search for the minimum value.

[0017] In operation S205, the CPU partitions the tensor data into multiple batch data (BDs) based on its innermost dimension. As previously described, the CPU can partition the tensor data into multiple batch data (BDs) based on its innermost dimension. For example, if the tensor data has dimensions (5, 4, 3, 2), the CPU can partition it into multiple batch data (BDs) based on its innermost dimension 2. Alternatively, the CPU can partition the tensor data into multiple batch data (BDs) based on the product of multiple consecutive dimensions including the innermost dimension (e.g., the product of the innermost dimension 2 and the neighboring dimension 3, which is 6).

[0018] In operation S210, the direct memory access controller 140 sequentially accesses multiple batch data BDs from the autonomous memory 101 to the memory 120. In operation S220, the controller circuit 150 controls the vector core circuit 110 to configure multiple mask data MDs corresponding to the multiple batch data BDs in the memory 130 based on the multiple batch data BDs.

[0019] Figure 3A is a schematic diagram illustrating multiple batch data BDs in Figure 1 according to some embodiments of this invention. Refer to Figure 3A for an explanation of operation S210. Figure 3A shows three batch data BDs stored in memory 120. Each batch data BD contains several data values. In some embodiments, the direct memory access controller 140 may selectively fill at least one predetermined value into the batch data BD so that the number of data values ​​in one batch data BD corresponds to a memory column of memory 120. For example, if each data value can be represented as a 16-bit signed integer, the aforementioned predetermined value may be set to -32768 (i.e., a predetermined minimum value that can be represented by 16 bits) when searching for the maximum value. Conversely, if searching for the minimum value, the predetermined value may be set to a predetermined maximum value that can be represented by 16 bits. To conform to the hardware access data method, the direct memory access controller 140 may store a batch data BD into a memory column of memory 120 and fill the blank data value positions in the batch data BD with the predetermined value -32768. As shown in Figure 3A, if a memory column of memory 120 can store 14 data values, the direct memory access controller 140 can fill in several predetermined values ​​-32768 in the blank data value positions after the first batch data BD (labeled as batch data 301) so that the number of data values ​​in a single batch data BD is 14. Similarly, the direct memory access controller 140 can fill in several predetermined values ​​-32768 in the blank data value positions of the second batch data BD (labeled as batch data 302) and the third batch data BD (labeled as batch data 303). In this way, each batch data BD can just fill one memory column of memory 120, and the additional predetermined value -32768 does not affect the TopK operator's operation of finding the maximum value.

[0020] Figure 3B is a schematic diagram illustrating multiple mask data MDs in Figure 1 according to some embodiments of this invention. Refer to Figure 3B for an explanation of operation S220. Figure 3B shows three mask data MDs stored in memory 130. In some embodiments, each mask data MD contains a plurality of bits, and the arrangement of these bits corresponds to the arrangement of several data values ​​in each batch of data BD. For example, the first mask data MD (labeled as mask data 311) corresponds to batch data 301 in Figure 3A, the second mask data MD (labeled as mask data 312) corresponds to batch data 302 in Figure 3A, and the third mask data MD (labeled as mask data 313) corresponds to batch data 303 in Figure 3A. Multiple bits of each of mask data 311, mask data 312, and mask data 313 are pre-configured to a first preset value (e.g., a logical value of 0). Since the complex number of bits corresponds to multiple data values ​​in the corresponding mask data MD in terms of position, the vector kernel circuit 110 can decide whether to ignore the corresponding data values ​​in the corresponding batch data BD during the execution of the TopK operator based on the above correspondence.

[0021] Referring again to Figure 2, in operation S230, the vector core circuit 110 finds the first maximum value of each of the multiple batch data BDs in memory 120, and stores the first maximum value and its first position index. In operation S240, the vector core circuit 110 adjusts the corresponding bit in the corresponding mask data MD according to the first position index.

[0022] Figure 3C is a schematic diagram illustrating the operation of finding the first maximum value of multiple batch data BD in Figure 3A according to some embodiments of this invention. To explain operation S230, please refer to Figures 3A and 3C together. In some embodiments, the vector core circuit 110 may utilize an internal comparator and a temporary register to perform operation S230. Taking batch data 301 as an example, the aforementioned comparator can compare data value 3 (its position index value is 0) and data value 5 (its position index value is 1) in batch data 301, and store the larger data value 5 in the aforementioned temporary register. Next, the comparator can compare the next data value 1 (its position index value is 2) in batch data 301 with the data value 5 currently stored in the temporary register. Since the data value 5 currently stored in the temporary register is larger, the comparator can continue to compare the next data value 4 in the comparable batch data 301 with the data value 5 currently stored in the temporary register. Similarly, after comparing all data values ​​in batch data 301, vector core circuit 110 can find the first maximum value of batch data 301 as data value 12, and its first position index value is 9. The aforementioned first position index value is used to indicate the position of data value 12 among the multiple data values ​​in batch data 301 (e.g., the position of the 9th bit). By the same operation, vector core circuit 110 can find the first maximum value of batch data 302 as data value 18, and its corresponding first position index value is 9. Vector core circuit 110 can find the first maximum value of batch data 303 as data value 29, and its corresponding first position index value is 2.

[0023] Figure 3D is a schematic diagram illustrating the operation of adjusting corresponding bits of multiple masking data MDs in Figure 3B according to some embodiments of this case. For an explanation of operation S240, please refer to Figures 3B and 3D. In the previous operation, the vector core circuit 110 finds that the first position index value of the first maximum value of the batch data 301 is 9. Based on this first position index value, the vector core circuit 110 can adjust the corresponding bit in the masking data 311 of the memory 130 corresponding to the batch data 301, corresponding to the first position index value, from a first preset value (e.g., logic 0) to a second preset value (e.g., logic 1). Similarly, based on the first position index value of the batch data 302, the vector core circuit 110 can adjust the corresponding bit in the masking data 312 of the memory 130 corresponding to the batch data 302, corresponding to the first position index value, from the first preset value to the second preset value. Based on the first position index value of the batch data 303, the vector core circuit 110 can adjust the corresponding bit in the mask data 313 of the memory 130 corresponding to the first position index value from the first preset value to the second preset value.

[0024] Referring to Figure 2, in operation S250, the vector core circuit 110 searches for the second maximum value of each of the multiple batch data BDs in the memory 120 according to the corresponding bit, and stores the second maximum value and the second position index value of the second maximum value.

[0025] Figure 3E is a schematic diagram illustrating the operation of finding the second maximum value of multiple batch data BDs in Figure 3C according to some embodiments of this case. For an explanation of operation S250, please refer to Figures 3C, 3D, and 3E. In some embodiments, the vector core circuit 110 can exclude the first maximum value in multiple batch data BDs based on the bits of the second preset value in multiple mask data in the memory 130, and search for the second maximum value among the multiple remaining data values ​​in the excluded batch data. Taking batch data 301 as an example, the vector core circuit 110 can determine that data value 12 is the previously found first maximum value based on the corresponding mask data 311 in the memory 130 (that is, the position of the first maximum value can be determined based on the corresponding bits of the second preset value in the mask data 311). Under these conditions, the vector core circuit 110 can exclude data value 12 from the batch data 301, and find the second maximum value as data value 9 and its second position index value as 8 from the remaining data values ​​3, 5, 1, 4, 2, 6, 7, 8, 9, and multiple predetermined values ​​-32768 (i.e., data value 9 is the 8th bit in the batch data 301). Based on a similar operation, the vector core circuit 110 can find the second maximum value as data value 14 and its second position index value as 2 from the batch data 302, and can find the second maximum value as data value 28 and its second position index value as 1 from the batch data 303.

[0026] Referring again to Figure 2, in operation S260, confirm whether the first K maximum values ​​of all batch data BDs in memory 120 have been found. If not, then execute operation S240 again to adjust a corresponding bit in the corresponding mask data MD according to the second position index value, and thereby find the third maximum value and its third position index value of each of the multiple batch data BDs, until the first K maximum values ​​of all batch data BDs in memory 120 are found. If yes, then execute operation S270. It should be understood that the extreme value N1 in Figure 1 can be the aforementioned first maximum value, the extreme value N2 can be the aforementioned second maximum value, and so on, the extreme value NK can be the Kth maximum value. Similarly, the position index value M1 is the aforementioned first position index value, the position index value M2 is the aforementioned second position index value, and the position index value MK is the Kth position index value.

[0027] In operation S270, the direct memory access controller 140 stores the first K maximum values ​​stored in memory 120 and their location indices into main memory 101. In operation S280, the controller circuit 150 checks with the direct memory access controller 140 whether there is any remaining batch data BD that has not been processed. If so, operation S210 is executed again to continue processing the remaining batch data BD. If not, the relevant operations of the TopK operator are terminated.

[0028] In some related technologies, the TopK operator is executed by the central processing unit (CPU) of the system. In these technologies, the CPU can only process multiple data sets sequentially to find the top K maximum values ​​for each set, resulting in relatively low processing efficiency. Compared to these related technologies, in some embodiments of this invention, the TopK operator can be executed concurrently on multiple batches of data (BD) by increasing the number of vector core circuits 110 in the intelligent processor 100, thus finding the top K maximum values ​​for each set of data (BD) more efficiently. In this way, the intelligent processor 100 can improve the decision-making efficiency and processing performance of machine learning, deep learning, and / or neural networks, thereby bringing significant improvements to the aforementioned application areas.

[0029] Figure 4 is a flowchart illustrating the operation of the intelligent processor 100 of Figure 1 in finding a second maximum value according to some embodiments of this invention. In some embodiments, the multiple operations of Figure 4 may be additional operations executable in operation S250 of Figure 2. In operation S410, the first maximum value and the second maximum value are compared to confirm whether the first maximum value is the same as the second maximum value. If yes, operation S420 is executed. In operation S420, the second maximum value is modified to a preset minimum value, and the current maximum value among the remaining data values ​​after excluding the first maximum value and the second maximum value is found from the plurality of data values, and the current maximum value is recorded as the second maximum value. If not, the operation ends.

[0030] For example, after finding the first maximum value (i.e., data value 12) of batch data 301, the vector core circuit 110 can store the first maximum value (i.e., data value 12) in its register. Next, in the previous example, the vector core circuit 110 finds the second maximum value of batch data 301 to be data value 9. The vector core circuit 110 can compare this second maximum value with the first maximum value stored in the register. Since data value 9 is different from data value 12 in this example, the vector core circuit 110 does not need to perform any further operations. In other examples, if the vector core circuit 110 finds that the second maximum value of batch data 301 is also data value 12, the vector core circuit 110 will know that the first maximum value is the same as the second maximum value. Under this condition, the vector core circuit 110 will modify the second maximum value to a preset minimum value, find the current maximum value among the remaining data values ​​in batch data 301 after excluding the first and second data values, record this current maximum value as the new second maximum value, and store the second maximum value and its second position index value. By performing the above operations, it can be ensured that the first K maximum values ​​found by the intelligent processor 100 are all different values.

[0031] As mentioned above, the processes in the foregoing embodiments are illustrated using the search for the maximum value as an example, but this invention is not limited to this. In other embodiments, the processes in the foregoing embodiments can also be modified to search for the minimum value.

[0032] Figure 5 is a flowchart of a method 500 for finding extreme values ​​according to some embodiments of this invention. In some embodiments, the method 500 for finding extreme values ​​may be executed by, but is not limited to, the intelligent processor 100 of Figure 1.

[0033] In operation S510, a first extreme value among a plurality of data values ​​is found, and the first extreme value and a first position index value of the first extreme value are stored in a first memory of the intelligent processor. In operation S520, a corresponding bit in the mask data is adjusted according to the first position index value. In operation S530, a second extreme value among the plurality of data values ​​is found according to the corresponding bit, and the second extreme value and a second position index value of the second extreme value are stored in the first memory, wherein the second extreme value is different from the first extreme value.

[0034] The description of the above-described method 500 for finding extreme values ​​can be found in the foregoing embodiments, and therefore will not be repeated here. The above-described operations are merely examples and are not limited to being performed in the order shown in these examples. Without departing from the operation mode and scope of the embodiments of this case, these operations may be appropriately added, replaced, omitted, performed in a different order, or performed simultaneously or partially simultaneously.

[0035] In summary, the intelligent processor and the method for finding extreme values ​​provided in some embodiments of this case can process multiple batches of data in parallel, thereby improving the processing efficiency of executing the TopK operator.

[0036] Although the embodiments of this case are described above, these embodiments are not intended to limit this case. Those skilled in the art can make variations to the technical features of this case based on the express or implied content of this case. All such variations may fall within the scope of patent protection sought in this case. In other words, the scope of patent protection in this case shall be determined by the scope of the patent application in this specification.

[0037] 100: Intelligent Processor 101: Main Memory 110: Vector Core Circuit 120, 130: Memory 140: Direct Memory Access Controller 150: Controller circuit 301, 302, 303: Batch Data 311, 312, 313: Masking data -32768: Pre-determined value 500: Methods for Finding Extreme Values BD: Batch Data CMD: Preset Commands M1~MK: Location index values MD: Masking Data N1~NK: Extreme values S205, S210, S220, S230, S240, S250, S260, S270, S280: Operation S410, S420: Operation S510, S520, S530: Operation

Claims

1. A smart processor, comprising: a first memory for storing a batch of data; a second memory for storing masking data; and a vector core circuit for: finding a first extreme value among a plurality of data values ​​of the batch of data, and storing the first extreme value and a first position index value of the first extreme value in the first memory; adjusting a corresponding bit in the masking data according to the first position index value; and finding a second extreme value among the plurality of data values ​​according to the corresponding bit, and storing the second extreme value and a second position index value of the second extreme value in the first memory, wherein the second extreme value is different from the first extreme value.

2. The intelligent processor of request item 1, wherein after finding the first extreme value, the vector core circuit adjusts the corresponding bit from a first preset value to a second preset value according to the first position index value.

3. The intelligent processor of claim 1, wherein the vector core circuit excludes the first extreme value from the plurality of data values ​​based on the corresponding bit, and searches for the second extreme value in the plurality of data values ​​after the exclusion.

4. The intelligent processor of claim 1, wherein the vector core circuit further compares the first extreme value with the second extreme value, and if the second extreme value is the same as the first extreme value, the vector core circuit further modifies the second extreme value to a predetermined value, finds a current extreme value among the remaining data values ​​excluding the first extreme value and the second extreme value from the plurality of data values, and records the current extreme value as the second extreme value.

5. The intelligent processor of claim 1 further includes: a direct memory access controller for reading the bulk data from a main memory and storing the bulk data into the first memory.

6. The intelligent processor of claim 5, wherein the direct memory access controller more selectively fills a predetermined value into the plurality of data values ​​such that the number of the plurality of data values ​​corresponds to a memory column of the first memory.

7. The intelligent processor of claim 1 further includes: a controller circuit that stores a preset instruction and executes the preset instruction to configure the vector core circuit, the first memory and the second memory to find the first extreme value and the second extreme value.

8. The intelligent processor of claim 1, wherein the masking data comprises a plurality of bits, the arrangement of the plurality of bits respectively corresponds to the arrangement of the plurality of data values, and the plurality of bits includes the corresponding bits.

9. The intelligent processor of claim 1, wherein the first position index value indicates a position of the first extreme value among the plurality of data values.

10. A method for finding an extreme value, executed by a smart processor, the method comprising: finding a first extreme value among a plurality of data values, and storing the first extreme value and a first position index value of the first extreme value in a first memory of the smart processor; adjusting a corresponding bit in mask data according to the first position index value; and finding a second extreme value among the plurality of data values ​​according to the corresponding bit, and storing the second extreme value and a second position index value of the second extreme value in the first memory, wherein the second extreme value is different from the first extreme value.

Images