Intelligent processor and method for searching extreme value
By designing an intelligent processor, using vector core circuits to process multiple batches of data side by side, the problem of low processing efficiency of the central processor when executing the TopK operator is solved, and more efficient data processing is achieved.
Patent Information
- Application Number
- CN202411525361.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-10-30
- Publication Date
- 2025-02-25
AI Technical Summary
In the prior art, the central processor can only process data sequentially when executing the TopK operator, resulting in low processing efficiency, especially when processing a large amount of data.
An intelligent processor is designed, including a vector core circuit, a first memory and a second memory, by processing multiple batch data in parallel, and adjusting using mask data to find multiple extreme values.
It improves the processing efficiency of the TopK operator and can process multiple batches of data in parallel, thereby improving the decision-making efficiency and processing efficiency of machine learning, deep learning and neural networks.
Smart Images
Figure CN119514723A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of processor technology, and in particular to an intelligent processor and a method for finding extreme values. Background Art
[0002] The TopK operator is a type of operation commonly used in machine learning and deep learning. Its main function is to select the top K largest (or smallest) values from a data set or tensor data. Therefore, it is often used in scenarios that require sorting, filtering data, or selecting important features. In the prior art, the TopK operator is usually executed on the data to be processed by the central processing unit in the system. Since the central processing unit can only process data sequentially, if the number of data to be processed is large, the central processing unit can only execute the TopK operator on these data to be processed one by one, resulting in relatively low overall processing efficiency. Summary of the invention
[0003] In some embodiments, one of the purposes of the present application is to provide an intelligent processor capable of processing a plurality of batches of data in parallel to find a plurality of extreme values in the plurality of batches of data and a method for finding extreme values thereof, so as to improve the deficiencies of the prior art.
[0004] In some embodiments, the intelligent processor includes a first memory, a second memory, and a vector core circuit. The first memory stores a batch of data. The second memory stores a mask data. The vector core circuit is used to: find a first extreme value among multiple data values of the batch of data, and store the first extreme value and a first position index value of the first extreme value to the first memory; adjust a corresponding bit in the mask data according to the first position index value; and find a second extreme value among the multiple data values according to the corresponding bit, and store the second extreme value and a second position index value of the second extreme value to the first memory, wherein the second extreme value is different from the first extreme value.
[0005] In some embodiments, the method for finding extreme values performed by an intelligent processor includes the following operations: finding a first extreme value among multiple data values, and storing the first extreme value and a first position index value of the first extreme value to a first memory of the intelligent processor; adjusting a corresponding bit in a mask data according to the first position index value; and finding a second extreme value among the multiple data values according to the corresponding bit, and storing the second extreme value and a second position index value of the second extreme value to the first memory, wherein the second extreme value is different from the first extreme value.
[0006] The features, implementation and effects of the present application are described in detail below with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS
[0007] Figure 1 A schematic diagram of an intelligent processor is drawn according to some embodiments of the present application;
[0008] Figure 2 According to some embodiments of the present application, Figure 1 Flowchart of the intelligent processor executing the TopK operator;
[0009] Figure 3A According to some embodiments of the present application, Figure 1 Schematic diagram of multiple batches of data in;
[0010] Figure 3B According to some embodiments of the present application, Figure 1 Schematic diagram of multiple mask data in;
[0011] Figure 3C To draw a pair according to some embodiments of the present application Figure 3A Schematic diagram of the operation of finding the first maximum value among multiple batch data;
[0012] Figure 3D To draw adjustments according to some embodiments of the present application Figure 3B A schematic diagram of operations on corresponding bits of a plurality of mask data in;
[0013] Figure 3E To draw a pair according to some embodiments of the present application Figure 3C Schematic diagram of the operation of finding the second maximum value from multiple batches of data;
[0014] Figure 4 According to some embodiments of the present application, Figure 1 An operation flow chart of the intelligent processor for finding the second maximum value; and
[0015] Figure 5 The present invention is a flowchart of a method for finding an extreme value according to some embodiments of the present application.
[0016] 100: Intelligent processor;
[0017] 101: main memory;
[0018] 110: vector core circuit;
[0019] 120, 130: memory;
[0020] 140: direct memory access controller;
[0021] 150: controller circuit;
[0022] 301, 302, 303: batch data;
[0023] 311, 312, 313: mask data;
[0024] -32768: Predetermined value;
[0025] 500: Methods for finding extreme values;
[0026] BD: Bulk Data;
[0027] CMD: preset command;
[0028] M1~MK: position index value;
[0029] MD: mask data;
[0030] N1~NK: extreme value;
[0031] S205, S210, S220, S230, S240, S250, S260, S270, S280: operation;
[0032] S410, S420: operation;
[0033] S510, S520, S530: operation. DETAILED DESCRIPTION
[0034] All words used herein have their ordinary meanings. The definitions of the above words in commonly used dictionaries, including any use examples of words discussed herein in the content of this application are only examples and should not limit the scope and meaning of this application. Similarly, this application is not limited to the various embodiments shown in this specification.
[0035] Figure 1 A schematic diagram of an intelligent processor 100 is drawn according to some embodiments of the present application. In some embodiments, the intelligent processor 100 can be applied to machine learning or deep learning, and can execute the TopK operator in the field of machine learning (or deep learning). In some embodiments, the TopK operator can be used to find the top K maximum values and the position index values of the top K maximum values in a data set (for example, but not limited to tensor data), and the above functions can be used to sort data, filter data or select important features. It should be understood that in some embodiments, the TopK operator can also be modified to find the top K minimum values and the position index values of the K minimum values. Therefore, in different embodiments, the intelligent processor 100 can find the extreme value (which can be the maximum value or the minimum value) in a data set by executing the TopK operator. For the convenience of explanation, the embodiments described below will be described as finding the maximum value, but the present application is not limited to this.
[0036] The intelligent processor 100 includes a vector core circuit 110, a memory 120, a memory 130, a direct memory access (DMA) controller 140, and a controller circuit 150. The controller circuit 150 can configure and / or control the vector core circuit 110, the memory 120, and the direct memory access controller 140. The direct memory access controller 140 can read a plurality of batch data BD from the main memory 101 and store the batch data BD in the memory 120 in sequence. In some embodiments, the direct memory access controller 140 can be coupled to the main memory 101 via an external memory interface (EMI). In some embodiments, the main memory 101 can obtain a plurality of batch data BD from a central processor (not shown) in the system, wherein the central processor can divide the tensor data according to the innermost dimension of the tensor data or a plurality of continuous dimensions including the innermost dimension to generate the batch data BD.
[0037] The controller circuit 150 stores a preset instruction CMD, which can execute the preset instruction to control the vector core circuit 110, the memory 120, and the direct memory access controller 140 to start executing the operation corresponding to the To pK operator to find K extreme values and the positions of the K extreme values in the batch data BD. For example, the controller circuit 150 can execute the preset instruction CMD to initialize the vector core circuit 110, thereby configuring the relevant calculation parameters, operation mode and / or type of execution operation in the vector core circuit 110, so that the vector core circuit 110 is configured to be in a state where an extreme value search can be performed. Similarly, the controller circuit 150 can execute the preset instruction CMD to initialize the memory 120 and the memory 130, so that the data in the memory 120 and the memory 130 have a corresponding mapping relationship, which can be used to assist in performing an extreme value search. In some embodiments, the preset instruction CMD can be, but is not limited to, a jump instruction. Since the data volumes of these batch data BD are the same, the controller circuit 150 can repeatedly execute the jump instruction to sequentially read these batch data from the main memory 101 via the direct memory access controller 140 according to a fixed address offset, thereby performing operations corresponding to the Top K operator on these batch data BD. In this way, the instruction size required to execute the Top K operator can be significantly reduced.
[0038] The vector core circuit 110 may include, but is not limited to, comparators, registers, multipliers, adders, multiplication-addition-accumulation circuits, and other computing circuits to perform relevant calculations required for machine learning and / or deep learning. In some embodiments, the vector core circuit 110 may use comparators and registers to perform relevant operations corresponding to the TopK operator. Figure 2 as well as FIG. 3A to FIG. 3E illustrate.
[0039] The memory 120 stores a plurality of batches of data BD, and stores the operation results of the operation corresponding to the TopK operator executed by the vector core circuit 110 (i.e., the first K extreme values N1-NK and the position index values M1-MK of the first K extreme values N1-NK), where K can be a positive integer greater than or equal to 2. In some embodiments, the memory 120 can be, but is not limited to, an L2 memory. The memory 130 stores a plurality of mask data MD corresponding to the plurality of batches of data BD, where each mask data MD corresponds to a specific batch of data BD, and the corresponding relationship will be referred to below. Figure 3A and Figure 3B In some embodiments, the memory 130 may be, but is not limited to, a static random access memory.
[0040] In some embodiments, after finding the top K extreme values N1~NK of each of all batch data BD stored in the memory 120, the controller circuit 150 can control the direct memory access controller 140 to store the top K extreme values N1~NK stored in the memory 120 and the position index values M1~MK of the top K extreme values to the main memory 101, so as to release the storage space of the memory 120 to continue processing the subsequent TopK operator operations.
[0041] Figure 2 According to some embodiments of the present application, Figure 1 The intelligent processor 100 executes the flow chart of the TopK operator. Figure 2 In the example, the operations that can be performed by the intelligent processor 100 include operations S210 to S280. For ease of description, Figure 2 The process will be described by taking finding the maximum value as an example, but the present application is not limited to this. It should be understood that Figure 2 The process can also be modified to find the minimum value.
[0042] In operation S205, the central processor divides the tensor data into a plurality of batches of data BD according to the innermost dimension of the tensor data. As previously described, the central processor may divide the tensor data into a plurality of batches of data BD according to the innermost dimension of the tensor data. For example, if the dimension of the tensor data is (5, 4, 3, 2), the central processor may divide the tensor data into a plurality of batches of data BD according to its innermost dimension 2. Alternatively, the central processor may divide the tensor data into a plurality of batches of data BD according to the product of a plurality of continuous dimensions including the innermost dimension (for example, the product 6 of the innermost dimension 2 and the adjacent dimension 3).
[0043] In operation S210, the direct memory access controller 140 sequentially accesses a plurality of batch data BD from the main memory 101 to the memory 120. In operation S220, the controller circuit 150 controls the vector core circuit 110 to configure a plurality of mask data MD corresponding to the plurality of batch data BD in the memory 130 based on the plurality of batch data BD.
[0044] Figure 3A According to some embodiments of the present application, Figure 1 For explanation of operation S210, please refer to Figure 3A . Figure 3A Three batch data BD stored in the memory 120 are shown. Each batch data BD includes several data values. In some embodiments, the direct memory access controller 140 may selectively fill at least one predetermined value into the batch data BD so that the number of data values of a batch data BD may correspond to a memory column of the memory 120. For example, if each data value can be represented as a signed integer using 16 bits, the aforementioned predetermined value may be set to -32768 (i.e., a predetermined minimum value that can be represented by 16 bits) when searching for the maximum value. Conversely, when searching for the minimum value, the predetermined value may be set to a predetermined maximum value that can be represented by 16 bits. In order to comply with the way hardware accesses data, the direct memory access controller 140 may store a batch data BD in a memory column of the memory 120, and fill the position of the blank data value in the batch data BD with the predetermined value -32768. Figure 3AAs shown, if the number of data values that can be stored in one memory column of the memory 120 is 14, the direct memory access controller 140 can fill the positions of the blank data values in the first batch data BD (labeled as batch data 301) with a number of predetermined values -32768, so that the number of data values in a single batch data BD is 14. Similarly, the direct memory access controller 140 can fill the positions of the blank data values in the second batch data BD (labeled as batch data 302) and the third batch data BD (labeled as batch data 303) with a number of predetermined values -32768. In this way, each batch data BD can just fill one memory column of the memory 120, and the additional filled predetermined value -32768 does not affect the operation of the TopK operator to find the maximum value.
[0045] Figure 3B According to some embodiments of the present application, Figure 1 To illustrate operation S220, please refer to Figure 3B . Figure 3B 3 mask data MD stored in the memory 130 are shown. In some embodiments, each mask data MD includes a plurality of bits, and the data arrangement of the plurality of bits corresponds to the arrangement of a plurality of data values in each batch of data BD. For example, the first mask data MD (labeled as mask data 311) corresponds to Figure 3A The batch data 301, the second mask data MD (labeled as mask data 312) corresponds to Figure 3A The batch data 302, and the third mask data MD (labeled as mask data 313) corresponds to Figure 3A The plurality of bits of each of the mask data 311, the mask data 312 and the mask data 313 are pre-configured as a first preset value (e.g., a logic value of 0). Since the plurality of bits and the plurality of data values in the corresponding mask data MD have a corresponding relationship in position, the vector core circuit 110 can decide whether to ignore the corresponding data value in the corresponding batch data BD during the execution of the TopK operator according to the above corresponding relationship.
[0046] Continue to refer to Figure 2 In operation S230, the vector core circuit 110 finds the first maximum value of each of the plurality of batch data BD in the memory 120, and stores the first maximum value and the first position index value of the first maximum value. In operation S240, the vector core circuit 110 adjusts the corresponding bit in the corresponding mask data MD according to the first position index value.
[0047] Figure 3C To draw a pair according to some embodiments of the present application Figure 3ASchematic diagram of the operation of finding the first maximum value of the plurality of batch data BD in FIG. To illustrate operation S230, please refer to Figure 3A and Figure 3C . In some embodiments, the vector core circuit 110 may utilize an internal comparator and register to perform operation S230. Taking the batch data 301 as an example, the aforementioned comparator may compare the data value 3 (whose position index value is 0) and the data value 5 (whose position index value is 1) in the batch data 301, and store the larger data value 5 in the aforementioned register. Next, the comparator may compare the next data value 1 (whose position index value is 2) in the batch data 301 with the data value 5 currently stored in the register. Since the data value 5 currently stored in the register is larger, the comparator may continue to compare the next data value 4 in the comparable batch data 301 with the data value 5 currently stored in the register. Similarly, after comparing all the data values in the batch data 301, the vector core circuit 110 may find that the first maximum value of the batch data 301 is the data value 12, and the first position index value is 9. The aforementioned first position index value is used to indicate the position of the data value 12 in the multiple data values of the batch data 301 (for example, the position of the 9th bit). Through the same operation, the vector core circuit 110 can find out that the first maximum value of the batch data 302 is data value 18, and its corresponding first position index value is 9. The vector core circuit 110 can find out that the first maximum value of the batch data 303 is data value 29, and its corresponding first position index value is 2.
[0048] Figure 3D To draw adjustments according to some embodiments of the present application Figure 3B Schematic diagram of the operation of corresponding bits of a plurality of mask data MD in FIG. Figure 3B and Figure 3D . In the previous operation, the vector core circuit 110 finds out that the first position index value of the first maximum value of the batch data 301 is 9. According to the first position index value, the vector core circuit 110 may adjust the corresponding bit corresponding to the first position index value in the mask data 311 corresponding to the batch data 301 in the memory 130 from the first preset value (for example, a logical value 0) to the second preset value (for example, a logical value 1). Similarly, according to the first position index value of the batch data 302, the vector core circuit 110 may adjust the corresponding bit corresponding to the first position index value in the mask data 312 corresponding to the batch data 302 in the memory 130 from the first preset value to the second preset value. According to the first position index value of the batch data 303, the vector core circuit 110 may adjust the corresponding bit corresponding to the first position index value in the mask data 313 corresponding to the batch data 303 in the memory 130 from the first preset value to the second preset value.
[0049] Continue to refer to Figure 2In operation S250 , the vector core circuit 110 searches for a second maximum value of each of the plurality of batch data BD in the memory 120 according to the corresponding bit, and stores the second maximum value and a second position index value of the second maximum value.
[0050] Figure 3E To draw a pair according to some embodiments of the present application Figure 3C Schematic diagram of the operation of finding the second maximum value from the plurality of batch data BD in FIG. Figure 3C , Figure 3D and Figure 3E . In some embodiments, the vector core circuit 110 may exclude the first maximum value in the plurality of batch data BD according to the bits with the second preset value in the plurality of mask data in the memory 130, and search for the second maximum value in the plurality of remaining data values in the plurality of batch data after exclusion. Taking the batch data 301 as an example, the vector core circuit 110 may know that the data value 12 is the first maximum value previously found according to the corresponding mask data 311 in the memory 130 (that is, the position of the first maximum value may be known according to the corresponding bit with the second preset value in the mask data 311). Under this condition, the vector core circuit 110 may exclude the data value 12 from the batch data 301, and find out that the second maximum value is data value 9 and its second position index value is 8 (that is, data value 9 is the 8th bit in the batch data 301) among the remaining data values 3, 5, 1, 4, 2, 6, 7, 8, 9 and the plurality of predetermined values -32768. Based on similar operations, the vector core circuit 110 can find out from the batch data 302 that the second maximum value is data value 14 and its second position index value is 2, and can find out from the batch data 303 that the second maximum value is data value 28 and its second position index value is 1.
[0051] Continue to refer to Figure 2 In operation S260, it is confirmed whether the first K maximum values of all the batch data BD in the memory 120 have been found. If not, operation S240 is performed again to adjust a corresponding bit in the corresponding mask data MD according to the second position index value, and accordingly find the third maximum value of each of the plurality of batch data BD and its third position index value, until the first K maximum values of all the batch data BD in the memory 120 are found. If yes, operation S270 is performed. It should be understood that Figure 1 The extreme value N1 may be the aforementioned first maximum value, the extreme value N2 may be the aforementioned second maximum value. Similarly, the extreme value NK may be the Kth maximum value. Similarly, the position index value M1 is the aforementioned first position index value, the position index value M2 is the aforementioned second position index value, and the position index value MK is the Kth position index value.
[0052] In operation S270, the direct memory access controller 140 stores the top K maximum values stored in the memory 120 and the position index values of the top K maximum values to the main memory 101. In operation S280, the controller circuit 150 confirms whether there is any remaining batch data BD that has not been processed through the direct memory access controller 140. If so, operation S210 is performed again to continue processing the remaining batch data BD. If not, the related operations of the TopK operator are terminated.
[0053] In some related technologies, the TopK operator is executed by the central processing unit in the system. In these related technologies, the central processing unit can only process multiple data one by one in sequence to find the top K maximum values of each of these data in order, resulting in relatively low processing efficiency. Compared with the above-mentioned related technologies, in some embodiments of the present application, the TopK operator can be executed in parallel on multiple batch data BD by increasing the hardware quantity of the vector core circuit 110 in the intelligent processor 100 to more efficiently find the top K maximum values of each of these batch data BD. In this way, the intelligent processor 100 can improve the decision-making efficiency and processing performance of machine learning, deep learning and / or neural networks, thereby bringing clear improvements to the above-mentioned application fields.
[0054] Figure 4 According to some embodiments of the present application, Figure 1 Flowchart of the operation of the intelligent processor 100 to find the second maximum value. In some embodiments, Figure 4 Multiple operations can be Figure 2 In operation S410, the first maximum value is compared with the second maximum value to confirm whether the first maximum value is the same as the second maximum value. If so, operation S420 is performed. In operation S420, the second maximum value is modified to a preset minimum value, and a current maximum value among the data values remaining after excluding the first maximum value and the second maximum value from the plurality of data values is found, and the current maximum value is recorded as the second maximum value. If not, the operation is terminated.
[0055] For example, after finding the first maximum value (i.e., data value 12) of the batch data 301, the vector core circuit 110 may store the first maximum value (i.e., data value 12) in its register. Next, in the previous example, the vector core circuit 110 finds the second maximum value of the batch data 301 to be data value 9. The vector core circuit 110 may compare the second maximum value with the first maximum value stored in the register. Since in this example, data value 9 is different from data value 12, the vector core circuit 110 may not perform other operations. In other examples, if the vector core circuit 110 finds the second maximum value of the batch data 301 to be data value 12, the vector core circuit 110 will know that the first maximum value is the same as the second maximum value. Under this condition, the vector core circuit 110 will modify the second maximum value to a preset minimum value, and find the current maximum value from the remaining data values in the batch data 301 excluding the first data value and the second data value, and record the current maximum value as the new second maximum value, and store the second maximum value and its second position index value. Through the above operations, it can be ensured that the first K maximum values found by the intelligent processor 100 all have different values.
[0056] As mentioned above, the processes of the aforementioned embodiments are described by taking finding the maximum value as an example, but the present application is not limited thereto. In other embodiments, the processes of the aforementioned embodiments can also be modified to find the minimum value.
[0057] Figure 5 FIG. 5 is a flow chart of a method 500 for finding an extreme value according to some embodiments of the present application. In some embodiments, the method 500 for finding an extreme value may be, but is not limited to, Figure 1 The intelligent processor 100 executes.
[0058] In operation S510, a first extreme value among a plurality of data values is searched, and the first extreme value and a first position index value of the first extreme value are stored in a first memory of an intelligent processor. In operation S520, a corresponding bit in the mask data is adjusted according to the first position index value. In operation S530, a second extreme value among the plurality of data values is searched according to the corresponding bit, and the second extreme value and a second position index value of the second extreme value are stored in the first memory, wherein the second extreme value is different from the first extreme value.
[0059] The description of the method 500 for finding an extreme value can refer to the aforementioned multiple embodiments, so it will not be repeated here. The above multiple operations are only examples and are not limited to being performed in the order in this example. Without violating the operation mode and scope of each embodiment of the present application, these operations can be appropriately added, replaced, omitted, performed in a different order, or can be performed simultaneously or partially simultaneously.
[0060] In summary, the intelligent processor and the method for finding extreme values provided in some embodiments of the present application can process multiple batches of data in parallel, thereby improving the processing efficiency of executing the TopK operator.
[0061] Although the embodiments of the present application are described above, these embodiments are not intended to limit the present application. Technicians in this technical field may change the technical features of the present application based on the explicit or implicit contents of the present application. All these changes may fall within the scope of patent protection sought by the present application. In other words, the scope of patent protection of the present application shall be based on the scope of the patent application of this specification.
Claims
1. An intelligent processor, characterized in that: include: a first memory storing a batch of data; A second memory storing mask data; as well as A vector core circuit for: Finding a first extreme value among a plurality of data values of the batch data, and storing the first extreme value and a first position index value of the first extreme value in the first memory; Adjust a corresponding bit in the mask data according to the first position index value; as well as A second extreme value among the plurality of data values is searched according to the corresponding bit, and the second extreme value and a second position index value of the second extreme value are stored in the first memory, wherein the second extreme value is different from the first extreme value.
2. The intelligent processor according to claim 1, characterized in that After finding the first extreme value, the vector core circuit adjusts the corresponding bit from a first preset value to a second preset value according to the first position index value.
3. The intelligent processor according to claim 1, characterized in that The vector core circuit excludes the first extreme value from the plurality of data values according to the corresponding bit, and searches for the second extreme value in the plurality of data values after being excluded.
4. The intelligent processor according to claim 1, characterized in that The vector core circuit is also used to compare the first extreme value with the second extreme value, and if the second extreme value is the same as the first extreme value, the vector core circuit will also modify the second extreme value to a predetermined value, and find a current extreme value among the remaining data values of the multiple data values excluding the first extreme value and the second extreme value, and record the current extreme value as the second extreme value.
5. The intelligent processor according to claim 1, characterized in that: Also includes: A direct memory access controller reads the batch data from a main memory and stores the batch data in the first memory.
6. The intelligent processor according to claim 5, characterized in that The direct memory access controller further selectively fills a predetermined value into the plurality of data values so that the number of the plurality of data values corresponds to a memory column of the first memory.
7. The intelligent processor according to claim 1, characterized in that Also includes: A controller circuit stores a preset instruction and executes the preset instruction to configure the first memory and the second memory so that the batch data and the mask data have a corresponding mapping relationship to find the first extreme value and the second extreme value.
8. The intelligent processor according to claim 1, characterized in that The mask data includes a plurality of bits, the arrangement of the plurality of bits respectively corresponds to the arrangement of the plurality of data values, and the plurality of bits include the corresponding bits.
9. The intelligent processor according to claim 1, wherein: The first position index value indicates a position of the first extreme value among the plurality of data values.
10. A method for finding an extreme value, executed by an intelligent processor, the method for finding an extreme value comprising: Finding a first extreme value among a plurality of data values, and storing the first extreme value and a first position index value of the first extreme value in a first memory of the intelligent processor; Adjust a corresponding bit in a mask data according to the first position index value; as well as searching for a second extreme value among the plurality of data values according to the corresponding bit, and storing the second extreme value and a second position index value of the second extreme value in the first memory, The second extreme value is different from the first extreme value.