Operator splitting method, control method and device for storage and computing integrated chip
Through splitting and slicing technology, the hardware parameters of the integrated memory chip are adapted to the computing and computing chip, and the computing problem when the operator scale exceeds the maximum computing block is solved, and the neural network computing efficiency and chip utilization are improved, so as to avoid repeated access or omission of data.
Patent Information
- Application Number
- CN202210752183.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-06-29
- Publication Date
- 2024-12-03
- Estimated Expiration
- 2042-06-29
AI Technical Summary
The existing integrated computing chips provide maximum operation block limitations when designing, resulting in the fact that neural network operations cannot be directly executed when the operator's weight matrix scale exceeds the maximum operation block, and there are problems with computing efficiency and power consumption.
By judging whether the number of rows and columns of the target operator is greater than the number of rows and columns of the maximum running block, split the target operator into multiple sub-operators, and use Slice technology to slice it to adapt to the hardware parameters of the integrated memory chip, and realize multi-weight protection splits and row-first storage in any dimension.
The operation problem when the operator scale exceeds the maximum operation block is solved, the operation efficiency is improved, the repeated access or missing data is avoided, and the memory computing chips of different specifications is adapted to the chip utilization and allocation efficiency are improved.
Smart Images

Figure CN114968182B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of semiconductor technology, and in particular to an operator splitting method, a control method, and related devices, equipment, and storage media for a storage-computing integrated chip. Background Art
[0002] In the traditional von Neumann architecture, the memory and processor are two completely separate units, and data needs to be exchanged between them during operation. When the amount of data is huge, the high energy consumption and high latency cause the bottleneck of the von Neumann architecture. In order to solve the problems caused by the separation of storage and computing, the storage-computing technology came into being. This technology transfers the operation from the processor to each flash memory unit. By storing the weight matrix (i.e., operator) of the neural network on the conductance of each flash memory unit of the flash memory unit array in the storage-computing chip, and then sending the data source represented by voltage to the flash memory unit array, the multiplication and addition operation of the data source and the weight matrix is realized.
[0003] However, when designing the integrated storage and computing chip, a maximum operation block will be given, that is, the maximum scale of a single operation, which is a fixed parameter. When the weight matrix scale of the operator exceeds the maximum operation block, the integrated storage and computing chip cannot be directly used to perform neural network operations. Summary of the invention
[0004] In response to the problems in the prior art, the present invention provides an operator splitting method, a control method, related devices, equipment and storage medium for a storage and computing integrated chip, which can at least partially solve the problems in the prior art.
[0005] In order to achieve the above object, the present invention adopts the following technical solution:
[0006] In a first aspect, a method for splitting operators for a storage-computation integrated chip is provided, comprising:
[0007] Determine whether the number of rows of the target operator is greater than the number of rows of the maximum operation block, and whether the number of columns of the target operator is greater than the number of columns of the maximum operation block, where the number of rows and columns of the maximum operation block is a parameter of the flash memory cell array of the target storage-computing integrated chip;
[0008] The target operator is split into multiple sub-operators according to the judgment result.
[0009] Furthermore, the target operator is split into a plurality of sub-operators according to the judgment result, including:
[0010] If the number of columns of the target operator is not greater than the number of columns of the maximum running block and the number of rows of the target operator is greater than the number of rows of the maximum running block, the target operator is split into multiple sub-operators according to the number of rows of the maximum running block and the number of rows and columns of the target operator;
[0011] If the number of columns of the target operator is greater than the number of columns of the maximum running block, split the target operator into multiple sub-operators according to the number of rows and columns of the maximum running block and the number of rows and columns of the target operator;
[0012] The number of rows of the sub-operators is not greater than the number of rows of the maximum running block, and the number of columns of the sub-operators is not greater than the number of columns of the maximum running block.
[0013] Further, the step of splitting the target operator into a plurality of sub-operators according to the number of rows and columns of the maximum running block and the number of rows and columns of the target operator includes:
[0014] Obtaining the number of rows and columns of each sub-operator according to the number of rows and columns of the maximum running block and the number of rows and columns of the target operator;
[0015] According to the number of rows and columns of each sub-operator, the target operator is sliced using the Slice technology to obtain the parameter value of each element of each sub-operator.
[0016] Furthermore, the target operator is a multidimensional array stored in a row-first manner, and slicing is performed according to a certain dimension;
[0017] The method of slicing the target operator according to the number of rows and columns of each sub-operator using the Slice technology to obtain the parameter value of each element of each sub-operator includes:
[0018] Calculating the cardinality of the current dimension according to the target operator;
[0019] The number of elements in a single continuous memory block is obtained according to the length of the slice required for the current dimension and the cardinality;
[0020] Calculate the first address offset of the first continuous memory block according to the starting position of the slice and the cardinality;
[0021] Obtaining the first address offset of each continuous memory block according to the length of the slice required for the current dimension, the cardinality, and the first address offset of the first continuous memory block;
[0022] Calculating the number of continuous memory blocks according to the target operator size and the number of elements in a single continuous memory block;
[0023] The parameter value of each element of each sub-operator is obtained according to the first address offset of each continuous memory block, the length of the slice required for the current dimension, and the number of continuous memory blocks.
[0024] Further, the calculating the cardinality of the current slice dimension according to the target operator includes:
[0025] The cardinality is obtained by calculating the product of the number of elements in all dimensions after the current slice dimension of the target operator.
[0026] Further, the acquiring the number of rows and columns of each sub-operator according to the number of rows of the maximum running block and the number of rows and columns of the target operator includes:
[0027] If the number of rows of the target operator is greater than the number of rows of the maximum running block, split the target operator into rows and columns to obtain the number of rows and columns of each sub-operator;
[0028] If the number of rows of the target operator is not greater than the number of rows of the maximum running block, the target operator is split by columns to obtain the number of rows and columns of each operator.
[0029] Further, splitting the target operator into a plurality of sub-operators according to the number of rows of the maximum running block and the number of rows and columns of the target operator includes:
[0030] Splitting the target operator by rows according to the number of rows of the maximum running block and the number of rows and columns of the target operator to obtain the number of rows and columns of each sub-operator;
[0031] Obtain the parameter value of each element of each sub-operator according to the number of rows and columns of each sub-operator.
[0032] Furthermore, it also includes:
[0033] The bias array corresponding to the target operator is combined with the corresponding sub-operator to obtain the data to be finally mapped to the flash memory cell array.
[0034] Further, combining the offset array corresponding to the target operator to the corresponding sub-operator includes:
[0035] If the sub-operator is obtained by splitting the target operator by rows, the offset array corresponding to the target operator is combined on any sub-operator;
[0036] If the sub-operator is obtained by splitting the target operator by columns, split the offset array corresponding to the target operator by columns and combine them with the corresponding sub-operator;
[0037] If the sub-operator is obtained by splitting according to rows and columns, the offset array corresponding to the target operator is split according to columns and then combined with any sub-operator of the corresponding column.
[0038] In a second aspect, an operator splitting device for a storage-computation integrated chip is provided, comprising:
[0039] A judgment module, judging whether the number of rows of the target operator is greater than the number of rows of the maximum operation block, and whether the number of columns of the target operator is greater than the number of columns of the maximum operation block, wherein the number of rows and columns of the maximum operation block is a parameter of the flash memory cell array of the target storage-computing integrated chip;
[0040] An operator splitting module splits the target operator into a plurality of sub-operators according to the judgment result, and the sub-operators are used to be mapped to the flash memory cell array to realize the operation of the target operator.
[0041] In a third aspect, a control method for a storage-computing integrated chip is provided, comprising:
[0042] The operation result of the previous operator is split according to the preset splitting rule and then loaded into the corresponding multiple flash memory unit sub-arrays, each of the flash memory unit sub-arrays has a corresponding sub-operator pre-stored therein, each sub-operator is obtained by splitting according to the above splitting method, and the preset splitting rule is set according to the operator splitting rule for splitting the current operator into multiple sub-operators;
[0043] The operation results of each sub-operator are obtained and combined to obtain the operation result of the current operator.
[0044] Furthermore, the operation result of the previous operator is a multidimensional array stored in a row-first manner;
[0045] If the preset splitting rule involves splitting by column, the Slice technology is used to slice the operation result of the previous operator according to a certain dimension to obtain the parameter value of each element of the split operator result.
[0046] Furthermore, the use of the Slice technology to slice the operation result of the previous operator according to a certain dimension to obtain the parameter value of each element of the split operator result includes:
[0047] Calculate the cardinality of the current dimension according to the operation result;
[0048] The number of elements in a single continuous memory block is obtained according to the length of the slice required for the current dimension and the cardinality;
[0049] Calculate the first address offset of the first continuous memory block according to the starting position of the slice and the cardinality;
[0050] Obtaining the first address offset of each continuous memory block according to the length of the slice required for the current dimension, the cardinality, and the first address offset of the first continuous memory block;
[0051] Calculating the number of continuous memory blocks according to the size of the operation result and the number of elements in a single continuous memory block;
[0052] The parameter value of each element of each operator result is obtained according to the first address offset of each continuous memory block, the length of the slice required for the current dimension, and the number of continuous memory blocks.
[0053] Further, the calculating the cardinality of the current slice dimension according to the operation result includes:
[0054] The cardinality is obtained by calculating the product of the number of elements of all dimensions after the current slice dimension of the operation result.
[0055] Furthermore, it also includes:
[0056] After combining the calculation results of each sub-operator, the activation operation is performed using the preset activation function.
[0057] Furthermore, when combining the operation results of each sub-operator, they are combined according to the principle of row splitting and addition, and column splitting and sequential connection.
[0058] In a fourth aspect, a storage and computing integrated chip is provided, comprising: a digital domain and an analog domain, the analog domain comprising: a flash memory cell array, in which data split according to the above-mentioned operator splitting method is mapped; and the above-mentioned control method is executed in the digital domain.
[0059] In a fifth aspect, an electronic device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above-mentioned operator splitting method when executing the program.
[0060] In a sixth aspect, a computer-readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the above-mentioned operator splitting method are implemented.
[0061] The embodiment of the present invention provides an operator splitting method, control method, related device and equipment for a storage-computation integrated chip, the splitting method includes: judging whether the number of rows of the target operator is greater than the number of rows of the maximum operation block, and whether the number of columns of the target operator is greater than the number of columns of the maximum operation block, the number of rows and columns of the maximum operation block being the parameters of the flash memory cell array of the target storage-computation integrated chip; splitting the target operator into multiple sub-operators according to the judgment result. By adopting the above technical solution, the scale of the operator can be adapted to the hardware parameters of the storage-computation integrated chip, solving the problem that when the scale of the weight matrix of the operator exceeds the maximum operation block, the storage-computation integrated chip cannot be directly used to perform neural network operations.
[0062] In addition, the Slice technology is used in the embodiment of the present invention to slice the target operator and the operation results of the previous operator, which can meet the splitting of any dimension of the multi-dimensional weight and adapt to the row-first storage method. Experiments have proved that this solution is feasible and can cooperate with the operator splitting solution to solve the limitations of the ARRAY scale of the storage and computing chip and avoid repeated access to data or missing data.
[0063] In order to make the above and other purposes, features and advantages of the present invention more obvious and easy to understand, preferred embodiments are specifically cited below and described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS
[0064] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the prior art descriptions. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work. In the drawings:
[0065] Figure 1 A flow chart of an operator splitting method for a storage-computation integrated chip in an embodiment of the present invention is shown;
[0066] Figure 2 The data row priority storage order is shown;
[0067] Figure 3 It is a schematic diagram of slices according to different dimensions in an embodiment of the present invention;
[0068] Figure 4 The memory diagram of the 0th dimension slice of the three-dimensional array A<3,4,5> in the embodiment of the present invention is given as an example;
[0069] Figure 5 The first dimension Slice memory of the three-dimensional array A<3,4,5> in the embodiment of the present invention is illustrated as an example;
[0070] Figure 6 The second dimension Slice memory of the three-dimensional array A<3,4,5> in the embodiment of the present invention is illustrated;
[0071] Figure 7 The control architecture after splitting operators by rows in an embodiment of the present invention is illustrated;
[0072] Figure 8 The control architecture after the operator is split by column in the embodiment of the present invention is illustrated;
[0073] Fig. 9 The control architecture after splitting operators by rows and columns in the embodiment of the present invention is given as an example;
[0074] Fig.10 A structural diagram of an operator splitting device in an embodiment of the present invention is shown;
[0075] Fig.11 4 is a structural diagram of an electronic device according to an embodiment of the present invention. DETAILED DESCRIPTION
[0076] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present application.
[0077] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.
[0078] It should be noted that the terms "including" and "having" and any variations thereof in the specification and claims of the present application and the above-mentioned drawings are intended to cover non-exclusive inclusions. For example, a process, method, system, product or apparatus comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or apparatuses.
[0079] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.
[0080] Terminology explanation:
[0081] Flash memory cell array: The memory space in the integrated computing chip used to store the weight matrices and bias matrices of operators such as Gemm and TDNN, and to perform analog matrix multiplication and addition operations on the weight array.
[0082] MV operator: multiply-accumulate operator.
[0083] splitNode: Split operator, splits the input into multiple outputs, each output is a subset of the input, the intersection is empty, and the union is equal to the input.
[0084] sliceNode: Slice operator, which takes a part of the input as output.
[0085] actNode: Activation operator, which activates the input data as output.
[0086] addNode: A point addition operator that receives two inputs and adds the corresponding data as output.
[0087] ConcateNode: Concatenates multiple inputs in sequence as output.
[0088] The single calculation amount of the existing storage and computing integrated chip is limited by the number of rows and columns of the maximum running block. The calculation that exceeds the maximum calculation block size needs to be split. If the scale of a single calculation is too small, it will also lead to increased power consumption of the chip and reduced calculation efficiency. When the rows or columns of the weight matrices of operators such as MV, TDNN and conv are too large, the operator splitting method provided in the embodiment of the present invention can scientifically split the operator on the host, and then map the split operators to the flash memory cell array of the storage and computing integrated chip to perform analog matrix multiplication and addition operations. During the calculation process of the storage and computing integrated chip, the control technology of the digital domain is used to merge the calculation results of the split sub-operators to ensure the correctness of the calculation logic.
[0089] Figure 1 A flowchart of an operator splitting method for a storage-computation integrated chip in an embodiment of the present invention is shown; Figure 1 As shown, the operator splitting method may include the following contents:
[0090] Step S100: Determine whether the number of rows of the target operator is greater than the number of rows of the maximum operation block, and whether the number of columns of the target operator is greater than the number of columns of the maximum operation block, wherein the number of rows and columns of the maximum operation block is a parameter of the flash memory cell array of the target storage-computing integrated chip;
[0091] In actual applications, since too much current in the analog calculation process will have a significant impact on the calculation results, the storage and computing integrated chip will provide a maximum operation block when designing, that is, the maximum scale of a single operation. If the operation scale exceeds the maximum operation block, it may be necessary to perform the operation multiple times. For example, the flash memory cell array of the chip can be divided into 2×4 blocks. The division is based on the actual performance of the chip, and the embodiment of the present invention does not impose any restrictions on this.
[0092] For example, the target operator can be the weight matrix of a certain layer of a neural network in a neural network model, such as the weight matrix of a certain convolutional layer in a CNN model.
[0093] Step S200: split the target operator into multiple sub-operators according to the judgment result.
[0094] Among them, if the number of rows of the target operator is greater than the number of rows of the maximum running block, the target operator can be split by rows; if the number of columns of the target operator is greater than the number of columns of the maximum running block, the target operator can be split by columns; if the number of rows of the target operator is greater than the number of rows of the maximum running block and the number of columns of the target operator is greater than the number of columns of the maximum running block, the target operator can be split by rows first and then the operator obtained after the split can be split by columns, or the target operator can be split by columns first and then the operator obtained after the split can be split by rows.
[0095] By adopting the above technical solution, the operator scale can be adapted to the hardware parameters of the storage-computing integrated chip, thereby solving the problem that when the weight matrix scale of the operator exceeds the maximum operation block, the storage-computing integrated chip cannot be directly used to perform neural network operations.
[0096] It should be noted that the embodiment of the present invention assumes that the weight matrix (i.e., operator) is 2-dimensional data. Of course, those skilled in the art will understand that this assumption is only for the convenience of explaining the technical solution. For the splitting solution, operators with weight matrices of any dimensionality can be supported.
[0097] In an optional embodiment, step S200 may include the following contents:
[0098] Step I: if the number of columns of the target operator is not greater than the number of columns of the maximum running block and the number of rows of the target operator is greater than the number of rows of the maximum running block, split the target operator into multiple sub-operators according to the number of rows of the maximum running block and the number of rows and columns of the target operator;
[0099] If the number of columns of the target operator is not greater than the number of columns of the maximum running block and the number of rows of the target operator is greater than the number of rows of the maximum running block, the target operator is split by row.
[0100] Specifically, two splitting methods can be used when splitting by row:
[0101] Method 1: If the number of rows of the largest operation block is a, the number of columns is b, and the number of rows of the target operator is x, and the number of columns is y, then when splitting, the target operator is preferentially split into an integer n a*y (a rows and y columns) and 1 (xn*a)*y (xn*a rows and y columns); for example, if the number of rows of the largest operation block is 3, the number of columns is 10, and the number of rows of the target operator is 7, and the number of columns is 7, then according to method 1, the target operator can be split into 2 sub-operators with 3 rows and 7 columns and 1 sub-operator with 1 row and 7 columns.
[0102] Method 2: If the number of rows of the maximum operation block is a, the number of columns is b, the number of rows of the target operator is x, and the number of columns is y, then when splitting, the number of sub-operators into which the target operator is split is a divided by x and rounded up, and the number of rows of each sub-operator can be arbitrarily allocated; for example, if the number of rows of the maximum operation block is 3, the number of columns is 10, and the number of rows of the target operator is 7, and the number of columns is 7, then according to method 2, the target operator can be split into 3 sub-operators, specifically 2 sub-operators with 3 rows and 7 columns and 1 sub-operator with 1 row and 7 columns, or 1 sub-operator with 3 rows and 7 columns and 2 sub-operators with 2 rows and 7 columns.
[0103] It is worth noting that if the number of columns of the target operator is not greater than the number of columns of the maximum running block and the number of rows of the target operator is not greater than the number of rows of the maximum running block, no splitting is required.
[0104] Step II: if the number of columns of the target operator is greater than the number of columns of the maximum running block, split the target operator into multiple sub-operators according to the number of rows and columns of the maximum running block and the number of rows and columns of the target operator;
[0105] The number of rows of the sub-operators is not greater than the number of rows of the maximum running block, and the number of columns of the sub-operators is not greater than the number of columns of the maximum running block.
[0106] Specifically, if the number of columns of the target operator is greater than the number of columns of the maximum operation block, and the number of rows of the target operator is not greater than the number of rows of the maximum operation block, then split by column; if the number of columns of the target operator is greater than the number of columns of the maximum operation block, and the number of rows of the target operator is also greater than the number of rows of the maximum operation block, then split by row and column, specifically, split by row first, then by column, or split by column first, then by row. The rules for splitting by row are the same as the above-mentioned methods 1 and 2 for splitting by row, see above for details.
[0107] When splitting by columns, similar to splitting by rows, two methods can also be used:
[0108] Method 1: If the number of rows and columns of the maximum operation block is a, the number of columns is b, the number of rows and columns of the target operator is x, and the number of columns is y, then when splitting, the target operator is preferentially split into an integer n x*b (x rows and b columns) and 1 x*(yn*b) (x rows and yn*b columns); for example, if the number of rows and columns of the maximum operation block is 10, the number of rows and columns is 10, the number of rows and columns of the target operator is 7, and the number of columns is 37, then according to method 1, the target operator can be split into 3 sub-operators with 7 rows and 10 columns and 1 sub-operator with 7 rows and 7 columns.
[0109] Method 2: If the number of rows and columns of the maximum operation block is a, and the number of columns is b, and the number of rows and columns of the target operator is x, and the number of columns is y, then when splitting, the number of sub-operators into which the target operator is split is b divided by y and rounded up, and the number of columns of each sub-operator can be arbitrarily adjusted; for example, if the number of rows and columns of the maximum operation block is 10, and the number of columns is 10, and the number of rows and columns of the target operator is 7, and the number of columns is 37, then according to method 2, the target operator can be split into 4 sub-operators, specifically 3 sub-operators with 7 rows and 10 columns and 1 sub-operator with 7 rows and 7 columns, or 3 sub-operators with 7 rows and 9 columns and 1 sub-operator with 7 rows and 10 columns.
[0110] It is worth noting that when splitting by rows and columns, you can first split by rows in any of the two ways of splitting by rows, without dealing with the number of columns for the time being, and then further split the sub-operators after splitting by rows in any of the two ways of splitting by columns. Of course, you can also split by columns first, and then further split the split sub-operators by rows.
[0111] It is worth mentioning that the operator is generally a multidimensional array, and its storage method in the host is generally row-first. Simply put, row-first storage is to fill up the subscripts on the right first. Suppose there is a three-dimensional array A, and its shape (the shape of the weight array) is <3,4,5>. The three dimensions are called rows, columns, and pages, and all 5 elements of each page are connected together. When storing array A in row-first mode, the page of row 0 and column 0 is stored first, and then the page of row 0 and column 1 is stored. When the corresponding pages of all columns of row 0 are stored, the pages corresponding to row 1 and all columns of row 2 are stored in sequence. See. Figure 2 , expand the three-dimensional array A and see that the label of each page in the figure indicates its storage order.
[0112] In an optional embodiment, step II further comprises:
[0113] Step a: obtaining the number of rows and columns of each sub-operator according to the number of rows and columns of the maximum running block and the number of rows and columns of the target operator;
[0114] Specifically, when splitting an operator, after knowing the scale of the operator, first split the row and column scale of the operator. After determining the scale of the sub-operator, that is, after the row and column of each sub-operator is determined, then access the data in the storage space of the operator to obtain the parameter value of each element.
[0115] Step b: According to the number of rows and columns of each sub-operator, the target operator is sliced using the Slice technology to obtain the parameter value of each element of each sub-operator.
[0116] Among them, after the rows and columns of each sub-operator are determined, the parameter value of each element in the sub-operator must be obtained from the operator's storage space. Especially for column splitting, since data is generally stored in row priority, it is necessary to read the data of certain specific columns in each row.
[0117] Specifically, the target operator is generally a multidimensional array stored in a row-major manner, and slicing is performed according to a certain dimension;
[0118] It is worth noting that some of the data that needs to be sliced are continuous in memory. The continuous data in memory is called a block, that is, a continuous memory block. The first continuous memory block is the first continuous data block. The offset refers to the offset of the first address of the current block relative to the first address of the entire original multidimensional array.
[0119] Slices can include the following:
[0120] (1) calculating the cardinality of the current dimension according to the target operator;
[0121] The cardinality is obtained by calculating the product of the number of elements in all dimensions after the current slice dimension of the target operator.
[0122] (2) obtaining the number of elements in a single continuous memory block according to the length of the slice required for the current dimension and the cardinality;
[0123] (3) calculating the first address offset of the first continuous memory block according to the starting position of the slice and the cardinality;
[0124] (4) obtaining the first address offset of each continuous memory block according to the length of the slice required for the current dimension, the cardinality, and the first address offset of the first continuous memory block;
[0125] (5) calculating the number of continuous memory blocks according to the target operator size and the number of elements in a single continuous memory block;
[0126] (6) The parameter value of each element of each sub-operator is obtained according to the first address offset of each continuous memory block, the length of the slice required for the current dimension, and the number of continuous memory blocks.
[0127] In order to enable those skilled in the art to better understand the present application, the Slice technology is described in detail below:
[0128] The purpose of Slice is to slice a multidimensional target array in a certain dimension, and the required part is cut out to form a new array, and its storage still meets the row-major principle. For an N-dimensional array, its Slice can have N dimensions. For example, for a three-dimensional array A<3,4,5>, its different Slice methods are as follows Figure 3As shown in the figure, the slice dimensions are schematic diagrams of slicing array A from the 0th, 1st, and 2nd dimensions respectively. It can be seen that there is a big difference in the access order of memory when slicing from different dimensions. How to access all the required elements without duplication or omission is the key to the design.
[0129] See also Figures 4 to 6 , Figure 4 The following figure shows the memory diagram of the 0th dimension slice of the three-dimensional array A<3,4,5>. Figure 5 The figure shows the memory diagram of the first dimension slice of the three-dimensional array A<3,4,5>. Figure 6 The following figure shows the memory diagram of the second-dimensional slice of the three-dimensional array A<3,4,5>. The slice implementation scheme is as follows:
[0130] According to the row-first storage order, we can know that all elements of the dimensions behind the current dimension are continuous in memory. For example, for the three-dimensional array A<3,4,5>, for the 0th dimension, that is, each row, the corresponding 4*5=20 elements of the last two dimensions are continuous in memory; for the 1st dimension, that is, each column, the corresponding 5 elements of the 2nd dimension are continuous in memory; and for the 2nd dimension, that is, each page, only one element can be guaranteed to be continuous in memory, because the three-dimensional array has no higher dimensions. Based on this, the implementation steps of Slice are as follows:
[0131] 1) Calculate the cardinality base_num of the current dimension, that is, the number of elements in the minimum continuous memory behind it. The calculation method is to multiply the number of elements in all dimensions behind the current dimension to be split. Take the three-dimensional array A<3,4,5> as an example. Assuming that the current slice dimension is the first dimension, the product of the number of elements in the second dimension behind the first dimension is taken as the cardinality of the current dimension.
[0132] 2) When slicing, you need to consider not only the dimension, but also slice_len, that is, the length of the slice required for that dimension, such as Figure 2 and Figure 3 As shown, the slice_len base_num elements of the current dimension are also connected end to end in memory. For example, if you need to slice the 1st to 2nd rows of the 0th dimension of the array A<3,4,5>, slice_len=2, and the two base_num elements of the 1st and 2nd rows are connected end to end in memory; and if you slice the 1st to 2nd columns of the 1st dimension of the array A, slice_len=2, and the two base_num pages corresponding to each row are connected end to end. From this, we can calculate the number of elements in a single continuous memory block as block_size=base_num*slice_len, where the number of rows of the slice is determined by the split result when the column is split.
[0133] 3) After the number of elements in a single continuous memory block is determined in step 2, it is necessary to consider the location of the memory block and calculate its offset relative to the first element of the original array. For a multidimensional array, Figure 3 As shown in the figure, the part without Slice is completely identical to the part that needs Slice in structure, and the only difference between them is the first address (i.e., the offset relative to the initial position of the original array). Based on this principle, we can derive the calculation method of the offset fir_offset of the first continuous memory block that needs Slice, which is also based on base_num. If the starting position of the required Slice is slice_start, then fir_offset = base_num*slice_start.
[0134] Among them, slice_start is the starting position of the dimension that needs to be sliced, such as rows 1 to 2 of the 0th dimension of the slice, then slice_start is 1.
[0135] 4) It is not enough to determine the offset fir_offset of the first continuous memory block. It is also necessary to determine the offsets of all continuous memory blocks in order to access all required elements. It can be observed that the increment between the first addresses of adjacent continuous memory blocks is constant. Only by calculating this increment can the offset of the first address of all subsequent continuous memory blocks be determined. It can be observed that the increment is equal to the product of the current dimension length dim_len and base_num. This principle is consistent with the principle of calculating block_size and fir_offset above, and the offset increment bias_increment = dim_len(axis)*base_num is obtained, where axis represents the dimension of the required Slice.
[0136] 5) Finally, we need to calculate the number of continuous memory blocks to ensure that all the data to be sliced can be accessed without duplication or omission. For the array A<3,4,5>, we can observe that block_num is equal to the total size of the slice required divided by block_size.
[0137] Through the above technical solution, the design of the Slice solution satisfies the splitting of the operator's multi-dimensional weights in any dimension, and can cooperate with the operator's splitting solution to solve the limitations of the computing scale of the storage and computing integrated chip.
[0138] In the embodiment of the present invention, the Slice technology is used to slice the target operator and the operation results of the previous operator, which can meet the splitting of any dimension of the multi-dimensional weight and adapt to the row-first storage method. Experiments have proved that this solution is feasible and can cooperate with the operator splitting solution to solve the limitations of the ARRAY scale of the storage and computing chip and avoid repeated access to data or missing data.
[0139] In an optional embodiment, step a may specifically include the following contents:
[0140] If the number of rows of the target operator is greater than the number of rows of the maximum running block, split the target operator into rows and columns to obtain the number of rows and columns of each sub-operator;
[0141] If the number of rows of the target operator is not greater than the number of rows of the maximum running block, the target operator is split by columns to obtain the number of rows and columns of each operator.
[0142] The methods of splitting by rows and by columns are as described above and will not be described in detail here.
[0143] In an optional embodiment, step I may include the following:
[0144] First, the target operator is split into rows according to the number of rows of the maximum running block and the number of rows and columns of the target operator to obtain the number of rows and columns of each sub-operator; then, the parameter value of each element of each sub-operator is obtained according to the number of rows and columns of each sub-operator.
[0145] The method of splitting by row is as mentioned above and will not be described again here.
[0146] It is worth noting that for the neural network model, in the operation of each neural network layer, in addition to the weight array, the bias array also needs to be operated. Each operator has its own bias. If the bias data is assigned to each sub-operator, it will cause multiple additions of bias, resulting in incorrect operation results. Since the shape of the bias of each sub-operator split by row is the same as that of the original operator, it is only necessary to assign the bias of the original operator to one of the sub-operators, and then set the bias of the remaining sub-operators to zero. This calculation has its logical support. For example, an operator is split into two sub-operators by row, and the results of matrix multiplication are A and B respectively. The final result is A+B+bias. The bias is given to the first operator, and the result of the first operator is A+bias, while the bias of the second operator is all set to zero, and the result is B. Then the operators are summed, and the result is still A+B+bias.
[0147] Therefore, the operator splitting method in the embodiment of the present invention not only includes splitting the weight array of a neural network layer, but also includes the process of processing the bias corresponding to the neural network layer. Specifically, the operator splitting method may also include: combining the bias array corresponding to the target operator to the corresponding sub-operator to obtain the data to be finally mapped to the flash memory cell array.
[0148] Specifically, the step includes:
[0149] (1) If the sub-operator is obtained by splitting the target operator by rows, the offset array corresponding to the target operator is combined on any sub-operator;
[0150] (2) If the sub-operator is obtained by splitting the target operator by columns, split the offset array corresponding to the target operator by columns and combine them with the corresponding sub-operator;
[0151] (3) If the sub-operator is obtained by splitting into rows and columns, the bias array corresponding to the target operator is split into columns and then combined with any sub-operator of the corresponding column.
[0152] It is worth noting that in step (3), the offset array is split by columns in the same manner as the operator is split by columns. The number of columns of the operator is equal to the number of columns of the offset array. Assuming that the 1st to 3rd columns of the operator are split into a sub-operator, the 1st to 3rd columns corresponding to the offset array are split out; assuming that the sub-operators split from the 1st to 3rd columns are further split into 3 sub-arrays (array 1, array 2 and array 3) according to rows, the sub-arrays split from the 1st to 3rd columns corresponding to the offset array are combined on any one of the 3 sub-arrays (array 1, array 2 and array 3) formed by the sub-operators split from the 1st to 3rd columns according to rows.
[0153] It is worth noting that the above-mentioned splitting method is executed on the tool chain, which can be understood as a program running on a terminal device or server or host or chip burning device. By writing the split data to the target storage and computing integrated chip to perform calculations, the chip can be used in various application scenarios, such as being installed on toys to perform voice recognition neural network calculations; of course, the storage and computing integrated chip can also be installed on a face recognition device. The embodiment of the present invention does not limit the application scenarios of the chip, and it can be various devices and scenarios that require neural network calculations.
[0154] Since the operator is split, in order to obtain the overall operation result of the operator, the storage-computation integrated chip needs to control the input and output of each sub-operator during operation to achieve the fusion of the sub-operators. Based on this, an embodiment of the present invention also provides a control method for the storage-computation integrated chip, including:
[0155] Step S1: splitting the operation result of the previous operator according to a preset splitting rule and loading it into corresponding multiple flash memory unit sub-arrays, each of the flash memory unit sub-arrays pre-stores a corresponding sub-operator, each sub-operator is obtained by splitting according to the above splitting method, and the preset splitting rule is set according to the operator splitting rule for splitting the current operator into multiple sub-operators;
[0156] Step S2: Obtain the operation results of each sub-operator and combine them to obtain the operation result of the current operator.
[0157] Specifically, the calculation result of the previous operator is a multidimensional array stored in the digital domain of the storage and computing integrated chip in a row-first manner; if the preset splitting rule involves splitting by columns, the Slice technology is used to slice the calculation result of the previous operator according to a certain dimension to obtain the parameter value of each element of the split operator result.
[0158] Among them, Slice slicing technology specifically includes:
[0159] (1) Calculating the cardinality of the current dimension according to the calculation result;
[0160] The cardinality is obtained by calculating the product of the number of elements of all dimensions after the current slice dimension of the operation result.
[0161] (2) obtaining the number of elements in a single continuous memory block according to the length of the slice required for the current dimension and the cardinality;
[0162] (3) calculating the first address offset of the first continuous memory block according to the starting position of the slice and the cardinality;
[0163] (4) obtaining the first address offset of each continuous memory block according to the length of the slice required for the current dimension, the cardinality, and the first address offset of the first continuous memory block;
[0164] (5) calculating the number of continuous memory blocks according to the size of the operation result and the number of elements in a single continuous memory block;
[0165] (6) Obtain the parameter value of each element of each operator result according to the first address offset of each continuous memory block, the length of the slice required for the current dimension, and the number of continuous memory blocks.
[0166] The detailed scheme of Slice slicing technology is mentioned above and will not be repeated here.
[0167] In an optional embodiment, the control method further includes:
[0168] After combining the calculation results of each sub-operator, the activation operation is performed using the preset activation function.
[0169] It is worth noting that the activation function is nonlinear. If you activate each sub-operator and then add the results together, it is generally not the same as adding the multiplication and accumulation results first and then activating. For example, the tanh function:
[0170]
[0171] It can be seen that tanh(2+3)≠tanh(2)+tanh(3).
[0172] Therefore, in order to solve the problem of nonlinear activation function, when splitting the operator, set the activation enable of the sub-operator to false, and then wait until the results of all sub-operators are combined before activation.
[0173] In an optional embodiment, when combining the operation results of each sub-operator, they are combined according to the principle of row splitting and addition and column splitting and sequential connection.
[0174] For example:
[0175] (1) For the case of the sub-operators after the original operator is split by rows, since the original operator is split by rows, the output of the previous operator also needs to be split accordingly so that it can be input into the corresponding sub-operator for calculation. At this time, an intermediate operator splitNode is added between the output of the previous operator and the current sub-operator to perform the work of splitting the output of the previous operator. splitNode receives the output of the previous operator and splits it into the corresponding number of parts as the input of each sub-operator after the original operator is split. Each sub-operator inherits multiple outputs of splitNode (one-to-one correspondence). Finally, the results of each sub-operator after calculation are added together. Every two sub-operators are connected with addNode, and the last addNode is used as the predecessor operator of the next operator. For details, see Figure 7 Now there are three MV operators mvNode0, mvNode1, and mvNode2. The number of rows of the weight matrix of mvNode1 exceeds the number of rows of the maximum operation block ARRAY_ROW_SIZE=2. Therefore, mvNode1 is split into two MV sub-operators with a weight matrix of 2*7 and a weight matrix of 1*7. In order to match the budget of each sub-operator, the output result of mvNode0 is split. A splitNode is added after mvNode0, and the output of mvNode0 is split into three parts as the input of three sub-MVs respectively. Then the outputs of these three MVs are added in pairs, and processed by addNode. The last addNode is used as the predecessor operator of mvNode2.
[0176] Those skilled in the art will appreciate that, for dual-input scenarios, the results of the sub-operators may be added in pairs, but for scenarios supporting multiple inputs, the outputs of multiple sub-operators may be directly accumulated.
[0177] It is worth noting that in the above technical solution, for tool chains that do not support multiple outputs of operators, splitNode can be replaced with multiple sliceNodes, each sliceNode receives the output of the previous operator of the current operator and slices it, and then takes out the part required by the corresponding sub-operator as its output. Each sliceNode corresponds to a sub-operator one by one. In this way, the multi-output problem can be converted into a single output of multiple operators.
[0178] It is worth noting that split outputs multiple multidimensional arrays, and Slice simply picks the corresponding one from these multiple multidimensional arrays as the input of the next operator.
[0179] (2) For the case where the original operator is a sub-operator after column splitting, column splitting does not require processing the output of the previous operator. Instead, it only needs to use its output as the input of all the sub-operators after splitting. After that, the outputs of all MV sub-operators are sequentially connected using concateNode and used as the input of the next MV operator. For details, see Figure 8 , the number of columns of mvNode1 exceeds the number of columns of the maximum operation block ARRAY_COLUMN_SIZE=3, so it needs to be split by column, split into three child mvNodes, each of which accepts the output of mvNode0 as its input, and then the outputs of the three are connected in sequence using concateNode as the input of mvNode2.
[0180] (3) For the case where the original operator is split into sub-operators by rows and columns at the same time, the simultaneous splitting can be regarded as a two-step process. First, split it by rows, and then split each of the split sub-operators by columns. Combining the above specific implementation schemes of splitting by rows and splitting by columns, Fig. 9 First, split the 5*7 mvNode1 into three child mvNodes by row, and then split each by column.
[0181] From the above description of the implementation plan, it can be seen that this solution can effectively solve the problem of fusion of the calculation results of the split sub-operators when they are calculated on the storage and computing integrated chip, ensuring the correctness of the results.
[0182] Experiments have shown that the introduction of intermediate operators such as splitNode can improve the utilization and allocation efficiency of the flash memory cell array with limited overhead.
[0183] By adopting the above technical solution, the problem of being unable to deploy operators due to limited specifications of storage and computing integrated chips can be solved. Operators of any dimension that need to be calculated on the storage and computing chip can be split. It has a high degree of universality. In addition, it is highly portable to different chips and is applicable to chips of different specifications. It can also adapt to composite operators, such as bias and act processing. In addition, each split operator has its own unique structure with strong cohesion and weak coupling.
[0184] An embodiment of the present invention also provides a storage and computing integrated chip, including: a digital domain and an analog domain, the analog domain includes: a flash memory cell array, the flash memory cell array maps the data split according to the above-mentioned operator splitting method; the above-mentioned control method is executed in the digital domain.
[0185] It is worth noting that the integrated storage and computing chip provided in the embodiment of the present invention can be applied to various electronic devices, such as: smart phones, tablet electronic devices, network set-top boxes, portable computers, desktop computers, personal digital assistants (PDAs), vehicle-mounted devices, smart wearable devices, toys, smart home control devices, assembly line equipment controllers, etc. Among them, the smart wearable devices may include smart glasses, smart watches, smart bracelets, etc.
[0186] Based on the same inventive concept, the embodiments of the present application also provide an operator splitting device for a storage and computing integrated chip, which can be used to implement the method described in the above embodiment, as described in the following embodiments. The principle of solving the problem by the operator splitting device for the storage and computing integrated chip is similar to the above method, so the implementation of the operator splitting device for the storage and computing integrated chip can refer to the implementation of the above method, and the repeated parts will not be repeated. As used below, the term "unit" or "module" can be a combination of software and / or hardware that implements predetermined functions. Although the device described in the following embodiments is preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceived.
[0187] Fig.10 1 is a structural block diagram of an operator splitting device for a storage-computing integrated chip in an embodiment of the present invention. The operator splitting device for a storage-computing integrated chip comprises: a judgment module 10 and an operator splitting module 20.
[0188] The judgment module 10 is used to judge whether the number of rows of the target operator is greater than the number of rows of the maximum operation block, and whether the number of columns of the target operator is greater than the number of columns of the maximum operation block, and the number of rows and columns of the maximum operation block is a parameter of the flash memory cell array of the target storage and computing integrated chip;
[0189] The operator splitting module 20 splits the target operator into a plurality of sub-operators according to the judgment result, and the sub-operators are used to be mapped to the flash memory cell array to realize the operation of the target operator.
[0190] The devices, modules or units described in the above embodiments may be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is an electronic device, and the electronic device may be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.
[0191] In a typical example, the electronic device specifically includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, the steps of the above-mentioned operator splitting method for a storage-computing integrated chip are implemented.
[0192] Reference below Fig.11 , which shows a schematic structural diagram of an electronic device 600 suitable for implementing an embodiment of the present application.
[0193] like Fig.11 As shown, electronic device 600 includes a central processing unit (CPU) 601, which can perform various appropriate operations and processes according to a program stored in a read-only memory (ROM) 602 or a program loaded from a storage portion 608 into a random access memory (RAM) 603. In RAM 603, various programs and data required for the operation of system 600 are also stored. CPU 601, ROM 602, and RAM 603 are connected to each other via a bus 604. An input / output (I / O) interface 605 is also connected to bus 604.
[0194] The following components are connected to the I / O interface 605: an input section 606 including a keyboard, a mouse, etc.; an output section 607 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 608 including a hard disk, etc.; and a communication section 609 including a network interface card such as a LAN card, a modem, etc. The communication section 609 performs communication processing via a network such as the Internet. A drive 610 is also connected to the I / O interface 605 as needed. A removable medium 611, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 610 as needed, so that a computer program read therefrom is installed as needed as the storage section 608.
[0195] In particular, according to an embodiment of the present invention, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present invention includes a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the above-mentioned operator splitting method for a storage-computation integrated chip.
[0196] In such an embodiment, the computer program may be downloaded and installed from a network via the communication section 609 , and / or installed from the removable medium 611 .
[0197] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.
[0198] For the convenience of description, the above device is described in various units according to their functions. Of course, when implementing the present application, the functions of each unit can be implemented in the same or multiple software and / or hardware.
[0199] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.
[0200] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.
[0201] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process in the computer or other programmable device. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.
[0202] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.
[0203] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.
[0204] The present application may be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present application may also be practiced in distributed computing environments where tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules may be located in local and remote computer storage media, including storage devices.
[0205] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.
[0206] The above is only an embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.
Claims
1. A method for splitting operators for a storage-computing integrated chip, It is characterized in that include: Determine whether the number of rows of the target operator is greater than the number of rows of the maximum running block, and whether the number of columns of the target operator is greater than the number of columns of the maximum running block, where the number of rows and columns of the maximum running block is a parameter of the flash memory cell array of the storage-computing integrated chip; The target operator is split into multiple sub-operators according to the judgment result, including: If the number of columns of the target operator is not greater than the number of columns of the maximum running block and the number of rows of the target operator is greater than the number of rows of the maximum running block, the target operator is split into multiple sub-operators according to the number of rows of the maximum running block and the number of rows and columns of the target operator; If the number of columns of the target operator is greater than the number of columns of the maximum running block, split the target operator into multiple sub-operators according to the number of rows and columns of the maximum running block and the number of rows and columns of the target operator; The number of rows of the sub-operators is not greater than the number of rows of the maximum running block, and the number of columns of the sub-operators is not greater than the number of columns of the maximum running block.
2. The operator splitting method for a storage-computation integrated chip according to claim 1, It is characterized in that The step of splitting the target operator into a plurality of sub-operators according to the number of rows and columns of the maximum running block and the number of rows and columns of the target operator comprises: Obtaining the number of rows and columns of each sub-operator according to the number of rows and columns of the maximum running block and the number of rows and columns of the target operator; According to the number of rows and columns of each sub-operator, the target operator is sliced using the Slice technology to obtain the parameter value of each element of each sub-operator.
3. The operator splitting method for a storage-computation integrated chip according to claim 2, It is characterized in that The target operator is a multidimensional array stored in a row-first manner, and slicing is performed according to a certain dimension; The method of slicing the target operator according to the number of rows and columns of each sub-operator using the Slice technology to obtain the parameter value of each element of each sub-operator includes: Calculating the cardinality of the current dimension according to the target operator; The number of elements in a single continuous memory block is obtained according to the length of the slice required for the current dimension and the cardinality; Calculate the first address offset of the first continuous memory block according to the starting position of the slice and the cardinality; Obtaining the first address offset of each continuous memory block according to the length of the slice required for the current dimension, the cardinality, and the first address offset of the first continuous memory block; Calculating the number of continuous memory blocks according to the target operator size and the number of elements in a single continuous memory block; The parameter value of each element of each sub-operator is obtained according to the first address offset of each continuous memory block, the length of the slice required for the current dimension, and the number of continuous memory blocks.
4. The operator splitting method for a storage-computation integrated chip according to claim 3, It is characterized in that The calculating the cardinality of the current slice dimension according to the target operator includes: The cardinality is obtained by calculating the product of the number of elements in all dimensions after the current slice dimension of the target operator.
5. The operator splitting method for a storage-computation integrated chip according to claim 2, It is characterized in that The acquiring the number of rows and columns of each sub-operator according to the number of rows and columns of the maximum running block and the number of rows and columns of the target operator includes: If the number of rows of the target operator is greater than the number of rows of the maximum running block, split the target operator into rows and columns to obtain the number of rows and columns of each sub-operator; If the number of rows of the target operator is not greater than the number of rows of the maximum running block, the target operator is split by columns to obtain the number of rows and columns of each operator.
6. The operator splitting method for a storage-computation integrated chip according to claim 1, It is characterized in that The step of splitting the target operator into a plurality of sub-operators according to the number of rows of the maximum running block and the number of rows and columns of the target operator comprises: Splitting the target operator by rows according to the number of rows of the maximum running block and the number of rows and columns of the target operator to obtain the number of rows and columns of each sub-operator; Obtain the parameter value of each element of each sub-operator according to the number of rows and columns of each sub-operator.
7. The operator splitting method for a storage-computation integrated chip according to claim 1, It is characterized in that Also includes: The bias array corresponding to the target operator is combined with the corresponding sub-operator to obtain the data to be finally mapped to the flash memory cell array.
8. The operator splitting method for a storage-computation integrated chip according to claim 7, It is characterized in that The combining the offset array corresponding to the target operator to the corresponding sub-operator includes: If the sub-operator is obtained by splitting the target operator by rows, the offset array corresponding to the target operator is combined on any sub-operator; If the sub-operator is obtained by splitting the target operator by columns, split the offset array corresponding to the target operator by columns and combine them with the corresponding sub-operator; If the sub-operator is obtained by splitting according to rows and columns, the offset array corresponding to the target operator is split according to columns and then combined with any sub-operator of the corresponding column.
9. An operator splitting device for a storage and computing integrated chip, It is characterized in that include: A judgment module, judging whether the number of rows of the target operator is greater than the number of rows of the maximum running block, and whether the number of columns of the target operator is greater than the number of columns of the maximum running block, wherein the number of rows and columns of the maximum running block is a parameter of the flash memory cell array of the storage-computing integrated chip; An operator splitting module is used to split the target operator into multiple sub-operators according to the judgment result, and the sub-operators are used to map to the flash memory cell array to realize the operation of the target operator. The step of splitting the target operator into multiple sub-operators according to the judgment result includes: If the number of columns of the target operator is not greater than the number of columns of the maximum running block and the number of rows of the target operator is greater than the number of rows of the maximum running block, the target operator is split into multiple sub-operators according to the number of rows of the maximum running block and the number of rows and columns of the target operator; If the number of columns of the target operator is greater than the number of columns of the maximum running block, split the target operator into multiple sub-operators according to the number of rows and columns of the maximum running block and the number of rows and columns of the target operator; The number of rows of the sub-operators is not greater than the number of rows of the maximum running block, and the number of columns of the sub-operators is not greater than the number of columns of the maximum running block.
10. A control method for a storage-computing integrated chip, It is characterized in that include: The operation result of the previous operator is split according to a preset splitting rule and then loaded into corresponding multiple flash memory unit sub-arrays, each of which has a corresponding sub-operator pre-stored on the flash memory unit sub-array, each sub-operator is obtained by splitting according to the splitting method according to any one of claims 1 to 8, and the preset splitting rule is set according to the operator splitting rule for splitting the current operator into multiple sub-operators; The operation results of each sub-operator are obtained and combined to obtain the operation result of the current operator.
11. The control method for a storage-computing integrated chip according to claim 10, It is characterized in that The operation result of the previous operator is a multidimensional array stored in a row-first manner; If the preset splitting rule involves splitting by column, the Slice technology is used to slice the operation result of the previous operator according to a certain dimension to obtain the parameter value of each element of the split operator result.
12. The control method for a storage-computing integrated chip according to claim 11, It is characterized in that The method of using the Slice technology to slice the operation result of the previous operator according to a certain dimension to obtain the parameter value of each element of the split operator result includes: Calculate the cardinality of the current dimension according to the operation result; The number of elements in a single continuous memory block is obtained according to the length of the slice required for the current dimension and the cardinality; Calculate the first address offset of the first continuous memory block according to the starting position of the slice and the cardinality; Obtaining the first address offset of each continuous memory block according to the length of the slice required for the current dimension, the cardinality, and the first address offset of the first continuous memory block; Calculating the number of continuous memory blocks according to the size of the operation result and the number of elements in a single continuous memory block; The parameter value of each element of each operator result is obtained according to the first address offset of each continuous memory block, the length of the slice required for the current dimension, and the number of continuous memory blocks.
13. The control method for a storage-computing integrated chip according to claim 12, It is characterized in that The calculating the cardinality of the current slice dimension according to the operation result includes: The cardinality is obtained by calculating the product of the number of elements of all dimensions after the current slice dimension of the operation result.
14. The control method for a storage-computing integrated chip according to claim 10, It is characterized in that Also includes: After combining the calculation results of each sub-operator, the activation operation is performed using the preset activation function.
15. The control method for a storage-computing integrated chip according to claim 10, It is characterized in that When combining the operation results of each sub-operator, they are combined according to the principle of row splitting and addition, and column splitting and sequential connection.
16. A storage and computing integrated chip, It is characterized in that include: A digital domain and an analog domain, wherein the analog domain comprises: a flash memory cell array, in which the data split by the operator splitting method according to any one of claims 1 to 8 is mapped; The control method according to any one of claims 10 to 15 is executed in the digital domain.
17. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, It is characterized in that When the processor executes the program, the steps of the operator splitting method according to any one of claims 1 to 8 are implemented.
18. A computer-readable storage medium having a computer program stored thereon, It is characterized in that When the computer program is executed by a processor, the steps of the operator splitting method described in any one of claims 1 to 8 are implemented.
Citation Information
Patent Citations
Model training method and related equipment
CN112541159A
Neural network weight matrix adjustment method, writing control method, and related device
WO2021163866A1
