Control method, related apparatus and readable storage medium

Abstract

The present application relates to the technical field of terminals, and particularly relates to a control method, a related apparatus and a readable storage medium. The method can comprise: acquiring data to be processed; when the quantization precision of said data is greater than the computation precision of a computing unit, performing data splitting on said data to obtain a plurality of pieces of sub-data, wherein the quantization precision of each of the plurality of pieces of sub-data is less than or equal to the computation precision of the computing unit; and sending a first memory access instruction for each of the plurality of pieces of sub-data according to a storage rule, wherein the first memory access instruction is used for instructing an internal memory to write the sub-data into a specified row or column. The embodiments of the present application can be compatible with computations with mixed quantization precisions and also save on circuit area overheads.

Description

A control method, related device and readable storage medium

[0001] The present application claims priority to the Chinese Patent Application No. 202411381085.2, filed on September 29, 2024, entitled “A control method, related device and readable storage medium”, and the Chinese Patent Application No. 202511394587.3, filed on September 26, 2025, entitled “A control method, related device and readable storage medium”, the contents of which are incorporated herein by reference in their entirety. TECHNICAL FIELD

[0002] The present application relates to the technical field of terminals, and in particular to a control method, related device and readable storage medium. BACKGROUND

[0003] With the rise of generative AI such as ChatGPT, deploying large models for AI inference on the terminal side to improve user experience and productivity is one of the main methods to improve product competitiveness. When large models are inferred, model parameters are generally loaded into memory, so memory capacity often limits the size of the model that can be deployed on the terminal side. At the same time, a large amount of data needs to be moved from memory to the processor (such as CPU, NPU, GPU, etc.) for processing during large model inference, and memory bandwidth is also a bottleneck that limits the inference performance on the terminal side.

[0004] In order to reduce the demand of large models for hardware resources, an important direction of algorithm optimization is quantization technology. Quantization technology can reduce model size by quantizing model parameters, thereby reducing the demand of large models for hardware resources, speeding up inference and reducing power consumption, but quantization may cause a loss of precision in the model. Therefore, hybrid quantization precision can be used to allow the model to use higher precision for quantization in key parameter parts and lower precision for quantization in other parts, thereby reducing the demand of large models for hardware resources while maintaining the capabilities of large models. In addition, in order to reduce data movement during inference, processing in memory (PIM) technology can be used to offload part of the computation to the memory, such as PIM-DRAM, which integrates independent computing units in the DRAM chip (Die) to implement processing in memory. By using memory to assist in calculation and utilizing the internal multi-Bank concurrency capability of the DRAM memory, the computing performance can be improved, the large model inference can be accelerated, the energy consumption can be reduced, and the endurance of the terminal product can be improved.

[0005] At present, the scheme of in-memory computing memory (PIM-DRAM) compatible hybrid quantization precision calculation needs to design the calculation circuit with corresponding calculation precision respectively for different quantization precisions, and there may be a problem of reserving more calculation circuits, causing additional circuit area overhead.

[0006] Therefore, the industry is exploring how to provide a scheme that can be compatible with hybrid quantization precision calculation while saving circuit area overhead. SUMMARY

[0007] The embodiments of the present application provide a control method, related device and readable storage medium, which can be compatible with hybrid quantization precision calculation while saving circuit area overhead.

[0008] The present application will be described from different aspects below. It should be understood that the implementation and advantages of the different aspects below can be mutually referred to.

[0009] In a first aspect, the present application provides a control method, which can be applied to a processor, the processor being coupled with an in-memory, the in-memory including at least one storage area Bank and at least one calculation unit; each Bank including M rows*N columns of storage units, the calculation precision of each calculation unit being K, M, N and K all being greater than 0; the method including:

[0010] obtaining to-be-processed data; the quantization precision of the to-be-processed data being L;

[0011] when the quantization precision L of the to-be-processed data is greater than the calculation precision K of the calculation unit, performing data splitting on the to-be-processed data to obtain a plurality of sub-data; the quantization precision of each sub-data being less than or equal to the calculation precision K of the calculation unit;

[0012] sending a first memory access instruction for each sub-data according to a storage rule; the first memory access instruction being used to instruct the in-memory to write the sub-data into a specified row or column (the specified row or column being different between different first memory access instructions in the plurality of first memory access instructions).

[0013] In the embodiments of the present application, when the quantization precision of the to-be-processed data is greater than the calculation precision of the calculation unit in the in-memory calculation storage, the subsequent in-memory storage cannot directly calculate the to-be-processed data through the calculation unit. The to-be-processed data can be first split into multiple sub-data, and the multiple sub-data are respectively written into different rows or columns of the storage area for storage. The quantization precision of each sub-data is less than or equal to the calculation precision of the calculation unit, so that the in-memory storage can subsequently calculate each sub-data through the calculation unit. For calculation compatible with mixed quantization precision, only the strategy of data splitting needs to be adjusted according to different combinations of quantization precision and calculation precision, so that the calculation circuit with one kind of calculation precision can complete the calculation of multiple quantization precision parameters, and there is no need to specially design a calculation circuit with a corresponding calculation precision for data with different quantization precision, thereby saving the area overhead of the calculation circuit.

[0014] In a possible implementation, the storage rule specifically includes that the multiple sub-data are written into adjacent rows or columns; or the Bank includes multiple partitions, and the storage rule specifically includes that the specified rows or columns where the multiple sub-data are written are located in different partitions; or the storage rule specifically includes that the specified rows or columns where the multiple sub-data are written are located in different Banks.

[0015] In the embodiments of the present application, the multiple sub-data after splitting can be stored in adjacent rows or columns, so that the calculation of the data stored in adjacent rows or columns can be completed through a single calculation instruction in subsequent calculation, and the scenario of calculating the calculation result of a small amount of row or column data is applicable; or the multiple sub-data after splitting can be stored in rows or columns of multiple independent partitions, so that the calculation of the data stored in multiple partitions can be completed through a single calculation instruction in subsequent calculation, and the scenario of calculating the calculation result of a large amount of row or column data is applicable. Alternatively, the multiple sub-data after splitting can be stored in rows or columns of different Banks, so that the data storage can be flexibly configured, and the sub-data in different Banks can be calculated in parallel in subsequent data calculation, and the efficiency is higher.

[0016] In a possible implementation, the maximum amount of data that can be stored in the specified row or column is an integer multiple of the quantization precision of the sub-data.

[0017] In the embodiments of the present application, the maximum amount of data that can be stored in the row or column storing the sub-data is an integer multiple of the quantization precision of the sub-data, for example, the maximum amount of data that can be stored is 256 bits, and the quantization precision of the sub-data can be 1 bit, 2 bits, 4 bits, etc., so as to ensure that the storage resources of the row or column can be fully utilized.

[0018] In a possible implementation, the method further includes:

[0019] sending a second memory access instruction; the second memory access instruction is used to instruct the memory to perform calculation on the plurality of sub-data.

[0020] In the embodiment of the application, after instructing the memory to store the to-be-processed data, the processor can instruct the memory to perform calculation on the to-be-processed data by using the internal calculation unit through the memory access instruction.

[0021] In a possible implementation, the second memory access instruction is further used to instruct a quantization precision L of the to-be-processed data.

[0022] In the embodiment of the application, the processor can enable the memory to perceive the quantization precision of the to-be-processed data through the memory access instruction.

[0023] In a possible implementation, the memory further includes a mode register, and the mode register includes an instruction field used to instruct the quantization precision of data; the method further includes:

[0024] sending a control instruction; the control instruction is used to instruct the quantization precision L of the to-be-processed data, and instruct the memory to save the quantization precision L of the to-be-processed data in the instruction field of the mode register.

[0025] In the embodiment of the application, the processor can enable the memory to perceive the quantization precision of the to-be-processed data by configuring the mode register in the memory.

[0026] In a possible implementation, the method further includes:

[0027] when the quantization precision L of the to-be-processed data is less than or equal to the calculation precision K of the calculation unit, sending a third memory access instruction; the third memory access instruction is used to instruct the memory to write and store the to-be-processed data in a specified row or column.

[0028] In the embodiment of the application, when the quantization precision of the to-be-processed data is less than or equal to the calculation precision of the calculation unit, the processor can directly write and store the to-be-processed data in the specified row / column through the memory access instruction without data splitting. Correspondingly, in the subsequent calculation process, the memory can perform calculation on the to-be-processed data stored in the specified row / column through the calculation unit, and directly obtain the final calculation result.

[0029] In a possible implementation, the method further includes:

[0030] obtaining an initial weight parameter of a model;

[0031] dividing the initial weight parameter to obtain one or more first parameter sets;

[0032] determining one or more second parameter sets from the one or more first parameter sets; initial weight parameters included in the second parameter sets support storage into the internal memory;

[0033] determining one or more third parameter sets from the one or more second parameter sets; quantization precision of initial weight parameters included in the third parameter sets is greater than calculation precision of the calculation unit.

[0034] In the embodiments of the present application, after the processor obtains the initial weight parameters of the model, the model can be analyzed first, the initial weight parameters with the same characteristics are divided together, and it is determined which parameters can be stored in the internal memory, and it is further determined which parameters need to be split, facilitating subsequent model loading and calculation.

[0035] In a possible implementation, the plurality of sub-data includes first sub-data and second sub-data, the first sub-data is one or more high-bit data in the to-be-processed data, and the second sub-data is one or more low-bit data in the to-be-processed data; the method further includes:

[0036] receiving a fifth result and a sixth result output by the internal memory; the fifth result is determined based on the first sub-data, and the sixth result is determined based on the second sub-data;

[0037] determining a seventh result based on a shift coefficient and the fifth result; the shift coefficient is determined based on an initial position of the first sub-data in the to-be-processed data;

[0038] determining a final result based on the sixth result and the seventh result.

[0039] In the embodiments of the present application, the shift processing of the intermediate result corresponding to the high-bit data can be performed by the processor, and can be applicable to a scenario in which no shift circuit is arranged in the calculation unit.

[0040] In a second aspect, the present application provides a control method, which can be applied to an internal memory, the internal memory is coupled with a processor, the internal memory includes at least one storage area Bank and at least one calculation unit; each Bank includes M rows*N columns of storage units, and the calculation precision of each calculation unit is K, M, N and K are all greater than 0; the method includes:

[0041] receive the second memory access instruction sent by the processor; the second memory access instruction is used to instruct the memory to perform calculation on a plurality of sub-data; the plurality of sub-data are a plurality of sub-data obtained by the processor performing data splitting on the to-be-processed data when quantization precision L of the to-be-processed data is greater than calculation precision K of the calculation unit; the quantization precision of the sub-data is less than or equal to the calculation precision K of the calculation unit;

[0042] determine storage addresses of the plurality of sub-data according to the second memory access instruction and a storage rule;

[0043] read the plurality of sub-data from the storage addresses in sequence in response to the second memory access instruction;

[0044] perform calculation using the plurality of sub-data to determine a final result.

[0045] In the embodiment of the application, when the memory receives the calculation instruction sent by the processor and the quantization precision of the to-be-processed data is greater than the calculation precision of the calculation unit in the in-memory calculation memory, the calculation unit can perform calculation on the sub-data stored in a plurality of different rows / columns respectively to determine the final result, so that the purpose of completing high-quantization-precision data calculation by using a low-calculation-precision calculation unit is achieved, and circuit area overhead is saved.

[0046] In a possible implementation, the plurality of sub-data includes first sub-data and second sub-data, the first sub-data is one or more high-order data in the to-be-processed data, and the second sub-data is one or more low-order data in the to-be-processed data; and the performing calculation using the plurality of sub-data to determine the final result includes:

[0047] performing multiplication calculation on the first sub-data and then performing shift addition to obtain a first result;

[0048] performing multiplication-addition calculation on the second sub-data to obtain a second result;

[0049] determining the final result based on the first result and the second result.

[0050] In the embodiment of the application, part of the plurality of sub-data is high-order data of the to-be-processed data, and part of the plurality of sub-data is low-order data of the to-be-processed data. After multiplication calculation is performed on the high-order data, shift addition processing is performed to obtain an intermediate result corresponding to the high-order data. Multiplication-addition calculation is performed on the low-order data without shift processing to obtain an intermediate result corresponding to the low-order data. Then, the final result is determined based on the two intermediate results, so that the accuracy of calculation is ensured.

[0051] In a possible implementation, the plurality of sub-data includes first sub-data and second sub-data, the first sub-data is one or more high-order data in the to-be-processed data, and the second sub-data is one or more low-order data in the to-be-processed data; and the calculating the final result by using the plurality of sub-data includes:

[0052] performing a multiplication-addition calculation on the first sub-data and then performing a shift processing to obtain a third result;

[0053] performing a multiplication-addition calculation on the second sub-data to obtain a fourth result;

[0054] determining the final result based on the third result and the fourth result.

[0055] In the embodiments of the present application, part of the plurality of sub-data is high-order data of the to-be-processed data, and part of the plurality of sub-data is low-order data of the to-be-processed data. After performing a multiplication-addition calculation on the high-order data, a shift processing is performed to obtain an intermediate result corresponding to the high-order data. After performing a multiplication-addition calculation on the low-order data, a shift processing is not performed to obtain an intermediate result corresponding to the low-order data. Then, the final result is determined based on the two intermediate results, so as to ensure the accuracy of the calculation.

[0056] In a possible implementation, the shift bit number of each intermediate result in the plurality of intermediate results corresponding to the plurality of sub-data is determined based on an initial position of the corresponding sub-data in the to-be-processed data.

[0057] In the embodiments of the present application, when the memory performs a shift processing on the plurality of intermediate results by using the shift circuit, the initial position of each intermediate result in the original data (the to-be-processed data before being split) corresponding to the sub-data can be determined, so as to ensure the accuracy of the final result.

[0058] In a possible implementation, the storage rule specifically includes that the specified row or column to which the plurality of sub-data is written is an adjacent row or column; or the Bank includes a plurality of partitions, and the storage rule specifically includes that the specified row or column to which the plurality of sub-data is written is located in different partitions; or the storage rule specifically includes that the specified row or column to which the plurality of sub-data is written is located in different Banks.

[0059] In a possible implementation, the maximum amount of data that can be stored in the specified row or column to which the sub-data is written is an integer multiple of the quantization precision of the sub-data.

[0060] In a possible implementation, the second memory access instruction is further used to indicate the quantization precision L of the to-be-processed data.

[0061] In a possible implementation, the internal memory further includes a mode register, and the mode register includes an indication field used to indicate data quantization precision; and the method further includes:

[0062] receiving the control instruction sent by the processor; the control instruction is used to indicate the quantization precision L of the to-be-processed data, and indicate that the internal memory saves the quantization precision L of the to-be-processed data in the indication field of the mode register.

[0063] In a possible implementation, the method further includes:

[0064] when the quantization precision L of the to-be-processed data is less than or equal to the calculation precision K of the calculation unit, receiving the third memory access instruction sent by the processor; the third memory access instruction is used to instruct the internal memory to write and store the to-be-processed data in a specified row or column.

[0065] In a possible implementation, when the calculation precision K of the calculation unit is J times of the quantization precision L of the to-be-processed data, J is an integer greater than 1; and the method further includes:

[0066] dividing the calculation resource of the calculation unit into J sub-resources;

[0067] receiving J to-be-processed data;

[0068] processing the J to-be-processed data in parallel through the J sub-resources; and the quantization precision of the to-be-processed data is L.

[0069] In the embodiments of the present application, when the quantization precision of the to-be-processed data is less than the calculation precision of the calculation unit, and the calculation precision is an integer multiple of the quantization precision, the calculation resource of the calculation unit can be divided into multiple parts during calculation, so that the multiple resources of the calculation unit can perform calculation at the same time, and the calculation efficiency is improved.

[0070] In a possible implementation, the plurality of sub-data includes first sub-data and second sub-data, the first sub-data is one or more high-order data in the to-be-processed data, and the second sub-data is one or more low-order data in the to-be-processed data; and the using the plurality of sub-data to perform calculation to determine the final result includes:

[0071] performing calculation on the first sub-data to obtain a fifth result;

[0072] performing calculation on the second sub-data to obtain a sixth result;

[0073] outputting the fifth result and the sixth result to the processor.

[0074] In the embodiments of the present application, the shift processing of the intermediate result corresponding to the high-bit data can be performed by the processor, and can be applicable to the scenario where no shift circuit is arranged in the calculation unit.

[0075] In a third aspect, the present application provides a control method, which can be applied to an internal memory. The internal memory is coupled with a processor. The internal memory includes at least one storage area Bank and at least one calculation unit. Each Bank includes M rows*N columns of storage units. The calculation accuracy of each calculation unit is K. M, N, and K are all greater than 0. The method includes:

[0076] When the calculation accuracy K of the calculation unit is J times of the quantization accuracy L of the to-be-processed data, the calculation resource of the calculation unit is divided into J sub-resources. J is an integer greater than 1.

[0077] J to-be-processed data are received.

[0078] The J to-be-processed data are processed in parallel by the J sub-resources. The quantization accuracy of the to-be-processed data is L.

[0079] In a fourth aspect, the present application provides a processor. The processor is coupled with an internal memory. The internal memory includes at least one storage area Bank and at least one calculation unit. Each Bank includes M rows*N columns of storage units. The calculation accuracy of each calculation unit is K. M, N, and K are all integers greater than 0. The processor is configured to:

[0080] to-be-processed data are acquired. The quantization accuracy of the to-be-processed data is L.

[0081] When the quantization accuracy L of the to-be-processed data is greater than the calculation accuracy K of the calculation unit, the to-be-processed data are split to obtain a plurality of sub-data. The quantization accuracy of each sub-data is less than or equal to the calculation accuracy K of the calculation unit.

[0082] First memory access instructions are respectively sent to the plurality of sub-data according to a storage rule. The first memory access instructions are used to instruct the internal memory to write the sub-data into specified rows or columns. The specified rows or columns of different first memory access instructions in the plurality of first memory access instructions are different.

[0083] In a fifth aspect, the present application provides an internal memory. The internal memory is coupled with a processor. The internal memory includes at least one storage area Bank and at least one calculation unit. Each Bank includes M rows*N columns of storage units. The calculation accuracy of each calculation unit is K. M, N, and K are all greater than 0. The method includes:

[0084] receive the second memory access instruction sent by the processor; the second memory access instruction is used to instruct the internal memory to perform calculation on a plurality of sub-data; the plurality of sub-data are a plurality of sub-data obtained by the processor performing data splitting on the to-be-processed data when a quantization precision L of the to-be-processed data is greater than a calculation precision K of the calculation unit; the quantization precision of the sub-data is less than or equal to the calculation precision K of the calculation unit;

[0085] determine storage addresses of the plurality of sub-data according to the second memory access instruction and a storage rule;

[0086] read the plurality of sub-data from the storage addresses in sequence in response to the second memory access instruction;

[0087] perform calculation using the plurality of sub-data to determine a final result.

[0088] In a sixth aspect, the present application provides an electronic device, which is characterized by comprising one or more processors and one or more internal memories; the one or more processors are coupled with the one or more internal memories; the one or more processors are configured to perform the method described in any possible implementation manner of the first aspect, and the one or more internal memories are configured to perform the method described in any possible implementation manner of the second aspect or the third aspect.

[0089] In a seventh aspect, the present application provides a computer readable storage medium, which is characterized by storing a computer program or instructions; when the computer program or instructions are executed by a computer, the method described in any possible implementation manner of the first aspect, the second aspect or the third aspect is implemented.

[0090] In an eighth aspect, the present application provides a computer program, which is characterized by comprising instructions; when the computer program is executed by a computer, the method described in any possible implementation manner of the first aspect, the second aspect or the third aspect is implemented.

[0091] Optionally, the processor and the internal memory can be physically independent units, or the memory can be integrated with the processor.

[0092] It should be noted that the technical effects achieved by the above aspects can be mutually referred to or referred to the beneficial effects of the method embodiments shown below, which will not be described here. BRIEF DESCRIPTION OF DRAWINGS

[0093] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the background art, the drawings needed to be used in the embodiments of the present application or the background art will be described below.

[0094] FIG. 1 is a structural schematic diagram of an in-memory computing memory;

[0095] FIG. 2A is a structural schematic diagram of an electronic device provided by an embodiment of the present application;

[0096] FIG. 2B is a structural schematic diagram of a memory structure using in-memory computing technology provided by an embodiment of the present application;

[0097] FIG. 2C is a structural schematic diagram of a PIM-Die provided by an embodiment of the present application;

[0098] FIG. 2D is a structural schematic diagram of a Bank provided by an embodiment of the present application;

[0099] FIG. 2E is a structural schematic diagram of a computing unit provided by an embodiment of the present application;

[0100] FIG. 2F is a structural schematic diagram of another computing unit provided by an embodiment of the present application;

[0101] FIG. 3 is a flow schematic diagram of a control method provided by an embodiment of the present application;

[0102] FIG. 4 is a flow schematic diagram of another control method provided by an embodiment of the present application;

[0103] FIG. 5 is a data computing flow schematic diagram of an INT2 computing precision and INT2 quantization precision combination provided by an embodiment of the present application;

[0104] FIG. 6 is a data splitting schematic diagram of an INT8 quantization precision provided by an embodiment of the present application;

[0105] FIGS. 7A-7D are computing flow schematic diagrams of data splitting provided by an embodiment of the present application;

[0106] FIG. 8A is a data splitting storage schematic diagram of an INT2 computing precision and INT4 quantization precision combination provided by an embodiment of the present application;

[0107] FIG. 8B is a data computing flow schematic diagram of an INT2 computing precision and INT4 quantization precision combination provided by an embodiment of the present application;

[0108] FIG. 8C is another data splitting storage schematic diagram of an INT2 computing precision and INT4 quantization precision combination provided by an embodiment of the present application;

[0109] FIG. 8D is a data computing flow schematic diagram of another INT2 computing precision and INT4 quantization precision combination provided by an embodiment of the present application;

[0110] FIG. 8E is a data computing flow schematic diagram of still another INT2 computing precision and INT4 quantization precision combination provided by an embodiment of the present application;

[0111] FIG. 9A is a data split storage diagram of a combination of INT2 calculation precision and INT3 quantization precision according to an embodiment of the present application;

[0112] FIG. 9B is another data split storage diagram of a combination of INT2 calculation precision and INT3 quantization precision according to an embodiment of the present application;

[0113] FIG. 10A is a data split storage diagram of a combination of INT1 calculation precision and INT3 quantization precision according to an embodiment of the present application;

[0114] FIG. 10B is another data split storage diagram of a combination of INT1 calculation precision and INT3 quantization precision according to an embodiment of the present application;

[0115] FIG. 11 is a flow diagram of model analysis according to an embodiment of the present application;

[0116] FIG. 12 is a flow diagram of model loading according to an embodiment of the present application;

[0117] FIG. 13 is a data calculation flow diagram of shifting based on a processor according to an embodiment of the present application;

[0118] FIG. 14A is another data calculation flow diagram of shifting based on a processor according to an embodiment of the present application;

[0119] FIG. 14B is yet another data calculation flow diagram of shifting based on a processor according to an embodiment of the present application. DETAILED DESCRIPTION

[0120] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application.

[0121] In the description of the present application, the terms "first", "second", etc. are only used to distinguish different objects, and do not limit the quantity and execution order, and the terms "first", "second", etc. also do not necessarily distinguish different objects. For example, the first access instruction and the second access instruction are only used to distinguish different access instructions, and do not limit the order. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device, etc. including a series of steps or units, is not limited to the listed steps or units, but optionally includes other steps or units not listed, or optionally includes other steps or units inherent to the process, method, product or device, etc.

[0122] In the description of the present application, unless otherwise specified, " / " means "or", for example, A / B can mean A or B. "And / or" in this article is only a description of the relationship between the associated objects, which means that there can be three relationships, for example, A and / or B, which can mean: A exists alone, A and B exist together, B exists alone, and the like. In addition, "at least one item", "one or more items" or the like means any combination of these items, including any combination of single item or multiple items. For example, at least one of a, b, or c can mean: a, b, c; a and b; a and c; b and c; or a and b and c. Where a, b, and c can be single or multiple.

[0123] In the description of the present application, the words "exemplary", "exemplarily" or "for example" are used to mean an example, illustration or description. Any embodiment or design solution described in the present application as "exemplary", "for example" or "for example" should not be interpreted as more preferred or more advantageous than other embodiments or design solutions. Rather, the use of "exemplary", "for example" or "for example" is intended to present the relevant concept in a specific way.

[0124] It can be understood that in the description of the present application, "when", "if" and "if" all mean that the device will make corresponding processing under certain objective conditions, not limited to time, and also does not require the device to have a judgment action when it is implemented, nor does it mean that there are other limitations.

[0125] In the present application, "at the same time" can be understood as at the same time point, also can be understood as in a period of time, also can be understood as in the same cycle, which can be understood in combination with the context.

[0126] In the present application, the element expressed by the singular is intended to represent "one or more", rather than "one and only one", unless otherwise specified.

[0127] It can be understood that in each embodiment of the present application, "A and B correspond" means that B is associated with A, and B can be determined according to A. However, it should also be understood that the determination of B according to A does not mean that B is determined only according to A, but B can also be determined according to A and / or other information.

[0128] It can be understood that, in the embodiments of the present application, "for indicating" and "indicating" can include direct indication and indirect indication, and can also include explicit indication and implicit indication. When describing "the indication information for indicating A" or "the indication information of A", it can include that the indication information directly indicates A or indirectly indicates A, and it does not mean that A must be carried in the indication information. The information indicated by certain information is called to be indicated information, and in the specific implementation process, there are many ways to indicate the to-be-indicated information, for example but not limited to, the to-be-indicated information can be directly indicated, such as the to-be-indicated information itself or the index of the to-be-indicated information. The to-be-indicated information can also be indirectly indicated by indicating other information, where the other information and the to-be-indicated information have an association relationship. The to-be-indicated information can also be only indicated a part of the to-be-indicated information, and the other part of the to-be-indicated information is known or agreed in advance. For example, the indication of a specific information can also be achieved by means of the arrangement order of each information agreed in advance (for example, a protocol). Thus, to a certain extent, the indication overhead is reduced. At the same time, the common part of each information can be identified and uniformly indicated, so as to reduce the indication overhead caused by separately indicating the same information. In addition, the specific indication manner can also be various existing indication manners, for example but not limited to, the above indication manners and various combinations thereof. The specific details of various indication manners can refer to the prior art, and will not be described herein. As described above, for example, when multiple information of the same type needs to be indicated, the indication manner of different information can be different. In the specific implementation process, the required indication manner can be selected according to the specific needs, and the selected indication manner is not limited in the embodiments of the present application. In this way, the indication manner involved in the embodiments of the present application should be understood as covering various methods that can enable the to-be-indicated party to know the to-be-indicated information. The to-be-indicated information can be sent as a whole, or can be sent separately into multiple sub-information, and the sending period and / or sending time of the sub-information can be the same or different. The specific sending method is not limited in the present application.

[0129] In order to better understand the technical solutions of the embodiments of the present application, the following briefly introduces several terms or names related to the present application, so as to facilitate the understanding of those skilled in the art.

[0130] (1) Dynamic random access memory (DRAM), generally includes a plurality of memory areas (Banks), each Bank can include a plurality of rows and a plurality of columns of memory cells. DRAM is usually used as memory, accessing the memory cells of the memory requires two steps, one is to find the address of a certain row in the Bank, which can be specified by a row address command to access the Bank and the row address; two is to find the address of a certain column in the selected row, which can be specified by a column address command to access the col address. Generally, the last operation on the row is the precharge operation, which is also called closing the open row. After this operation, a new row can be accessed.

[0131] (2) The basic commands commonly used to access memory include activation, read, write, precharge and refresh. Among them, the activation command can be used to select a row in the memory cell in the idle state to enter the state ready to read / write. The read and write commands can be used to read and write data in a specific column of the activated row, respectively. The precharge command can be used to close the originally opened row after the read / write command is executed if other rows are to be addressed. The refresh command can be used to refresh all rows according to a fixed period to prevent long-term storage capacitor charge leakage, resulting in data loss.

[0132] (3) PIM-DRAM, compared with ordinary DRAM, in addition to the storage area for storing data and other peripheral circuits for reading and writing data and other structures, also integrates an independent computing unit (PU). PIM-DRAM can be used as ordinary DRAM, during use, the computing logic can be bypassed, and data reading and writing can be directly performed; or by issuing a computing command, the computing unit in the PIM-DRAM can be used for in-memory computing to reduce data movement. In the embodiment of the present application, when the data precision of the to-be-processed data is greater than the computing precision of the computing unit in the in-memory computing memory, the to-be-processed data can be split into multiple sub-data, and the multiple sub-data are respectively written into different rows or columns of the memory area for storage, and the data precision of each sub-data is less than or equal to the computing precision of the computing unit. After receiving the computing instruction, the multiple sub-data stored in the multiple different rows / cols can be taken out and input into the computing unit for computing to obtain multiple intermediate results, and the multiple intermediate results are input into the accumulator for calculation after being shifted by the shift circuit, thereby achieving the purpose of completing high-precision data calculation by a low-precision computing unit and saving circuit area overhead.

[0133] (4) Quantization technology is a method to reduce model calculation and storage overhead by reducing the number of bits representing model parameters. Traditional deep learning models usually use floating-point numbers to represent parameters and activation values, which can achieve higher accuracy but also higher computational cost. The goal of quantization technology is to reduce the number of bits representing parameters while maintaining model performance, thereby achieving higher efficiency in terms of computation and storage.

[0134] The core idea of quantization technology is to map parameters and activation values in the model to a smaller value range, usually finite integers or low-bit floating-point numbers. This mapping can be achieved in various ways, such as fixed-point quantization, floating-point quantization, and symmetric quantization. In fixed-point quantization, parameters and activation values are mapped to integers within a fixed range. By limiting the value range, fewer bits can be used to represent each value, reducing storage and computation overhead. Floating-point quantization is to map parameters and activation values to lower-bit floating-point representations. For example, half-precision floating-point numbers (16 bits) can be used instead of standard single-precision floating-point numbers (32 bits), reducing the number of bits representing parameters. Symmetric quantization is a special quantization method, where parameters and activation values are symmetric around zero, which can balance the positive and negative values of the quantized values, thereby reducing information loss. However, quantization may cause a loss of model accuracy. To reduce the demand for hardware resources while maintaining model capabilities, hybrid quantization precision can be used, allowing the model to use higher precision for key parameters and lower precision for other parts.

[0135] First, the technical problem to be solved by the present application is analyzed and proposed. Referring to FIG. 1, a PIM-DRAM stores processing units near the Banks inside. The weight parameters W of a large model can be stored in the Banks, and the input vector V can be transmitted to the PIM-DRAM by the host (or processor) and stored in the global buffer. To complete the multiplication and accumulation operation of W*V, the processing unit can include multiplication circuits (including multiple multipliers), addition circuits (including multiple adders), and accumulators. Taking a column inside the PIM-DRAM as an example, when the quantization precision of the weight parameter is Integer 2bit (INT2), the column can store 128 (i.e. 256 / 2) weight parameters; when the quantization precision of the weight parameter is Integer 4bit (INT4), the column can store 64 (i.e. 256 / 4) weight parameters. Therefore, when the PIM-DRAM needs to support the calculation of INT2 / INT4 mixed quantization precision, the processing unit needs to include at least 128 INT4 calculation circuits. When the quantization precision of the weight parameter is INT2, 128 INT4 calculation circuits are needed to perform the calculation, but only 2 bits (such as high 2 bits or low 2 bits) of each INT4 calculation circuit are used for calculation; when the quantization precision of the weight parameter is INT4, 64 INT4 calculation circuits are needed to perform the calculation, and all 4 bits of each INT4 calculation circuit are used.

[0136] In the scheme shown in FIG. 1, in order to support the calculation of mixed quantization precision (such as INT2 and INT4), the corresponding calculation circuit needs to be designed for different quantization precisions, for example, a column can store 128 INT2 parameters, and accordingly at least 128 INT2 calculation circuits are needed, a column can store 64 INT4, and accordingly at least 64 INT4 calculation circuits are needed, so as to realize the compatible calculation of mixed quantization precision, the calculation unit needs to include at least 128 INT4 calculation circuits, but in the actual calculation process, whether it is INT2 parameter calculation or INT4 parameter calculation, 128 INT4 calculation circuits cannot be fully utilized, causing additional circuit area overhead. In addition, the above scheme also has the problem of poor flexibility, for example, after the design of the calculation unit is determined, the quantization precision supported by the storage area for storing parameters is also fixed, and the precision of the model is also fixed.

[0137] To this end, an embodiment of the present application provides a control method, a related device and a readable storage medium. When it is necessary to use the in-memory computing memory to perform calculation, if the quantization precision of the to-be-processed data is greater than the calculation precision of the calculation unit in the in-memory computing memory, the multi-bit data with higher quantization precision can be split into multiple sub-data with less bits, the quantization precision of each sub-data with less bits is less than or equal to the calculation precision of the calculation unit, so as to ensure that the calculation unit can perform calculation on each sub-data, and then the calculation result corresponding to each sub-data is subjected to internal shift processing and accumulation, and finally the calculation of the multi-bit data with higher quantization precision is completed. For the calculation compatible with mixed quantization precision, only the data splitting strategy needs to be adjusted according to different combinations of quantization precision and calculation precision, so that the calculation circuit with one kind of calculation precision can complete the calculation of multiple quantization precision parameters, without the need to specially design the calculation circuit with corresponding calculation precision for data with different quantization precision. This not only saves the circuit area overhead, but also improves the flexibility of the quantization precision of the in-memory computing memory in storing data, thereby ensuring that the precision of the model can be flexibly adjusted according to different scene requirements.

[0138] Optionally, when the quantization precision of the to-be-processed data is greater than the calculation precision of the calculation unit in the in-memory computing memory, the to-be-processed data can be split into multiple sub-data, and the multiple sub-data are respectively written into different rows or columns of the storage area for storage. The quantization precision of each sub-data is less than or equal to the calculation precision of the calculation unit. After receiving the calculation instruction sent by the processor, the in-memory computing memory can perform calculation on the sub-data stored in the multiple different rows / columns by the calculation unit, obtain corresponding multiple intermediate results, and then perform shift processing on the multiple intermediate results and input them into the accumulator for calculation, so as to achieve the purpose of completing the calculation of high-quantization-precision data by the calculation unit with low calculation precision and save the circuit area overhead.

[0139] For the convenience of understanding, the technical solutions provided by the present application will be described below in combination with more drawings.

[0140] In the present application, the same or similar parts between various embodiments or implementation manners can be mutually referred to, unless otherwise specified. In the various embodiments and various implementation manners / implementation methods / realization methods in the embodiments, the terms and / or descriptions of different embodiments and various implementation manners / implementation methods / realization methods have consistency and can be mutually referred to, unless otherwise specified and logically conflicted. The technical features in different embodiments and various implementation manners / implementation methods / realization methods can be combined to form new embodiments, implementation manners, implementation methods or realization methods according to their inherent logical relationship. The implementation manners of the present application described below do not constitute a limitation on the protection scope of the present application.

[0141] The application will be described in a clear and easy-to-understand manner below. It should be understood that the structure and application scenarios of the electronic device described in the application are used to more clearly illustrate the technical solutions of the application, and do not constitute a limitation on the technical solutions provided by the application.

[0142] Referring to FIG. 2A, which is a structural schematic diagram of an electronic device provided by the application, the electronic device 01 provided by the application can include one or more processors 11, one or more memory controllers 12, and one or more memories 13. The electronic device can also include a power management unit, an external memory, and the like (not shown in the figure). The electronic device 01 can be a subscriber unit, a cellular phone, a smart phone, a personal digital assistant (PDA) computer, a tablet computer, a handset, a laptop computer, a machine type communication (MTC) terminal, an automatic driving terminal, a smart wearable device, a smart speaker, and the like. The electronic device can also be a non-mobile terminal device (i.e., a fixed terminal) that needs to use a memory, which is not limited here.

[0143] The processor 11 can be a central processing unit (CPU), a graphics processing unit (GPU), a neural network processing unit (NPU), a system on chip (SOC), a microcontroller unit (MCU), or a chip with other functions. The processor 11 can exchange data with the memory 13 through the memory controller 12. For example, when a CPU executes a memory access instruction, it can first send a read / write request to a memory management unit (MMU, not shown in the figure). The memory management unit converts the virtual address and the physical address, and then sends the memory access command to the bus. The bus then transmits the memory access command to the memory controller 12, which translates the address again and accesses the data in the corresponding row / column of the memory 13.

[0144] The memory controller 12, which can be a dynamic memory controller (DMC), is mainly used to control data transmission between the processor 11 and the memory 13. The memory controller 12 can be disposed on the same chip as the processor 11 or on a separate chip, that is, the memory controller 12 can be integrated into the processor 11 (such as a CPU) chip or be a separate chip independent of the processor 11, which is not specifically limited here. Generally, the memory controller can include a command processing module, a data processing module, a refresh module, and the like, wherein the command processing module is used to process instructions issued by the processor 11; the data processing module is used to process data read or written into the memory 13 by the processor 11; and the refresh module is used to refresh the memory 13 according to a refresh command. The processor 11 can send memory access instructions (such as read / write instructions), control instructions, and the like to the memory 13 through the memory controller 12.

[0145] The memory 13 can be an in-memory computing memory (referred to as an in-memory computing memory), which includes a storage part and a computing part. The structure of the storage part in the memory 13 can be the structure of a dynamic random access memory (DRAM). Generally, the memory 13 is a place for temporarily storing processes and data, and can also be referred to as a main memory or an internal memory, which can be made of semiconductor devices, and is a storage space that can be directly addressed by the processor 11 or other external devices (timers, analog-to-digital converters, etc., not shown in the figure) with various functions. In the embodiments of the present application, the memory 13 can be an in-memory computing memory, the storage part of which can be used to store weight parameters of a large model, and the computing part of which can be used to perform calculations. For example, when the quantization precision of the weight parameters is greater than the calculation precision of the computing part, the weight parameters can be split into multiple sub-data with smaller precision and stored in different rows / columns, the data precision of each sub-data is less than or equal to the calculation precision of the computing part, and the subsequent computing part can calculate each sub-data in units of rows / columns.

[0146] For the convenience of understanding, the structure of the in-memory computing memory is exemplarily illustrated below. Please refer to FIG. 2B, which is a schematic diagram of a memory structure using the in-memory computing technology provided in the embodiments of the present application. The memory 13 can include one or more ranks (Rank). One Rank refers to a set of memory chips (Die). Memory chips in the same group share the same address and control signals. Each Rank is usually 64 bits wide, which means it can process 64 bits of data at the same time. For example, the memory 13 can include two Ranks, each of which can be independently accessed, thereby improving the bandwidth and performance of the memory 13. As shown in FIG. 2B, the Rank 1 can include a plurality of Dies, and the Rank 1 can be independently used to access data. The Rank 2 can include a plurality of PIM-Dies, and the PIM-Die is a Die with a computing unit (PU) integrated inside. The Rank 2 can be independently used to access data and also can be used to perform computing tasks. It should be noted that the same structure of the Die can be included in the same Rank, or different structures of the Die can be included in the same Rank, which is not specifically limited in the present application.

[0147] For ease of understanding, the structure of the PIM-Die will be described again. Please refer to FIG. 2C, which is a structural schematic diagram of a PIM-Die according to an embodiment of the present application. Each PIM-Die can include a plurality of storage areas (Banks), such as Banks A-J shown in FIG. 2C. A Bank is a basic unit for storing and managing data, and each Bank can be regarded as a storage area and can process data requests independently. Different Banks allow parallel access and operation, which means that the memory controller 12 can access multiple Banks simultaneously, thereby improving data reading efficiency. For example, a PIM-Die includes 4 Banks, and data is stored in the 4 Banks. When data is accessed, the memory controller 12 can activate data in the 4 Banks simultaneously. In addition to the storage area, each PIM-Die also integrates a plurality of computing units (PUs), such as PUs 1-5 shown in FIG. 2C. The PU is used to perform a computing task to achieve direct computing in the memory 13, instead of transferring data to an external processor 11 for processing, thereby improving computing efficiency. Each PU can correspond to one or more Banks, such as PU1 corresponding to Banks A and F shown in FIG. 2C, that is, data read from Banks A and F can be directly transferred to PU1 for computing. In addition, each PIM-Die can also integrate a plurality of mode registers (MRs), such as mode registers 1-n shown in FIG. 2C. The MR can be used to configure the operating mode and function of the memory 13, for example, to set parameters such as data width, delay, refresh rate, etc. The presence of the MR enables the memory controller 12 to configure and optimize the memory 13 according to the needs of the system, thereby improving performance and compatibility. By programming these registers, the system can adjust the behavior of the memory 13 to adapt to different application scenarios and performance requirements. Each PIM-Die can also include a global buffer that can be used as a cache to store frequently accessed data to reduce access delay and improve system response speed.

[0148] For ease of understanding, the structure of the Bank will be described again. Please refer to FIG. 2D, which is a structural schematic diagram of a Bank according to an embodiment of the present application. The Bank can include a storage array and a row buffer. The storage array is responsible for actual data storage and is composed of a plurality of storage units, which are usually organized in the form of rows and columns, that is, the storage array can be composed of M rows*N columns of storage units, allowing efficient data access. The row buffer is used to store the row data of a row read from the storage array.

[0149] For the convenience of understanding, the structure of the calculation part (calculation unit) in the memory 13 is also exemplarily described below. Referring to FIGS. 2E and 2F, FIG. 2E is a structural schematic diagram of a calculation unit provided in an embodiment of the present application, and FIG. 2F is a structural schematic diagram of another calculation unit provided in an embodiment of the present application. The calculation unit can include a multiplication circuit, an addition circuit, a shift circuit and an accumulator, and is suitable for multiply-accumulate operation. The multiplication circuit can include one or more multipliers and is suitable for multiplication operation; the addition circuit can include one or more adders and is suitable for addition operation; the shift circuit can include one or more shifters and is suitable for shift processing of input data; and the accumulator is suitable for accumulation operation of input data. Exemplarily, the shift operation can be performed after the multiplication operation is completed, and the structure in FIG. 2E is suitable; or the shift operation can be performed after the addition operation is completed, and the structure in FIG. 2F is suitable. Understandably, the calculation part in the memory 13 can also be designed according to different operation requirements to achieve the purpose of completing other operations. If the calculation part in the memory 13 needs to complete other operations, the memory 13 can include other circuits accordingly, and the structure of the memory 13 is not specifically limited herein. In an embodiment of the present application, the multiplication circuit and the addition circuit can be used to perform multiply-accumulate calculation on the multiple sub-data stored in different rows / cols after splitting and the input vector respectively, the shift circuit is used to perform shift on one or more intermediate results obtained after calculation of the multiple sub-data and the input vector, and the accumulator can be used to accumulate the one or more intermediate results after shift to obtain the final data, so that the calculation unit with low calculation precision can complete the calculation of high-precision data.

[0150] It should be noted that the electronic device 01 can also include a power management unit, which can be used to supply power for the processor 11, the memory controller 12 and the memory 13, and can also supply power for other devices or modules in the electronic device 01. In addition, the electronic device 01 can also include one or more interfaces, which can be used to realize communication between the electronic device 01 and other devices. In some embodiments, the interface can include a wire for coupling a wired connection, or a pin for coupling a wireless transceiver, or a chip and / or pin for wireless connection. In some embodiments, the interface can include a transmitter, a receiver, a transceiver and / or an antenna.

[0151] Understandably, the embodiments of the present application can be applied to various computer system architectures, and the architecture in FIG. 2A is only an exemplary embodiment in the embodiments of the present application, and should not be construed as a limitation of the present application. The embodiments of the present application can be applied to architectures including but not limited to the above architectures. It should be understood that the electronic device 01 can have more or fewer units / modules than those shown in FIG. 2A, can combine two or more units / modules, or can have a different configuration of units / modules. The various units / modules shown in FIG. 2A can be implemented in hardware, software, or a combination of hardware and software including one or more signal processing and / or application specific integrated circuits, and the embodiments of the present application do not make specific limitations thereto.

[0152] In order to better understand the embodiments of the present application, the technical solutions provided by the embodiments of the present application will be described below in combination with more drawings.

[0153] The control method provided by the embodiments of the present application will be briefly described below taking the processing process of the weight parameters from the storage stage to the calculation stage as an example. First, the control method will be briefly described from the processing process of the storage stage. In the storage stage, the control method can at least include steps S300-S302 as shown in FIG. 3, which involve the processor and the internal memory. The processor and the internal memory are coupled, and the internal memory can include at least one storage area Bank and at least one calculation unit. Each Bank includes M rows*N columns of storage units, and the calculation accuracy of each calculation unit is K, and M, N, and K are all greater than 0. Steps S300-S302 are as follows:

[0154] S300: The processor obtains to-be-processed data.

[0155] The quantization accuracy of the to-be-processed data is L, and the to-be-processed data can be the weight parameters of the model. The weight parameters can be input by the user when constructing the model, or can be obtained by the processor reading from the external memory, which is not specifically limited herein. Optionally, when constructing the model, the user will explicitly specify the quantization accuracy of the required parameters of the model, and then the user will input the weight parameters according to the quantization accuracy required by the model, so that the processor can determine the quantization accuracy of the weight parameters when obtaining the weight parameters.

[0156] S301: When the quantization accuracy L of the to-be-processed data is greater than the calculation accuracy K of the calculation unit, the processor splits the to-be-processed data to obtain a plurality of sub-data.

[0157] The quantization accuracy of each sub-data in the plurality of sub-data is less than or equal to the calculation accuracy K of the calculation unit.

[0158] S302: The processor sends a first memory access instruction for each of the plurality of sub-data according to a storage rule.

[0159] The first memory access instruction can be a write instruction, used to instruct the memory to store the sub-data corresponding to the instruction in a specified row or column. Different first memory access instructions in the plurality of first memory access instructions specify different rows or columns. That is, the processor can instruct the memory to store a plurality of sub-data in different rows or columns through a plurality of memory access instructions. Accordingly, upon receiving each first memory access instruction, the memory can write the corresponding sub-data into the specified row or column according to the row or column address carried by each first memory access instruction, and store different sub-data in different rows or columns. Alternatively, the different rows or columns can be different rows or columns in the same bank, or different rows or columns in different banks, which are not limited here.

[0160] In the storage phase, after obtaining the to-be-processed data, the processor can first determine whether the quantization precision of the to-be-processed data is greater than the calculation precision of the calculation unit in the memory. If the quantization precision of the to-be-processed data is greater than the calculation precision of the calculation unit, the memory cannot directly calculate the to-be-processed data through the calculation unit subsequently. Therefore, the processor can first split the to-be-processed data to obtain a plurality of sub-data whose quantization precisions are less than or equal to the calculation precision, so that the memory can subsequently calculate each sub-data through the calculation unit. It should be noted that the calculation unit generally calculates data in a row or column as a unit, and the plurality of sub-data obtained after splitting are stored in different rows or columns, so as to ensure that the calculation unit can restore the accurate calculation result through shift accumulation in the subsequent calculation process.

[0161] For ease of understanding, the storage process is described by taking a to-be-processed data including one weight parameter as an example. For example, the quantization precision of the weight parameter is INT4, and the calculation precision of the calculation unit is INT2. The processor can split the weight parameter into two 2-bit data, one of which is the high 2-bit data of the weight parameter, and the other is the low 2-bit data of the weight parameter, that is, one INT4 quantization precision weight parameter is split into two INT2 quantization precision sub-data. The processor then writes the two INT2 quantization precision sub-data into different rows or columns through two memory access instructions for storage. In the subsequent calculation process, the processor can instruct the memory to input the two INT2 quantization precision sub-data into the calculation unit in sequence through the calculation instruction for calculation, thereby obtaining two intermediate results, and then restoring the final calculation result (i.e., the calculation result of the weight parameter with INT4 quantization precision) through shift accumulation of the two intermediate results.

[0162] Optionally, the storage rule can specifically include that the specified row or the specified column of the plurality of sub-data writes is an adjacent row or column, that is, the processor can instruct the memory to store the plurality of sub-data obtained after data splitting in a plurality of adjacent rows / columns after data splitting of the to-be-processed data. For example, 2 sub-data are obtained after splitting of 1 to-be-processed data, the processor can instruct the memory to store 1 sub-data in a specified column (such as col.k) through one memory access instruction and to store the other sub-data in a specified column (such as col.k+1) through another memory access instruction, and col.k and col.k+1 are adjacent columns.

[0163] Optionally, the storage rule can specifically include that the specified row or the specified column of the plurality of sub-data writes is in different partitions, in other words, the processor can instruct the memory to store the plurality of sub-data obtained after data splitting in a plurality of different rows / columns after data splitting of the to-be-processed data, and the interval between the different rows / columns is greater than 1. That is, the processor can instruct the memory to store the plurality of sub-data in a plurality of non-adjacent rows / columns. Alternatively, one storage area of the memory can be divided into a plurality of different partitions, and the processor can instruct the memory to store the plurality of sub-data in the plurality of different partitions, and each partition can include a plurality of rows / columns. For example, one storage area of the memory can include 2 partitions, partition 1 includes col.0-col.31, and partition 2 includes col.32-col.63, 2 sub-data are obtained after splitting of 1 to-be-processed data, the processor can instruct the memory to store 1 sub-data in a specified column (such as col.0) through one memory access instruction and to store the other sub-data in a specified column (such as col.32) through another memory access instruction, and col.0 and col.32 belong to different partitions and are not adjacent columns.

[0164] In a possible implementation, when the quantization precision of the to-be-processed data is less than or equal to the calculation precision of the calculation unit, the processor can directly store the to-be-processed data in the specified row / column through the memory access instruction without data splitting. In the subsequent calculation process, the memory can calculate the to-be-processed data stored in the specified row / column through the calculation unit to directly obtain the final calculation result. For example, the quantization precision of the to-be-processed data is INT2, the calculation precision of the calculation unit is INT2, and the calculation unit can support direct calculation of the to-be-processed data, so that the to-be-processed data with the INT2 quantization precision does not need to be split in the storage stage and can be directly stored in the specified row / column.

[0165] In a possible implementation, when the quantization precision L of the to-be-processed data is less than the calculation precision K of the calculation unit, and the calculation precision K is J times of the quantization precision L (J is an integer greater than 1), the to-be-processed data can be directly stored in the specified row / column without data splitting in the storage stage. In the subsequent calculation process, the calculation unit can simultaneously calculate the to-be-processed data stored in at most J rows / columns. For example, the quantization precision of the to-be-processed data is INT2 (i.e., 2-bit data), the calculation precision of the calculation unit is INT4 (4-bit data can be calculated), and the calculation precision is 2 times of the quantization precision. In this case, the calculation unit can support direct calculation of the to-be-processed data, and the calculation unit uses only half of the resources (i.e., only 2 bits of 4 bits) to calculate 1 row / column of INT2 data. Therefore, the calculation unit can calculate the INT2 data in at most 2 rows / columns in the calculation stage. That is, the calculation resources of the calculation unit can be divided into J sub-resources, and the calculation precision of each sub-resource is the same as the quantization precision L of the to-be-processed data. Therefore, the calculation unit can simultaneously calculate J to-be-processed data, and the quantization precision of the to-be-processed data is L.

[0166] It should be noted that, for the quantization precision, the processor can determine the quantization precision of the to-be-processed data when the to-be-processed data is acquired. For the calculation precision of the calculation unit, in a possible implementation, the processor can send an instruction to the internal memory through the bus to instruct the internal memory to report the calculation precision of the calculation unit, so that the processor can determine whether the quantization precision of the to-be-processed data is greater than the calculation precision of the calculation unit when the to-be-processed data is acquired.

[0167] In a possible implementation, in addition to the steps S300-S302, the control method provided by the embodiment of the present application can further include steps S303-S306 as shown in FIG. 11. The steps S303-S306 are as follows.

[0168] S303: The processor acquires the initial weight parameter of the model.

[0169] The processor can determine the quantization precision and the operator type corresponding to each initial weight parameter when the initial weight parameters are obtained. Optionally, the quantization precision can be an integer (INT), a floating point (FP), or other types, such as INT2, INT4, INT8, …, INTx, and FP16, FP32, etc. The operator type indicates what operation the initial weight parameter needs to perform, such as multiplication, addition, multiplication and addition, exponentiation, etc. The quantization precision and the operator type of different initial weight parameters can be the same or different. For example, the quantization precision and the operator type of the initial weight parameter A and the initial weight parameter B are the same, both of which are INT8 quantization precision and both of which need to perform multiplication and addition; or the quantization precision of the initial weight parameter A and the quantization precision of the initial weight parameter B are the same (such as both of which are INT4 quantization precision), but the operator types are different (such as one is multiplication and the other is addition); or the quantization precision of the initial weight parameter A and the quantization precision of the initial weight parameter B are different (such as one is INT4 and the other is INT8), but the operator types are the same (such as both of which are multiplication and addition); or the quantization precision and the operator type of the initial weight parameter A and the initial weight parameter B are different.

[0170] S304: The processor divides the initial weight parameters to obtain one or more first parameter sets.

[0171] After obtaining the initial weight parameters of the model, the processor can analyze the initial weight parameters of the model, divide the obtained initial weight parameters of the model based on the quantization precision and the operator type of each initial weight parameter, divide the initial parameters with the same quantization precision and the same operator type in the same parameter set, and thus obtain one or more first parameter sets, or model sub-module sets. Optionally, when dividing the initial weight parameters, the processor can also consider the parameter quantity of the model, the memory capacity of the memory, and the storage calculation ratio, etc. For example, when a parameter set exceeds the memory capacity limit, the parameter set can be further divided to avoid the situation that the memory capacity does not meet the storage requirement.

[0172] S305: The processor determines one or more second parameter sets from the one or more first parameter sets; the initial weight parameters included in the second parameter set support storage into the memory.

[0173] The processor can determine one or more second parameter sets, or PIM sub-module sets, from the one or more first parameter sets based on the calculation precision of the calculation unit in the internal memory and the operator type of the initial weight parameter. The initial weight parameter included in the second parameter set is supported to be stored in the internal memory and supported to be calculated by the calculation unit in the internal memory. For example, the quantization precision of the initial weight parameter and the calculation precision of the calculation unit belong to the same type (such as both are INT or both are FP), and the operator type of the initial weight parameter and the operator type of the calculation unit are the same (such as both are multiplication), the initial weight parameter is supported to be stored in the internal memory and calculated by the calculation unit, and the first parameter set corresponding to the initial weight parameter can be determined as the second parameter set. Otherwise, the initial weight parameter is not supported to be stored in the internal memory and calculated by the calculation unit.

[0174] For example, the initial weight parameter needs to be multiplied, but the calculation unit of the PIM can only be added, so the initial weight parameter cannot be calculated by the calculation unit, and thus should not be put into the PIM; or the initial weight parameter needs to be exponentiated, but the calculation unit of the PIM can only be multiplied, added, or multiplied and added, so the initial weight parameter cannot be calculated by the calculation unit, and thus should not be put into the PIM.

[0175] For example, the quantization precision of the initial weight parameter is INT4, and the initial weight parameter needs to be calculated by INT, but the calculation unit of the PIM does not support INT calculation, but supports FP (floating point) calculation, so the initial weight parameter cannot be calculated by the calculation unit, and thus should not be put into the PIM. Or, the quantization precision of the initial weight parameter is FP32, and the initial weight parameter needs to be calculated by FP, but the calculation unit of the PIM does not support FP calculation, but supports INT calculation, so the initial weight parameter cannot be calculated by the calculation unit, and thus should not be put into the PIM.

[0176] It should be noted that when the operator type of the initial weight parameter and the calculation unit is the same, but the quantization precision types of the two are different, the quantization precision type of the initial weight parameter can also be converted, such as INT to FP (or FP to INT), so that the initial weight parameter can be calculated by the calculation unit, but the data type conversion may cause a certain precision loss, and thus cause information loss.

[0177] S306: The processor determines one or more third parameter sets from the one or more second parameter sets; the quantization precision of the initial weight parameter included in the third parameter set is greater than the calculation precision of the calculation unit.

[0178] In the above process, the processor can further determine one or more third parameter sets (or split sub-modules) from the one or more second parameter sets to support data splitting, i.e., the quantization precision of the initial weight parameter included in the third parameter set is greater than the calculation precision of the calculation unit. For example, the quantization precision of the initial weight parameter is INT4, and the calculation precision of the calculation unit is INT2. INT4 is greater than INT2. The initial weight parameter needs to be split, such as splitting 2-bit data, one is the high 2-bit data of the initial weight parameter, and the other is the low 2-bit data of the initial weight parameter. The subsequent calculation unit can calculate the two 2-bit data respectively, and finally complete the calculation of the initial weight parameter of INT4.

[0179] Optionally, when the quantization precision of the initial weight parameter is less than or equal to the calculation precision of the calculation unit, the initial weight parameter can not be split.

[0180] After the processor completes the above model analysis process, it can obtain three types of sets, i.e., the first parameter set (model sub-module set), the second parameter set (PIM sub-module set), and the third parameter set (split sub-module set). In the subsequent model loading process, the processor can determine the storage form (PIM memory or non-PIM memory) and storage location (memory logic / physical address) of the initial weight parameter of the model and whether to split, and can determine the storage location of the split data if the initial weight parameter is split.

[0181] The processor can load the model after obtaining the first parameter set (model sub-module set), the second parameter set (PIM sub-module set), and the third parameter set (split sub-module set) through model analysis. The model loading process of the processor can refer to the flowchart shown in FIG. 12, including but not limited to the following steps:

[0182] S121: The processor can load the model sub-module set i (the first parameter set i), i is greater than or equal to 1 and less than or equal to N, and N is the total number of model sub-module sets.

[0183] S122: The processor further determines whether the model sub-module set i belongs to the PIM sub-module set (the second parameter set).

[0184] S123: When the model sub-module set i does not belong to the PIM sub-module set, the processor stores the initial weight parameter in the model sub-module set i into the non-PIM memory.

[0185] S124: When the model submodule set i belongs to the PIM submodule set, the processor further determines whether the initial weight parameter in the model submodule set i supports data splitting or belongs to the split submodule set.

[0186] S125: When the initial weight parameter in the model submodule set i does not support data splitting, the processor stores the initial weight parameter in the model submodule set i into the PIM memory.

[0187] S126: When the initial weight parameter in the model submodule set i supports data splitting, the processor can determine whether the initial weight parameter has been offline split.

[0188] S127: If the initial weight parameter has been offline split, the processor stores the split data obtained by splitting the initial weight parameter in the model submodule set i into the PIM memory respectively.

[0189] S128: If the initial weight parameter has not been offline split, the processor splits the initial weight parameter in the model submodule set i online, and stores the split data obtained by splitting into the PIM memory respectively.

[0190] Optionally, the initial weight parameter in the model submodule set i can be offline split or split online by the processor, which is not limited here.

[0191] Optionally, in the above model analysis stage, after determining the above model submodule set, and further determining the PIM submodule set and the split submodule set from the model submodule set, the processor can identify the model submodule set belonging to the PIM submodule set and the split submodule set. For example, a first mark is added to the model submodule set belonging to the PIM submodule set, and a second mark is added to the model submodule set belonging to the split submodule set. So that in the subsequent model loading stage, when the processor loads the model submodule set i, it can quickly determine whether the model submodule set i is the PIM submodule set, and whether the initial weight parameter therein supports splitting according to the identification.

[0192] Understandably, after the processor obtains the to-be-processed data, when the quantization precision L of the to-be-processed data is greater than the calculation precision K of the calculation unit, that is, when the to-be-processed data belongs to the above third parameter set (split submodule set), the processor can split the to-be-processed data to obtain a plurality of sub-data.

[0193] Optionally, the processor can preliminarily determine the storage form, storage location, whether data splitting is needed, and the storage location of the split data, and the like analysis results when performing model analysis; and the processor can load according to the analysis results and real-time hardware conditions (such as memory capacity, memory calculation ratio) when performing model loading. When the hardware conditions do not allow, the storage form and storage location of the initial weight parameters when loading can be different from the storage form and storage location when the model is analyzed, which is not limited here.

[0194] The control method provided by the embodiments of the present application is briefly described from the processing process of the calculation stage. In the calculation stage, the control method can at least include steps S400-S403 as shown in FIG. 4. Among them,

[0195] S400: The internal memory receives the second memory access instruction sent by the processor.

[0196] The second memory access instruction can be a calculation instruction, which is used to instruct the internal memory to calculate a plurality of sub-data. The plurality of sub-data is obtained by the processor splitting the to-be-processed data when the quantization precision L of the to-be-processed data is greater than the calculation precision K of the calculation unit; and the quantization precision of the sub-data is less than or equal to the calculation precision K of the calculation unit. Optionally, the second memory access instruction can also be used to indicate the quantization precision L of the to-be-processed data.

[0197] S401: The internal memory determines the storage addresses of the plurality of sub-data according to the second memory access instruction and the storage rule.

[0198] The second memory access instruction carries the storage address of one sub-data in the plurality of sub-data. After determining the storage address of the sub-data, the internal memory can determine the storage addresses of the other sub-data according to the storage rule (storing according to adjacent rows / columns, or storing according to different partitions). For example, 2 sub-data are stored according to the adjacent column rule col.0 and col.1, and the column address carried by the second memory access instruction is col.0. After the internal memory determines that one sub-data is stored in col.0 through the second memory access instruction, the internal memory can determine that the other sub-data is stored in col.1 according to the storage rule.

[0199] S402: The internal memory reads the plurality of sub-data from the storage addresses in sequence in response to the second memory access instruction.

[0200] S403: The internal memory uses the plurality of sub-data to calculate and determine the final result.

[0201] In the storage stage, since the quantization precision of the weight parameter is greater than the calculation precision of the calculation unit, the weight parameter is split into multiple sub-data and stored in multiple different rows / columns, and in the calculation stage, when the memory needs to calculate the to-be-processed data, the memory needs to read multiple sub-data from multiple different rows / columns and sequentially calculate each sub-data through the calculation unit. The calculation of each sub-data can obtain an intermediate result (such as a first result, a second result, a third result, or a fourth result) corresponding to the sub-data.

[0202] Optionally, after the intermediate result corresponding to each sub-data is calculated, the memory can determine the specific number of bits that each intermediate result needs to be shifted according to the position of each sub-data in the to-be-processed data, and then shift each intermediate result through a shift circuit. Finally, the calculation of the multiple shifted intermediate results can obtain the final result. In short, the to-be-processed data includes high-bit data and low-bit data. When the split sub-data is high-bit data, the intermediate result corresponding to the sub-data needs to be shifted. When the split sub-data is low-bit data, the intermediate result corresponding to the sub-data does not need to be shifted.

[0203] It should be noted that for the calculation precision, the calculation unit is one of the capabilities of the memory, and the calculation precision of the calculation unit is known to the memory. For the perception of the quantization precision, the memory can perceive the quantization precision of the to-be-processed data according to the predefined calculation instruction issued by the processor, or the processor can configure the mode register in the memory to enable the memory to perceive the quantization precision of the to-be-processed data.

[0204] Optionally, the quantization precision perception between the processor and the memory can be performed through a predefined calculation instruction. That is, the processor can indicate the quantization precision of the to-be-processed data of the memory by issuing a predefined calculation instruction to the memory. For example, the predefined calculation instruction can be configured in the form of the following instructions, such as COMP_INT1, COMP_INT2, COMP_INT3, COMP_INT4, COMP_INT8, COMP_INT16, COMP_AC_INT1, COMP_AC_INT2, COMP_AC_INT3, COMP_AC_INT4, COMP_AC_INT8, COMP_AC_INT16, ACT_INT1, ACT_INT2, ACT_INT4, ACT_INT8, ACT_INT16, and the like.

[0205] COMP INT1 instruction can indicate to perform a computation operation on data at a specified column address, and the quantization precision of the data stored in the column is INT1; COMP INT8 instruction can indicate to perform a computation operation on data at a specified column address, and the quantization precision of the data stored in the column is INT8; the meaning of other COMP INTx is basically the same as that of COMP INT1 and COMP INT8, and the quantization precision of the data stored is different, which is not described herein.

[0206] COMP AC INT1 instruction can indicate to perform a computation operation on data stored in all columns in a row, and the quantization precision of the data stored in all columns in the row is INT1; similarly, COMP AC INT16 indicates that the quantization precision of the data stored in all columns in the row is INT16. In addition, ACT INT1 instruction can also indicate to perform a computation operation on data stored in all columns in a row, and the quantization precision of the data stored in all columns in the row is INT1. Further optionally, the difference between ACT INT1 instruction and COMP AC INT1 instruction is that COMP AC INT1 instruction needs to be row-gated by a normal activation instruction ACT before being issued, while ACT INT1 itself is a row-gating command, and there is no need to additionally issue a normal ACT instruction for row-gating.

[0207] It should be noted that the pre-defined computation instructions in the embodiments of the present application can also indicate other data types (such as floating point type) in addition to the above-mentioned integer data types; and the pre-defined computation instructions can also indicate other quantization precisions (such as 32) in addition to the above-mentioned quantization precisions (including 1, 2, 3, 4, 8, 16), which are not limited herein.

[0208] Optionally, the mode register (Mode Register, MR) can also be used between the processor and the internal memory to realize the configuration of the quantization precision awareness. Exemplarily, the mode register for realizing the quantization precision awareness can be configured with reference to Table 1 below.

[0209] Table 1 Quantization precision awareness mode register

[0210] The MR# field can represent the index of the mode register.

[0211] The MA[6:0] field can represent the address of the mode register. Generally, the processor can find the mode register through the mode register index and address, and then adjust the configuration of the mode register.

[0212] Access: R / W, indicating that the mode register can be read and written. For example, the values of OP[5:4] and OP[3:0] can be read respectively by read command, so as to determine the number of Banks activated simultaneously in the in-memory computing memory and the quantization precision of the data stored in the in-memory computing memory. For another example, the values of OP[5:4] and OP[3:0] can be changed respectively by write command, so as to adjust the number of Banks activated simultaneously in the in-memory computing memory and the quantization precision of the data stored in the in-memory computing memory.

[0213] OP[7]: The length can be 1 bit, and when the value is 1, it indicates an all columns mode, automatically sequentially performing the calculation operation of the data in all columns (such as col.0~col.63), which is similar to the above-mentioned COMP_AC_INTx instruction or ACT_INTx instruction indicating the calculation operation of the data stored in all columns in a row; when the value is 0, it indicates a normal mode, performing the calculation operation of the data in the specified column, which is similar to the above-mentioned COMP_INTx instruction indicating the calculation operation of the data stored in the specified column.

[0214] OP[6]: Reserved field, the length can be 1 bit.

[0215] OP[5:4]: The length of the field can be 2 bits, which is used to configure the number of Banks activated simultaneously. For example, the values can be 1, 4, 8, 16, etc., or other values such as 2, 6, 12, etc. Exemplarily, the values of the field can directly indicate the number of Banks activated simultaneously, for example, the value of OP[5:4] is configured as binary 00, indicating that the number of Banks activated simultaneously is 1; the value of OP[5:4] is configured as binary 01, indicating that the number of Banks activated simultaneously is 4; the value of OP[5:4] is configured as binary 10, indicating that the number of Banks activated simultaneously is 8; the value of OP[5:4] is configured as binary 11, indicating that the number of Banks activated simultaneously is 16. Understandably, the length of the field can be configured according to the maximum number of Banks activated simultaneously, and the above field length and the values of the number of Banks activated simultaneously are only examples and should not constitute a specific limitation of the configuration of the mode register.

[0216] OP[3:0]: the field length can be 4 bits, used to configure the quantization precision, for example, can be configured as INT1, INT2, INT3, INT4, INT8, INT16, FP16 or FP32, etc. Exemplarily, the 4 bits of OP[3:0] can be used to indicate the type and quantization precision of the data, for example, the highest bit in the 4 bits can be used to indicate that the data type is integer or floating point, and the remaining 3 bits are used to indicate the quantization precision of the data. Alternatively, the highest bit is 0, indicating that the data type is floating point, and the highest bit is 1, indicating that the data type is integer. For example, OP[3:0]=1111 can be considered as configuring the quantization precision as INT8. Alternatively, each quantization precision can correspond to a unique OP[3:0] value, and the OP[3:0] value is configured to directly indicate a certain quantization precision, without the need for a certain bit to distinguish the data type. For example, OP[3:0]=0000 corresponds to INT1, OP[3:0]=0001 corresponds to INT2, OP[3:0]=0010 corresponds to INT3, …, OP[3:0]=1111 corresponds to FP32, etc. Understandably, the field length can be configured according to the maximum quantization precision to be configured. The above field length and quantization precision values, as well as the indication method of the quantization precision, are only examples and are not specifically limited herein.

[0217] Alternatively, for the quantization precision awareness configuration mode through the mode register, the computing instructions exchanged between the processor and the memory can only indicate the data of the specified row / column for computing, and can not indicate the quantization precision of the data. For example, the COMP instruction can indicate the data of the specified column address for computing operation, and the quantization precision of the data stored in the column can be determined according to the OP[3:0] field in the mode register. This mode can be suitable for a scenario where the number of reserved instruction types is limited and the existing number is insufficient to indicate multiple different quantization precisions. For example, the number of reserved instruction types is only 3, and currently 4 different quantization precisions (such as INT1, INT2, INT4 and INT8) need to be indicated. The reserved number is only enough to indicate three of the quantization precisions, such as COMP INT1, COMP INT2, and COMP INT4 indicating quantization precisions INT1, INT2, and INT4, respectively. COMP INT8 cannot be indicated due to insufficient reserved number.

[0218] For the convenience of understanding, the flow of the processor instructing the in-memory computing memory to perform data calculation is exemplarily illustrated based on the above-mentioned quantization precision perception manner. Taking a row in the in-memory computing memory as an example, there are 64 columns in the row, one column can store 256 bits of data, and the calculation unit has a calculation precision of INT2. When the quantization precision of the weight parameter is INT2, one column can store 128 weight parameters with a quantization precision of INT2, and one row can store 64*128 = 8192 weight parameters with a quantization precision of INT2.

[0219] Exemplarily, for the quantization precision perception manner between the processor and the in-memory computing memory through the pre-defined calculation instruction, the processor can issue a COMP_INT2 calculation instruction to instruct the in-memory computing memory to complete the calculation of 128 INT2 weight parameters in the specified column address. Correspondingly, when the processor needs to complete the calculation of the above-mentioned 8192 weight parameters, the processor can refer to the processing flow shown in FIG. 5. The processor can first activate the specified Banka and the specified rowi in the Banka through an activation instruction (ACTIVATE), and then sequentially issue 64 COMP_INT2 calculation instructions to sequentially calculate the weight parameters in col.0~col.63, thereby completing the calculation of 8192 weight parameters with a quantization precision of INT2 as shown in the following Formula I. Alternatively, after completing the calculation of all weight parameters in the 64 columns of the row (rowi), the processor can issue a pre-charge instruction (PRECHARGE) to close the opened rowi and Banka.

[0220] Formula I:

[0221] wherein the input vector V can be transmitted into the in-memory computing memory by the host (or the processor), and the number of elements of the input vector V corresponds to the number of weight parameters.

[0222] Exemplarily, for the quantization precision perception manner between the processor and the in-memory computing memory through the configuration mode register, the OP[3:0] field in the mode register can be configured as 0001, indicating that the in-memory computing memory stores weight parameters with a quantization precision of INT2. The processor can issue a COMP calculation instruction to complete the calculation of the specified column address, i.e., the calculation of 128 INT2 weight parameters. Further, when the processor needs to complete the calculation of the above-mentioned 8192 weight parameters, the processor can sequentially issue 64 COMP calculation instructions to sequentially calculate the weight parameters in col.0~col.63, thereby completing the calculation of 8192 weight parameters with a quantization precision of INT2 as shown in the above-mentioned Formula I. Understandably, the activation instruction and the pre-charge instruction are also involved in the processing process, which will not be described herein.

[0223] In a possible implementation, when the quantization precision of the weight parameter of the in-memory computing storage needs to be written is greater than the calculation precision of the calculation unit, the multi-bit quantized data can be first split into multiple less-bit sub-data and stored in different rows / columns of the in-memory computing storage. When calculating, the multiple sub-data can be sequentially read to the calculation unit for calculation, and the calculation results of the sub-data are shifted and accumulated by the shift circuit of the in-memory computing storage, so as to achieve the purpose of completing the calculation of the multi-bit quantized data by using the calculation unit with low calculation precision.

[0224] Exemplarily, taking the calculation precision of the calculation unit as INT4 and the quantization precision of the weight parameter as INT8, the INT8 quantized data of the weight parameter is split into two INT4 quantized data for calculation as an example, and the calculation process is simply explained. First, for any INT8 quantized data X, it can be expressed as the high 4-bit data left shifted by 4 bits plus the low 4-bit data, that is, X=(H<<4)+L. For example, X=01110010, the high 4-bit data H of X is 0111, the low 4-bit data L of X is 0010, the data H left shifted by 4 bits is H<<4=01110000, and H<<4(01110000)+L(0010)=X(01110010).

[0225] Based on this, the multiplication calculation of the INT8 quantized data X and another data Y can refer to the manner shown in FIG. 6, and the data precision of Y can be INT4. The multiplication of X and Y can be understood as splitting the INT8 quantized data X into high 4-bit (denoted as H) and low 4-bit (denoted as L), then adding the multiplication results of the high 4-bit data and the low 4-bit data with the data Y, that is, X*Y=[(H<<4)+L]*Y=(H<<4)*Y+L*Y=[(H*Y)<<4]+L*Y. Referring to FIGS. 7A-7D, the flow of the multiplication calculation of X and Y can include the following calculation steps:

[0226] (1) As shown in FIG. 7A, the high 4-bit data of X is multiplied with Y to obtain an intermediate result R1;

[0227] (2) As shown in FIG. 7B, the intermediate result R1 is left shifted by four bits to obtain a result RH;

[0228] (3) As shown in FIG. 7C, the low 4-bit data of X is multiplied with Y to obtain an intermediate result RL;

[0229] (4) As shown in FIG. 7D, RH and RL are added to obtain a final result R.

[0230] It should be noted that after the calculation of the plurality of sub-data obtained by splitting, a plurality of intermediate results can be obtained, and in the calculation of the final result, the plurality of intermediate results need to be shifted first, and then the plurality of intermediate results after shifting are calculated to ensure the accuracy of the final result. Among them, the specific number of bits of each intermediate result in the plurality of intermediate results is determined based on the position of the sub-data corresponding to the intermediate result in the original data (i.e. the data before splitting). For example, the INT8 quantization data is split into high 4-bit data (i.e. the first sub-data) and low 4-bit data (i.e. the second sub-data), since there are 4-bit data after the high 4-bit data before splitting, the intermediate result obtained by calculating this part of data needs to be left shifted by 4 bits; and the low 4-bit data before splitting has no data, so the intermediate result obtained by calculating this part of data can not be shifted. For another example, the INT3 quantization data is split into high 1-bit data (i.e. the first sub-data), middle 1-bit data, and low 1-bit data (i.e. the second sub-data), since there are 2-bit data after the high 1-bit data before splitting, the intermediate result obtained by calculating this part of data needs to be left shifted by 2 bits; and since there is 1-bit data after the middle 1-bit data before splitting, the intermediate result obtained by calculating this part of data needs to be left shifted by 1 bit; and since there is no data after the low 1-bit data before splitting, the intermediate result obtained by calculating this part of data can not be shifted.

[0231] Optionally, in addition to being split into 4+4, the weight parameter with the quantization precision of INT8 can also be split into 2+2+2+2, or 3+3+2 and 3+3+1+1, etc. When splitting data, the specific splitting method can be selected by comprehensively considering the resource utilization rate of the INT4 calculation unit and the maximum amount of data that can be stored in each column (such as 256 bits). For example, if it is split into 2+2+2+2, the storage resources of each column can be fully utilized (when 128 data is stored, 256 bits can be used up), but only 2-bit resources in the calculation unit are used each time, and the resource utilization rate of each calculation unit is only 50%. If you want to fully utilize the resources of the calculation unit, you also need to divide the resources of the calculation unit first, so that each calculation unit can calculate 2 2-bit data, and the calculation process is more complex than the 4+4 method. If it is split into 3+3+2, the resource utilization rate of the calculation unit is improved, but the storage resources in some columns cannot be fully utilized (256 cannot be divided by 3). If it is split into 4+4, not only the resources of the calculation unit can be fully utilized, but also the storage resources of each column can be fully utilized (256 is an integer multiple of 4), so for the combination of INT4 calculation precision and INT8 quantization precision, the splitting method of 4+4 can be preferred.

[0232] For the convenience of understanding, the following first takes the cooperation of the quantized perception calculation instruction COMP INT4 and the calculation unit with the calculation precision of INT2 to complete the calculation of INT4 quantized data as an example, and the control method in the embodiment of the application is exemplarily described in combination with the processing process of the storage stage and the calculation stage. That is, taking the use of the INT2 calculation unit to complete the INT4 quantized data as an example.

[0233] For example, the memory-in-computing memory can contain 64 columns (cols) per row, and each col can store 256 bits of data, that is, one col can store 128 2-bit data (quantized precision of INT2 data) or 64 4-bit data (quantized precision of INT4 data), so that one row can store 8192 INT2 weight parameters or 4096 INT4 weight parameters. Among them, the calculation result of all 4096 INT4 data contained in one row and the input vector V is as shown in formula two:

[0234] Formula two:

[0235] Exemplarily, in the storage stage, since the quantized precision of the weight parameter is greater than the calculation precision of the calculation unit (INT4 is greater than INT2), each weight parameter can be split into two 2-bit data, and the two 2-bit data are respectively stored in different two cols, and the two cols can be adjacent cols. For example, for the 4096 INT4 weight parameters W i,j , the data storage rule shown in FIG. 8A can be referred to for storage. Since one col can store 128 2-bit data, 128 W i,j 4-bit data can be split into high 2-bit data and low 2-bit data, 128 W i,j high 2-bit data is stored in the kth column (denoted as col.k, such as col.0) by one write instruction (corresponding to the first memory access instruction), and 128 W i,j low 2-bit data is stored in the k+1th column (denoted as col.k+1, such as col.1) by another write instruction (corresponding to the first memory access instruction), W i,j represents an INT4 weight parameter, the data length of which is 4 bits, the high 2-bit data of which can be represented as W i,j H2b , and the low 2-bit data of which can be represented as W i,j L2b . Alternatively, 128 W i,j low 2-bit data can also be stored in the kth column (denoted as col.k), and 128 W i,jThe high 2-bit data of the weight parameter W can be stored in the k+1th column (denoted as col.k+1), that is, the high-bit data can be stored in the front column or can also be stored in the rear column, and correspondingly, the low-bit data can also be stored in the front column or can also be stored in the rear column, which is not specifically limited here.

[0236] Correspondingly, the other 3968 (4096-128) INT4 weight parameters are stored according to the storage rule of the foregoing 128 W i,j , and finally, the 4096 W i,j are split and stored in different columns, and the high 2-bit data and the low 2-bit data of the same weight parameter W i,j can be stored in adjacent columns. It should be noted that the above is an example of the data access of the internal memory in the column unit, and it can be understood that when the internal memory can access data in the row unit, the weight parameter can also be stored in the row unit, that is, the technical solution of splitting and storing the weight parameter in different rows should also fall within the protection scope of the embodiments of the present application. For the convenience of understanding, the present application is exemplarily illustrated by taking the data access of the internal memory in the column unit as an example.

[0237] For the data storage mode shown in the foregoing FIG. 8A, the calculation instruction execution flow in the calculation stage can refer to the following implementation mode: the processor can issue a COMP_INT4 instruction to specify the column address col.k storing the high 2-bit data, and the internal calculation storage memory calculates the 128 W i,j H2b in the column (that is, the first sub-data), and transmits the calculation result into the internal shift circuit to be left shifted by 2 bits and then into the accumulator; the 128 W i,j L2b in the col.k+1 column (that is, the second sub-data) are read and calculated, and then the calculation result is transmitted into the accumulator to be accumulated with the calculation result of the shifted high 2-bit data, so as to achieve the purpose that the calculation circuit with the INT2 calculation precision completes the calculation of the weight parameter with the INT4 quantization precision.

[0238] Optionally, the above implementation manner can be completed based on the calculation unit structure shown in FIG. 2E. After the data in col.k (i.e., first sub-data) is calculated through the multiplication circuit, 128 multiplication results are obtained, which are sent to the shift circuit for shifting, and the shifted results are sent to the addition circuit for addition calculation, and then the added results (i.e., first results) are transmitted to the accumulator. After the data in col.k+1 (i.e., second sub-data) is calculated through the multiplication circuit, another 128 multiplication results are obtained, which are sent to the addition circuit for addition calculation, and then the added results (i.e., second results) are transmitted to the accumulator and added with the previous results. Optionally, the above implementation manner can also be completed based on the calculation unit structure shown in FIG. 2F. After the data in col.k (i.e., first sub-data) is calculated through the multiplication circuit, 128 multiplication results are obtained, which are sent to the addition circuit for addition calculation to obtain one added result, and then the added result is sent to the shift circuit for shifting, and the shifted results (i.e., third results) are transmitted to the accumulator. After the data in col.k+1 (i.e., second sub-data) is calculated through the multiplication circuit, another 128 multiplication results are obtained, which are sent to the addition circuit for addition calculation, and then the added results (i.e., fourth results) are transmitted to the accumulator and added with the previous results.

[0239] Based on this, for the above 4096 INT4 weight parameters W i,jThe calculation process can refer to the flow shown in FIG. 8B. The processor can first issue an activation instruction to activate a specified row rowi in Banka, and then issue 32 COMP INT4 calculation instructions in sequence to calculate the data in 64 columns included in rowi. Each COMP INT4 instruction calculates the data stored in a specified two columns. For example, the first COMP INT4 can calculate the data stored in col.0 and col.1, and after shifting the calculation result of col.0, the calculation result of col.1 is accumulated to complete the calculation of the first group of 128 INT4 weight parameters. The second COMP INT4 can calculate the data stored in col.2 and col.3, and after shifting the calculation result of col.2, the calculation result of col.3 is accumulated to complete the calculation of the second group of 128 INT4 weight parameters, which can be accumulated with the calculation result of the previous group of INT4 weight parameters. The third COMP INT4 can calculate the data stored in col.4 and col.5, and after shifting the calculation result of col.4, the calculation result of col.5 is accumulated to complete the calculation of the third group of 128 INT4 weight parameters, which can be accumulated with the calculation result of the previous groups of INT4 weight parameters. Similarly, the 32nd COMP INT4 can calculate the data stored in col.62 and col.63, and after shifting the calculation result of col.62, the calculation result of col.63 is accumulated to complete the calculation of the 32nd group of 128 INT4 weight parameters, which can be accumulated with the calculation result of the previous groups of INT4 weight parameters, thereby completing the calculation of 4096 INT4 weight parameters. Optionally, after completing the calculation of all data in rowi, the processor can issue a pre-charge instruction to close the row rowi.

[0240] It should be noted that for the calculation unit with INT2 calculation precision, since the weight parameters with INT4 quantization precision are split into two 2-bit data stored in different columns (cols), one COMP INT4 instruction takes out the data stored in two cols for calculation, and one COMP INT2 instruction takes out the data in one col for calculation. Therefore, the time delay of the COMP INT4 instruction is twice that of the COMP INT2 instruction, but the number of COMP INT4 instructions is more economical, which is half of the number of COMP INT2 instructions. It should be noted that in addition to the above-mentioned pre-defined COMP INT4 instruction for quantization precision sensing, the processor can also perform quantization precision sensing through the configuration mode register. At this time, the execution flow of the calculation instruction can refer to the execution flow of the COMP INT4 instruction in FIG. 8B, and the COMP INT4 instruction is replaced by the COMP instruction.

[0241] Optionally, in addition to the above data storage manner of splitting the same weight parameter and storing in adjacent columns in FIG. 8A, the 4096 INT4 weight parameters W i,j may also be stored in a partitioned manner according to the storage rule shown in FIG. 8C. It can be considered that the 4096 W i,j high 2-bit data are stored in the area of the 0th column to the 31st column (i.e., col.0-col.31, a total of 32 columns) (for example, indicated by 32 write instructions (corresponding to a plurality of first memory access instructions)), and the 4096 W i,j low 2-bit data are stored in the area of the 32nd column to the 63rd column (i.e., col.32-col.63, a total of 32 columns) (for example, indicated by 32 write instructions (corresponding to a plurality of first memory access instructions)), and W i,j represents an INT4 weight parameter, and the data length is 4 bits, the high 2-bit data can be represented as W i,j H2b , and the low 2-bit data can be represented as W i,j L2b .

[0242] For the data storage manner as shown in FIG. 8C, the calculation instruction execution flow in the calculation phase can refer to the following implementation manner: the processor can issue a COMP AC INT4 or ACT INT4 instruction, and the on-memory calculation storage sequentially takes out the data in each column from col.0 to perform calculation, and when the data stored in col.0-col.31 is calculated, the accumulated result is first sent to the internal shift circuit to left shift 2 bits, and then the calculation result after left shifting is transmitted to the accumulator to wait for the next calculation; then the data in each column from col.32 to col.63 is sequentially taken out to perform multiplication and accumulation calculation, and the calculation result of this part is also transmitted to the accumulator to be accumulated with the data calculation result after left shifting 2 bits, and thus the calculation of the 4096 INT4 weight parameters is completed. It should be noted that the execution flow of the COMP AC INT4 and ACT INT4 calculation instructions can be different, as shown in FIG. 8D, for the manner based on the COMP AC INT4 instruction, an activation instruction ACT needs to be issued before the COMP AC INT4 instruction is issued to perform row gating, and for the manner based on the ACT INT4 instruction, as shown in FIG. 8E, since the ACT INT4 instruction is a row gating command itself, there is no need to additionally issue an ACT instruction to perform row gating. Optionally, after the calculation based on the COMP AC INT4 or ACT INT4 calculation instruction is completed, the processor can close the opened row by issuing a pre-charge instruction.

[0243] Optionally, the above implementation manner can be completed based on the calculation unit structure shown in FIG. 2E. Data in any column of col. 0-col. 31 (i.e., first sub-data) is calculated by the multiplication circuit to obtain 128 multiplication results, the 128 multiplication results are sent to the shift circuit for shifting, the results after shifting are sent to the addition circuit for addition calculation, and the addition results (i.e., first results) are transmitted to the accumulator. Data in any column of col. 32-col. 63 (i.e., second sub-data) is calculated by the multiplication circuit to obtain another 128 multiplication results, the multiplication results are sent to the addition circuit for addition calculation, and the addition results (i.e., second results) are transmitted to the accumulator for accumulation with the previous results. Optionally, the above implementation manner can also be completed based on the calculation unit structure shown in FIG. 2F. Data in any column of col. 0-col. 31 (i.e., first sub-data) is calculated by the multiplication circuit to obtain 128 multiplication results, the multiplication results are sent to the addition circuit for addition calculation to obtain one addition result, and the addition result is sent to the shift circuit for shifting. The results after shifting (i.e., third results) are transmitted to the accumulator. Data in any column of col. 32-col. 63 (i.e., second sub-data) is calculated by the multiplication circuit to obtain another 128 multiplication results, the multiplication results are sent to the addition circuit for addition calculation to obtain another addition result (i.e., fourth result), and the addition result is transmitted to the accumulator for accumulation with the previous results.

[0244] It should be noted that in the data storage manner shown in FIG. 8C, one row of 64 columns is divided into two partitions (e.g., the partition of col. 0-col. 31 and the partition of col. 32-col. 63), and the high 2-bit data is stored in the partition of col. 0-col. 31, and the low 2-bit data is stored in the partition of col. 32-col. 63. Optionally, one row of 64 columns can be divided into more partitions, such as four partitions, including the partition a of col. 0-col. 15, the partition b of col. 16-col. 31, the partition c of col. 32-col. 47, and the partition d of col. 48-col. 63. The partition a can store 2048 high 2-bit data, the partition b can store 2048 low 2-bit data, the partition c can store other 2048 high 2-bit data, and the partition d can store other 2048 low 2-bit data. Understandably, the number of partitions of the storage part can also be other values, such as 8, 16, etc., which are not limited here. Understandably, when the number of rows / columns contained in each partition is 1, the data storage manner shown in FIG. 8A can be obtained by dividing one row of 64 columns into multiple partitions.

[0245] In a possible implementation, the above-mentioned shift processing can be performed by the processor in addition to being performed by the shift circuit in the calculation unit, which is applicable to the structure without the shift circuit in the calculation unit. Optionally, the processor can first receive the intermediate results (e.g., the fifth result and the sixth result) calculated by the memory for the plurality of sub-data, then perform shift processing on the intermediate result corresponding to the high-bit data, and finally add the shifted result (e.g., the seventh result) and the intermediate result (e.g., the sixth result) corresponding to the low-bit data to obtain the final result. For example, for the data storage mode shown in FIG. 8C, the implementation process of the shift processing performed by the processor can be as follows: the data in any column of col. 0 to col. 31 (i.e., the first sub-data) is calculated by the multiplication circuit to obtain 128 multiplication results, which are input into the addition circuit for addition calculation, and then the addition result (i.e., the fifth result) is output to the processor; the data in any column of col. 32 to col. 63 (i.e., the second sub-data) is calculated by the multiplication circuit to obtain another 128 multiplication results, which are input into the addition circuit for addition calculation, and then the addition result (i.e., the sixth result) is also output to the processor; after receiving the fifth result and the sixth result, the processor can first perform shift processing on the fifth result to obtain the shifted result (the seventh result), and then add the seventh result and the sixth result to obtain the final result.

[0246] Optionally, the processor can perform shift processing on the fifth result by using a shift coefficient, for example, fifth result * shift coefficient = seventh result. The shift coefficient is determined according to the initial position of the first sub-data in the to-be-processed data. For example, shift coefficient = 2 n , and n represents the number of bits by which the intermediate result corresponding to the first sub-data needs to be shifted. For example, 4-bit data of INT4 is split into 2-bit data of two INT2s, the intermediate result corresponding to the high 2-bit data needs to be shifted by 2 bits, and accordingly the shift coefficient is 4 (2 2 ); for another example, 8-bit data of INT8 is split into 4-bit data of two INT4s, the intermediate result corresponding to the high 4-bit data needs to be shifted by 4 bits, and accordingly the shift coefficient is 16 (2 4 ); for another example, 3-bit data of INT3 is split into 1-bit data of three INT1s, the intermediate result corresponding to the high 1-bit data needs to be shifted by 2 bits, and accordingly the shift coefficient is 4 (2 2 ), the intermediate result corresponding to the middle 1-bit data needs to be shifted by 1 bit, and accordingly the shift coefficient is 2 (2 1 ), and the intermediate result corresponding to the low 1-bit data does not need to be shifted.

[0247] Taking the data storage mode shown in FIG. 8C as an example, the 4-bit data of INT4 can be split into 2-bit data of two INT2, and the instruction execution flow of the processor for shift processing is briefly described. Referring to the flowchart shown in FIG. 13, the following steps can be included:

[0248] 1. Instruction preparation: generate a calculation instruction based on the information of high-bit data, and the calculation instruction can contain the physical address information of each block of data in the memory and the like.

[0249] 2. Send the input vector to the Global Buffer of the memory for use as the input of subsequent calculation.

[0250] 3. Send the calculation instruction to the memory, and the memory reads the data from the corresponding col in the DRAM array to the calculation input register, while the input vector to be calculated is also read from the Global Buffer to the calculation input register.

[0251] 4. After the data is ready, the memory starts to calculate the high-bit data in col.x to obtain the result x, and transmits the result x to the accumulator.

[0252] 5. For the high-bit data in other cols.y, repeat steps 1-4, and accumulate the previous result x in the accumulator until the high-bit data stored in 32 cols is calculated to obtain the intermediate result 1 (i.e., the fifth result), and return the intermediate result 1 to the processor (xPU, such as CPU, NPU or GPU, etc.).

[0253] 6. For the calculation of low-bit data, repeat the above steps 1-5, replace the high-bit data with low-bit data to obtain the intermediate result 2 (i.e., the sixth result), and also return the intermediate result 2 to the processor.

[0254] 7. Process the intermediate result 1 and the intermediate result 2 to obtain the final result of the two vector inner products. Final result = intermediate result 1 * 2 n + intermediate result 2, 2 n is the shift coefficient, and n is 2, i.e., the calculation result of the high 2-bit data needs to be left shifted by 2 bits.

[0255] Optionally, the plurality of sub-data obtained after splitting can be placed in different rows / columns in the same bank (as shown in FIG. 8C), or can be placed in different banks, for example, the 4-bit data of INT4 can be split into 2-bit data of 2 INT2s, one bank (such as bank 0) can place the high 2-bit data, and the other bank (such as bank 1) can place the low 2-bit data, as shown in FIG. 14A, the high and low bit data can be calculated concurrently, and the calculation results can be returned to the processor at the same time. The calculation process of split data stored in different banks can refer to the process shown in FIG. 14B, including the following steps:

[0256] 1. Transmit input vector: send the input vector to the designated area, such as Global Buffer.

[0257] 2. Instruction preparation: the calculation instruction contains the address of the data in bank 0 and the address of the data in bank 1.

[0258] 3. Send the calculation instruction, read the data from different banks corresponding to the col to the calculation input register, and also read the input vector from the Global Buffer to the calculation input register of each bank. The PIM starts calculation, and the calculation result of each col is accumulated to the value of each bank, to obtain the result 0 corresponding to bank 0 and the result 1 corresponding to bank 1.

[0259] 4. Repeat steps 1-3 until the data stored in 64 cols in 2 banks is calculated, to obtain the intermediate result 0 and the intermediate result 1, and return the intermediate result 0 and the intermediate result 1 to the processor (if the data transmitted in step 1 has not been calculated, it can not be executed).

[0260] 5. Calculate the intermediate result 0 and the intermediate result 1 to obtain the final result. Final result = intermediate result 0 * 2 n + intermediate result 1, 2 n is the shift coefficient, and n is 2, that is, the calculation result of the high 2-bit data needs to be left shifted by 2 bits.

[0261] The following will take the cooperation of the quantized perception calculation instruction COMP INT3 and the calculation unit with the calculation precision of INT2 to complete the calculation of INT3 quantized data as an example, and the control method in the embodiment of the application will be exemplarily described in combination with the processing process of the storage stage and the calculation stage. That is, taking the use of the INT2 calculation unit to complete the INT3 quantized data as an example.

[0262] For example, each row of the in-memory computing memory can contain 64 columns (cols), and each col can store 256 bits of data. Since 256 and 64 cannot be divided by 3, to ensure the utilization of each column of storage resources, when the INT3 weight parameters are stored by splitting, only 63 cols in a row can be used for storage, and thus a row can store 5376 (63 * 256 / 3) INT3 weight parameters. Then, the calculation result of all 5376 INT3 data in a row and the input vector V is shown in Equation Three:

[0263] Equation Three:

[0264] Exemplarily, in the storage stage, since the quantization precision of the weight parameters is greater than the calculation precision of the calculation unit (INT3 is greater than INT2), each weight parameter can be split into 1 bit of 2-bit data and 1 bit of 1-bit data, and the two split data are respectively stored in different 2 cols. For example, for the 5376 INT3 weight parameters W i,j can be stored according to the data storage rule shown in FIG. 9A, and the 5376 INT3 weight parameters are grouped into 21 groups of 256 weight parameters, wherein each group of 256 INT3 weight parameters W i,j The high 2 bits of the first 128 W i,j can be stored in the kth col (denoted as col.k, such as col.0) by one write instruction (corresponding to the first memory access instruction), the high 2 bits of the last 128 W i,j can be stored in the k+1th col (denoted as col.k+1, such as col.1) by one write instruction (corresponding to the first memory access instruction), and the low 1 bit of the 256 W i,j can be stored in the k+2th col (denoted as col.k+2, such as col.2) by one write instruction (corresponding to the first memory access instruction). W i,j represents an INT3 parameter, the high 2 bits of which are represented as W i,j H2b , and the low 1 bit of which is represented as W i,j L1b . Alternatively, the high 2 bits of the first 128 W i,j can also be stored in the k+1th col (or the k+2th col), the high 2 bits of the last 128 W i,j can also be stored in the k+2th col (or the kth col), and the low 2 bits of the 256 W i,j can also be stored in the kth col (or the k+1th col), which is not specifically limited herein.

[0265] Accordingly, the 256 INT3 weight parameters in each of the other 20 groups are stored according to the storage rule of the 256 W i,j in the first group, and finally the 256 W i,j data in each group are split and stored in different columns, and the high 2-bit data and the low 1-bit data of the weight parameters W i,j in the same group can be stored in adjacent three columns.

[0266] For the data storage mode shown in FIG. 9A, the calculation instruction execution flow in the calculation stage can refer to the following implementation mode: the processor issues a COMP INT3 instruction to specify the column address col.k storing the high 2-bit, and the storage calculation memory calculates the first 128 high 2-bit data W i,j H2b stored in the column to obtain the calculation result Res H2b1 ; the next 128 high 2-bit data W i,j H2b stored in col.k+1 are read and calculated to obtain the calculation result Res H2b2 , which is accumulated with Res H2b1 , and the accumulated result is sent to the internal shift circuit for left shift by 1 bit, and then the shifted result is transmitted into the accumulator; the 256 low 1-bit data W i,j L1b stored in col.k+2 are read and calculated, and the calculation result is transmitted into the accumulator and accumulated with the calculation result of the shifted high 2-bit data, and thus the calculation of the 256 INT3 weight parameters is completed.

[0267] Optionally, the above implementation manner can be completed based on the calculation unit structure shown in FIG. 2E, the data in col.k (i.e., part of the first sub-data) is calculated by the multiplication circuit to obtain 128 multiplication results and is sent to the shift circuit for shifting, the shifted result is sent to the addition circuit for addition calculation, and the added result is transmitted into the accumulator; the data in col.k+1 (i.e., part of the first sub-data) is calculated by the multiplication circuit to obtain another 128 multiplication results and is sent to the shift circuit for shifting, the shifted result is sent to the addition circuit for calculation, and the added result is transmitted into the accumulator and added with the previous result (i.e., to obtain the first result); the data in col.k+2 (i.e., the second sub-data) is calculated by the multiplication circuit to obtain 256 multiplication results, which are sent to the addition circuit for addition calculation, and the added result (i.e., the second result) is transmitted into the accumulator and added with the previous result. Optionally, the above implementation manner can also be completed based on the calculation unit structure shown in FIG. 2F, the data in col.k (i.e., part of the first sub-data) is calculated by the multiplication circuit to obtain 128 multiplication results, which are sent to the addition circuit for addition calculation to obtain 1 added result, and the added result is sent to the shift circuit for shifting, and the shifted result is transmitted into the accumulator; the data in col.k+1 (i.e., part of the first sub-data) is calculated by the multiplication circuit to obtain another 128 multiplication results, which are sent to the addition circuit for addition calculation to obtain another 1 added result, and the added result is sent to the shift circuit for shifting, and the shifted result is transmitted into the accumulator and added with the previous result (i.e., to obtain the third result); the data in col.k+2 (i.e., the second sub-data) is calculated by the multiplication circuit to obtain 256 multiplication results, which are sent to the addition circuit for addition calculation, and the added result (i.e., the fourth result) is transmitted into the accumulator and added with the previous result. Optionally, the shift processing operation in the above implementation manner can also be performed by the processor without relying on the shift circuit in the calculation unit, and the specific calculation process can refer to the related description of FIG. 13, FIG. 14A or FIG. 14B above, which is not further expanded here.

[0268] Based on this, for the above 5376 INT3 weight parameters W i,jThe calculation process can refer to the following flow: the processor can first issue an activation instruction to activate a specified row rowi in Banka, and then sequentially issue 21 COMP INT3 instructions to calculate the data in the 63 columns included in rowi. Each COMP INT3 instruction calculates the data stored in a specified three columns. The first COMP INT3 can calculate the data stored in col.0, col.1 and col.2, and after accumulating and shifting the calculation results of col.0 data and col.1 data, the calculation result of col.2 data is accumulated to complete the calculation of the first group of 256 INT3 weight parameters. The second COMP INT3 can calculate the data stored in col.3, col.4 and col.5, and after accumulating and shifting the calculation results of col.3 data and col.4 data, the calculation result of col.5 data is accumulated to complete the calculation of the second group of 256 INT3 weight parameters, which can be accumulated with the calculation results of the previous group of INT3 weight parameters. Similarly, the 21st COMP INT3 can calculate the data stored in col.60, col.61 and col.62, and after accumulating and shifting the calculation results of col.60 data and col.61 data, the calculation result of col.62 data is accumulated to complete the calculation of the 21st group of 256 INT3 weight parameters, which can be accumulated with the calculation results of the previous groups of INT3 weight parameters, thereby completing the calculation of 5376 INT3 weight parameters. Alternatively, after completing the calculation of all data in rowi, the processor can issue a pre-charge instruction to close the row rowi.

[0269] It should be noted that for a calculation unit with an INT2 calculation accuracy, since one COMP INT3 instruction calculates the data in three cols and one COMP INT2 instruction calculates the data in one col, the time delay of COMP INT3 instruction is twice that of COMP INT2 instruction, but the number of COMP INT3 instructions is less than that of COMP INT2 instructions.

[0270] Alternatively, in addition to the above data storage mode in which the same group of weight parameters is split and stored in adjacent columns in FIG. 9A, the 5376 INT3 weight parameters W i,j The data can also be stored in a partitioned manner according to the storage rule shown in FIG. 9B. The 5376 INT3 weight parameters W i,jof the 5376 INT3 weight parameters W, each column (col) can store 128 2-bit data; in addition, the 5376 INT3 weight parameters W are stored in the region of col. 42~col. 62 i,j of the 5376 INT3 weight parameters W, each column (col) can store 256 1-bit data.

[0271] For the data storage mode shown in FIG. 9B, the calculation instruction execution flow in the calculation stage can refer to the following mode a and mode b:

[0272] Mode a: the processor can issue a COMP INT3 instruction, the COMP INT3 instruction specifies the column address col.k of the high 2 bits, and the on-memory calculation storage sequentially executes the calculation of the first 256 high 2-bit data in the col.k and col.k+1 addresses, then the two column calculation results are accumulated and sent to the internal shift circuit for left shifting by 1 bit, and then the shifted result is transmitted to the accumulator; the first 256 low 1-bit data in the col.(k / 2+42) address are taken out for calculation, and the calculation result is transmitted to the accumulator for accumulation with the calculation result of the high 2 bits, thereby completing the calculation of 256 INT3 weight parameters. Correspondingly, the calculation process of the remaining 5120 (5376-256) INT3 weight parameters can refer to the calculation of the aforementioned 256 INT3 weight parameters, and the calculation of the remaining 20 groups (60 columns) of data is sequentially completed by 20 COMP INT3 instructions, and finally the calculation results of 21 groups (63 columns) are accumulated, thereby completing the calculation of the 5376 INT3 weight parameters.

[0273] Mode b: the processor can issue a COMP AC INT3 or ACT INT3 instruction, and the on-memory calculation storage sequentially takes out the data in each column for calculation from col. 0, and after the data in col. 0~col. 41 are calculated, the accumulated result is sent to the internal shift circuit for left shifting by 1 bit, and then transmitted to the accumulator; the multiplication and accumulation calculation of the data in col. 42~col. 62 is sequentially executed, and the calculation result of this part is also transmitted to the accumulator, and then accumulated with the calculation result of the above left shifting by 1 bit, thereby completing the calculation of the 5376 INT3 weight parameters.

[0274] Optionally, the above-mentioned mode a and mode b can be completed based on the calculation unit structure shown in FIG. 2E or FIG. 2F, and the calculation process of the calculation unit with different structures can refer to the related description in the above-mentioned several implementation modes of FIG. 8A, FIG. 8C or FIG. 9A, which will not be repeated here.

[0275] The following example illustrates the control method in this application embodiment by using the quantization sensing calculation instruction COMP_INT3 in conjunction with a calculation unit with a calculation precision of INT1 to complete the calculation of INT3 quantized data, combined with the processing procedures of the storage and calculation stages. Specifically, it uses the INT1 calculation unit to complete the INT3 quantized data calculation as an example.

[0276] For example, each row in the in-memory computing memory can contain 64 columns, and each column can store 256 bits of data. Using 63 columns in a row for storage, a row can store 5376 (63*256 / 3) INT3 weight parameters. Therefore, the calculation result of all 5376 INT3 data points in a row with the input vector V is shown in Formula 3 above.

[0277] Optionally, during the storage stage, since the quantization precision of the weight parameters is greater than the computational precision of the computational unit (INT3 is greater than INT1), each weight parameter can be split into three 1-bit data units, and these three 1-bit data units can be stored in three different cols. For example, for these 5376 INT3 weight parameters W... i,j The data storage rules shown in Figure 10A can be used as a reference. These 5376 INT3 weight parameters should be grouped into 21 groups of 256 weight parameters each. Each group of 256 INT3 weight parameters W... i,j It is possible to write 256 Ws using a single write instruction (corresponding to the first memory access instruction). i,j The high 1 bit is stored in the k-th column (denoted as col.k, such as col.0), and can be moved by a single write instruction (corresponding to the first memory access instruction). i,j The middle 2 bits are stored in the (k+1)th column (denoted as col.k+1, e.g., col.1), and can be moved by a single write instruction (corresponding to the first memory access instruction). i,j The lower 1 bit is stored in the (k+2)th column (represented as col.k+2, e.g., col.2), W i,j This represents an INT3 parameter, with the high 1 bit representing W. i,j H1b 1 bit is represented as W i,j M1b The lower 1 bit is represented as W i,j L1b Optionally, the high 1 bit of data can also be stored in the (k+1)th column (or the (k+2)th column), the middle 1 bit of data can also be stored in the (k+2)th column (or the kth column), and the low 1 bit of data can also be stored in the (k)th column (or the (k+1)th column), without any specific restrictions.

[0278] Accordingly, each of the 256 INT3 weight parameters in the other 20 groups is stored according to the storage rule of the first group of 256 W i,j , and finally the data of each of the 256 W i,j is split and stored in different 3 columns, which can be adjacent 3 columns.

[0279] For the data storage mode shown in FIG. 10A, the calculation instruction execution flow in the calculation stage can refer to the following implementation: the processor issues a COMP INT3 instruction to specify the column address col.k where the high 1 bit is stored, the on-chip calculation storage calculates the 256 high 1 bit data stored in the column, and the calculation result is transmitted to the internal shift circuit left shifted by 1 bit to obtain Res H1b ; the 256 middle 1 bit data stored in col.k+1 is automatically read to obtain the calculation result, which is accumulated with Res H1b , and the accumulated result is left shifted by 1 bit and transmitted to the accumulator; then the 256 low 1 bit data stored in col.k+2 is automatically read for calculation, and the calculation result is transmitted to the accumulator to be accumulated with the previous result, thereby completing the calculation of the 256 INT3 weight parameters. Alternatively, after the processor issues a COMP INT3 instruction, the on-chip calculation storage can directly left shift the calculation result of the 256 high 1 bit data by 2 bits through the shift circuit and transmit it to the accumulator; subsequently, the on-chip calculation storage can left shift the calculation result of the 256 middle 1 bit data by 1 bit through the shift circuit and transmit it to the accumulator to be accumulated with the shifted result of the high 1 bit data; finally, the on-chip calculation storage calculates the 256 low 1 bit data and transmits the calculation result to the accumulator to be accumulated with the previous result, thereby completing the calculation of the 256 INT3 weight parameters. Alternatively, the above implementation can be completed based on the calculation unit structure shown in FIG. 2E or FIG. 2F, and the calculation process of the calculation unit with different structures can refer to the related description in the above several implementation modes of FIG. 8A, FIG. 8C or FIG. 9A, which will not be described here. Alternatively, the shift processing operation in the above implementation can be performed by the processor without relying on the shift circuit in the calculation unit, and the specific calculation process can refer to the related description in FIG. 13, FIG. 14A or FIG. 14B, which will not be described here.

[0280] Based on this, the calculation process of the above 5376 INT3 weight parameters W i,j can refer to the following flow: the processor can first issue an activation instruction to activate a specified row rowi in Bank a, and then sequentially issue 21 COMP INT3 instructions to calculate the data in the 63 columns included in rowi, and each COMP INT3 instruction calculates the data stored in the specified 3 columns.

[0281] Optionally, in addition to the data arrangement manner of splitting the same group of weight parameters and storing in adjacent columns in FIG. 10A, the 5376 INT3 weight parameters W i,j The data can also be stored in a partition manner according to the storage rule shown in FIG. 10B. The 5376 INT3 weight parameters W i,j high 1-bit data can be stored in col. 0~col. 20, each column (col) can store 256 1-bit data; the 5376 INT3 weight parameters W i,j middle 1-bit data can be stored in col. 21~col. 41, each column (col) can store 256 1-bit data; and the 5376 INT3 weight parameters W i,j low 1-bit data can be stored in col. 42~col. 62, each column (col) can store 256 1-bit data.

[0282] For the data storage manner shown in FIG. 10B, the calculation instruction execution flow in the calculation stage can refer to the following manner c and manner d:

[0283] Manner c: the processor can issue a COMP INT3 instruction, the COMP INT3 instruction specifies the column address col.k of high 1-bit, after the storage memory performs the calculation of 256 high 1-bit data in col.k, the calculation result is shifted through the shift circuit, and the shifted result is transmitted into the accumulator; 256 middle 1-bit data in col.k+21 address are automatically taken out for calculation, the calculation result is shifted through the shift circuit, and the shifted result is transmitted into the accumulator and accumulated with the shifted result of high 1-bit data; 256 low 1-bit data in col.k+42 address are automatically taken out for calculation, and the calculation result is transmitted into the accumulator and accumulated with the calculation result of the previous 2-bit data, thus the calculation of 256 INT3 weight parameters is completed. Correspondingly, the calculation process of the remaining 5120 (5376-256) INT3 weight parameters can refer to the calculation of the aforementioned 256 INT3 weight parameters, and the calculation of the remaining 20 groups (60 columns) of data is sequentially completed through 20 COMP INT3 instructions, finally the calculation results of 21 groups (63 columns) are accumulated, thereby the calculation of 5376 INT3 weight parameters is completed.

[0284] Mode d: the processor can issue a COMP AC INT3 or ACT INT3 instruction, and the in-memory computing memory sequentially takes out 256 high 1-bit data in each column from col. 0 for calculation. After the data in col. 0 to col. 20 are all calculated, the obtained accumulation result is sent to the internal shift circuit and left shifted by 1 bit, and then transmitted to the accumulator; the calculation of 256 middle 1-bit data stored in each column in col. 21 to col. 41 is sequentially executed, and the calculation result of the middle 1-bit is accumulated with the shifted result of the high 1-bit and then left shifted by 1 bit and transmitted to the accumulator; the multiplication and accumulation calculation of 256 low 1-bit data stored in each column in col. 42 to col. 62 is sequentially executed, and the calculation result of this part is also transmitted to the accumulator and accumulated with the previous shifted result, so as to complete the calculation of 5376 INT3 weight parameters.

[0285] Optionally, the above-mentioned mode c and mode d can be completed based on the calculation unit structure shown in FIG. 2E or FIG. 2F. The calculation process of the calculation unit with different structures can refer to the related description in the above-mentioned several implementation modes of FIG. 8A, FIG. 8C or FIG. 9A, and will not be repeated here.

[0286] It should be noted that in addition to the above-mentioned combination of INT2 calculation unit and INT4 weight parameter, INT2 calculation unit and INT3 weight parameter, and INT1 calculation unit and INT3 weight parameter, there can be more combinations, and the calculation precision value of the calculation unit and the quantization precision value of the weight parameter are not limited in the embodiments of the present application. For example, the combination of INT2 calculation unit and INT5 weight parameter. For this combination, the 5-bit weight parameter can be split into 2+2+1, 2+1+2 or 1+2+2 and written into 3 rows / columns for storage. The calculation process in the calculation stage can refer to the calculation of the combination of INT1 calculation unit and INT3 weight parameter, and will not be expanded here.

[0287] The present application also provides a processor coupled with an in-memory, the in-memory comprising at least one storage area Bank and at least one calculation unit; each of the Banks comprises M rows*N columns of storage units, and the calculation precision of each of the calculation units is K, wherein M, N and K are all greater than 0; the processor is used for:

[0288] obtaining to-be-processed data; the quantization precision of the to-be-processed data is L;

[0289] when the quantization precision L of the to-be-processed data is greater than the calculation precision K of the calculation unit, performing data splitting on the to-be-processed data to obtain a plurality of sub-data; the quantization precision of each of the sub-data is less than or equal to the calculation precision K of the calculation unit;

[0290] a first memory access instruction is respectively sent for each of the plurality of sub-data according to a storage rule; the first memory access instruction is used to instruct the memory to write the sub-data into a specified row or column, and the specified row or column is different between different first memory access instructions in the plurality of first memory access instructions.

[0291] The application further provides an internal memory coupled with a processor, the internal memory comprising at least one storage area Bank and at least one calculation unit; each of the Banks comprises M rows*N columns of storage units, and the calculation precision of each of the calculation units is K, and M, N and K are all greater than 0; the method comprises:

[0292] a second memory access instruction is received, the second memory access instruction being used to instruct the internal memory to perform calculation on a plurality of sub-data; the plurality of sub-data are obtained by splitting the to-be-processed data by the processor when the quantization precision L of the to-be-processed data is greater than the calculation precision K of the calculation unit; and the quantization precision of the sub-data is less than or equal to the calculation precision K of the calculation unit;

[0293] a storage address of the plurality of sub-data is determined according to the second memory access instruction and a storage rule;

[0294] the plurality of sub-data are sequentially read from the storage address in response to the second memory access instruction;

[0295] a final result is determined by using the plurality of sub-data for calculation.

[0296] In addition, the application further provides an electronic device, characterized in that comprising one or more processors and one or more internal memories; the one or more processors are coupled with the one or more internal memories; the one or more processors are used to implement the functions related to any of the embodiments, for example, splitting data and issuing memory access instructions; and the one or more internal memories are used to implement the functions related to any of the embodiments, for example, receiving memory access instructions and performing data storage, data calculation and the like according to the memory access instructions.

[0297] The application further provides a computer program, which is used to implement the operations and / or processes performed by any of the processors and internal memories in the method provided by the application.

[0298] The application further provides a readable storage medium, wherein computer codes are stored in the readable storage medium, and when the computer codes are run on a computer, the computer codes make the computer perform the operations and / or processes performed by any of the processors and internal memories in the method provided by the application.

[0299] The application further provides a computer program product comprising computer code or a computer program which, when run on a computer, causes the operations and / or processes performed by any one of the processor and the internal memory in the method provided by the application to be performed.

[0300] In the above embodiments, the description of each embodiment has its own focus, and the parts not described in detail in a certain embodiment can be referred to the relevant description of other embodiments.

[0301] It should be noted that, for the foregoing method embodiments, in order to simply describe, they are all expressed as a combination of a series of actions, but those skilled in the art should know that the application is not limited to the order of the actions described, because according to the application, certain steps can be performed in other orders or at the same time. Secondly, those skilled in the art should know that the embodiments described in the specification all belong to preferred embodiments, and the actions and modules involved are not necessarily necessary for the application.

[0302] In several embodiments provided by the application, the coupling or direct coupling or communication connection between the displayed or discussed each other can be indirect coupling or communication connection through some interfaces, devices or units, and can also be electrical, mechanical or other form of connection.

[0303] The above is merely specific implementation of the application, but the protection scope of the application is not limited to this, any person skilled in the art can easily think of changes or replacements within the technical range disclosed by the application, which should be covered in the protection scope of the application. Therefore, the protection scope of the application should be subject to the protection scope of the claims.

Claims

1. A control method characterized by, The application is applied to a processor coupled with an internal memory, and the internal memory comprises at least one storage area Bank and at least one calculation unit; each Bank comprises M rows*N columns of storage units, and the calculation precision of each calculation unit is K, M, N and K are all integers greater than 0; the method comprises: obtaining to-be-processed data; the quantization precision of the to-be-processed data is L; when the quantization precision L of the to-be-processed data is greater than the calculation precision K of the calculation unit, performing data splitting on the to-be-processed data to obtain a plurality of sub-data; the quantization precision of the sub-data is less than or equal to the calculation precision K of the calculation unit; sending a first memory access instruction for the plurality of sub-data according to a storage rule; the first memory access instruction is used for instructing the internal memory to write the sub-data into a specified row or a specified column.

2. The method of claim 1, wherein, The storage rule specifically comprises that the specified row or the specified column to which the plurality of sub-data is written is an adjacent row or column; or the Bank comprises a plurality of partitions, and the storage rule specifically comprises that the specified row or the specified column to which the plurality of sub-data is written is located in different partitions; or the storage rule specifically comprises that the specified row or the specified column to which the plurality of sub-data is written is located in different Banks.

3. The method of any one of claims 1-2, wherein, The maximum data amount that can be stored in the specified row or the specified column is an integer multiple of the quantization precision of the sub-data.

4. The method of any one of claims 1-3, wherein, The method further comprises: sending a second memory access instruction; the second memory access instruction is used for instructing the internal memory to calculate the plurality of sub-data, and is used for instructing the quantization precision L of the to-be-processed data.

5. The method of any one of claims 1-3, wherein, The internal memory further comprises a mode register comprising an indication field used for indicating data quantization precision; the method further comprises: sending a control instruction; the control instruction is used for instructing the quantization precision L of the to-be-processed data, and instructing the internal memory to save the quantization precision L of the to-be-processed data in the indication field of the mode register.

6. The method of any one of claims 1-5, wherein, The method further comprises: obtaining initial weight parameters of a model; dividing the initial weight parameters to obtain one or more first parameter sets; determining one or more second parameter sets from the one or more first parameter sets; the initial weight parameters included in the second parameter set support storage into the internal memory; determining one or more third parameter sets from the one or more second parameter sets; the quantization precision of the initial weight parameters included in the third parameter set is all greater than the calculation precision of the calculation unit.

7. The method of any one of claims 1-6, wherein, The plurality of sub-data comprises first sub-data and second sub-data, the first sub-data is one or more high-order data in the to-be-processed data, and the second sub-data is one or more low-order data in the to-be-processed data; The method further comprises: receiving a fifth result and a sixth result output by the internal memory; the fifth result is determined based on the first sub-data, and the sixth result is determined based on the second sub-data; determining a seventh result based on a shift coefficient and the fifth result; the shift coefficient is determined based on the initial position of the first sub-data in the to-be-processed data; determine a final result based on the sixth result and the seventh result.

8. A control method characterized by, The method is applied to an internal memory coupled with a processor, and the internal memory comprises at least one storage area Bank and at least one calculation unit; each Bank comprises M rows*N columns of storage units, and the calculation precision of each calculation unit is K, and M, N and K are all greater than 0; the method comprises the following steps: receiving a second memory access instruction sent by the processor; the second memory access instruction is used to instruct the internal memory to perform calculation on a plurality of sub-data; the plurality of sub-data are a plurality of sub-data obtained by data splitting of the to-be-processed data when the quantization precision L of the to-be-processed data is greater than the calculation precision K of the calculation unit; the quantization precision of the sub-data is less than or equal to the calculation precision K of the calculation unit; determining storage addresses of the plurality of sub-data according to the second memory access instruction and a storage rule; reading the plurality of sub-data from the storage addresses in sequence in response to the second memory access instruction; performing calculation using the plurality of sub-data to determine a final result.

9. The method of claim 8, wherein, The storage rule specifically comprises that the specified rows or columns to which the plurality of sub-data are written are adjacent rows or columns; or the Bank comprises a plurality of partitions, and the storage rule specifically comprises that the specified rows or columns to which the plurality of sub-data are written are located in different partitions; or the storage rule specifically comprises that the specified rows or columns to which the plurality of sub-data are written are located in different Banks.

10. The method of claim 8 or 9, wherein, The plurality of sub-data comprises first sub-data and second sub-data, the first sub-data is one or more high-order data in the to-be-processed data, and the second sub-data is one or more low-order data in the to-be-processed data. The performing calculation using the plurality of sub-data to determine a final result comprises: performing multiplication calculation on the first sub-data and then performing shift addition to obtain a first result; performing multiplication and addition calculation on the second sub-data to obtain a second result; determining the final result based on the first result and the second result.

11. The method of claim 8 or 9, wherein, The plurality of sub-data comprises first sub-data and second sub-data, the first sub-data is one or more high-order data in the to-be-processed data, and the second sub-data is one or more low-order data in the to-be-processed data. The performing calculation using the plurality of sub-data to determine a final result comprises: performing multiplication and addition calculation on the first sub-data and then performing shift processing to obtain a third result; performing multiplication and addition calculation on the second sub-data to obtain a fourth result; determining the final result based on the third result and the fourth result.

12. The method of any one of claims 8-11, wherein, The maximum data amount that can be stored in the specified rows or columns to which the sub-data are written is an integer multiple of the quantization precision of the sub-data.

13. The method of any one of claims 8-12, wherein, The second memory access instruction is also used to indicate the quantization precision L of the to-be-processed data.

14. The method of any one of claims 8-13, wherein, The internal memory further comprises a mode register, and the mode register comprises an indication field used to indicate data quantization precision; the method further comprises the following steps: receive the control instruction sent by the processor; the control instruction is used to indicate the quantization precision L of the to-be-processed data, and indicate that the internal memory saves the quantization precision L of the to-be-processed data in the indication field of the mode register.

15. The method of any one of claims 8-14, wherein, When the calculation precision K of the calculation unit is J times of the quantization precision L of the to-be-processed data, J is an integer greater than 1; the method further comprises: divide the calculation resource of the calculation unit into J sub-resources; receive J to-be-processed data; process the J to-be-processed data in parallel through the J sub-resources; the quantization precision of the to-be-processed data is L.

16. The method of any one of claims 8-15, wherein, The plurality of sub-data comprises first sub-data and second sub-data, the first sub-data is one or more high-order data in the to-be-processed data, and the second sub-data is one or more low-order data in the to-be-processed data; The using the plurality of sub-data to determine the final result comprises: calculating the first sub-data to obtain a fifth result; calculating the second sub-data to obtain a sixth result; outputting the fifth result and the sixth result to the processor.

17. A processor, comprising: The device comprises a module or unit for executing the method of any one of claims 1-7.

18. An in-memory storage, comprising: The device comprises a module or unit for executing the method of any one of claims 8-16.

19. An electronic device, comprising: The device comprises one or more processors and one or more internal memories; the one or more processors are coupled with the one or more internal memories; the one or more processors are used to execute the method of any one of claims 1-7, and the one or more internal memories are used to execute the method of any one of claims 8-16.

20. A computer-readable storage medium, characterized in that, The storage medium stores a computer program or instructions, when the computer program or instructions are executed by a computer, the method of any one of claims 1-7 or 8-16 is realized.

21. A computer program, characterized in that, The computer program comprises instructions, when the computer program is executed by a computer, the method of any one of claims 1-7 or 8-16 is realized.

Images