Control method, related device and readable storage medium

Abstract

The invention relates to the technical field of terminals, in particular to a control method, a related device and a readable storage medium, and the method can comprise the following steps: obtaining to-be-processed data; when the quantization precision of the to-be-processed data is greater than the calculation precision of the calculation unit, performing data splitting on the to-be-processed data to obtain a plurality of sub-data; the quantization precision of each piece of sub-data in the multiple pieces of sub-data is smaller than or equal to the calculation precision of the calculation unit; respectively sending a first memory access instruction for the plurality of sub-data according to a storage rule; the first memory access instruction is used for indicating the internal memory to write the sub-data into a specified row or column. The embodiment of the invention can be compatible with mixed quantization precision calculation, and meanwhile, the circuit area overhead is saved.

Description

Technical Field

[0001] The present application relates to the field of terminal technology, and in particular to a control method, related devices and readable storage media. Background Art

[0002] With the rise of generative AI such as ChatGPT, deploying large models on the terminal side for AI reasoning to improve user experience and productivity is one of the main ways to enhance product competitiveness. When reasoning with large models, all model parameters are generally loaded into memory, so memory capacity often limits the size of models that can be deployed on the terminal side. At the same time, during the large model reasoning process, a large amount of data needs to be moved from memory to the processor (such as CPU, NPU, GPU, etc.) for processing, and memory bandwidth is also a bottleneck that limits the reasoning performance on the terminal side.

[0003] In order to reduce the demand for hardware resources of large models, an important direction of algorithm optimization is quantization technology. Quantization technology can reduce the size of the model by quantizing the model parameters, thereby reducing the demand for hardware resources of large models, speeding up the reasoning speed and reducing power consumption, but quantization may cause a loss of accuracy to the model. To this end, hybrid quantization accuracy can be used to allow the model to use higher accuracy for quantization in key parameter parts and lower accuracy for quantization in other parts, thereby reducing the demand for hardware resources of large models while maintaining the capabilities of large models. In addition, in order to reduce data movement during reasoning, processing in memory (PIM) technology can be used to offload part of the calculation to the memory. For example, the processing in memory (PIM-DRAM) implemented by integrating independent computing units in the dynamic random access memory (DRAM) chip (Die) can improve computing performance, accelerate large model reasoning, reduce energy consumption, and improve the endurance of terminal products by using memory to assist computing and utilizing the multi-Banks concurrency capability of DRAM memory.

[0004] At present, the solution of in-memory computing memory (PIM-DRAM) compatible with mixed quantization precision computing requires designing computing circuits with corresponding computing precisions for different quantization precisions. There may be a problem of needing to reserve more computing circuits, resulting in additional circuit area overhead.

[0005] To this end, the industry is exploring how to provide a solution that is compatible with mixed quantization precision calculations while saving circuit area overhead. Summary of the invention

[0006] The embodiments of the present application provide a control method, related devices and readable storage medium, which are compatible with mixed quantization precision calculations while saving circuit area overhead.

[0007] The present application is introduced below from different aspects. It should be understood that the implementation methods and beneficial effects of the following different aspects can be referenced to each other.

[0008] In a first aspect, the present application provides a control method, which can be applied to a processor, wherein the processor is coupled to an internal memory, wherein the internal memory includes at least one storage area Bank and at least one computing unit; each of the Banks includes M rows and N columns of storage units, and the computing precision of each of the computing units is K, and M, N, and K are all greater than 0; the method includes:

[0009] Acquire data to be processed; the quantization accuracy of the data to be processed is L;

[0010] When the quantization accuracy L of the data to be processed is greater than the calculation accuracy K of the calculation unit, the data to be processed is split into multiple sub-data; the quantization accuracy of the sub-data is less than or equal to the calculation accuracy K of the calculation unit;

[0011] A first memory access instruction is sent to each of the multiple sub-data according to the storage rule; the first memory access instruction is used to instruct the internal memory to write the sub-data into a specified row or column (the specified row or column is different between different first memory access instructions among the multiple first memory access instructions).

[0012] In an embodiment of the present application, when the quantization accuracy of the data to be processed is greater than the calculation accuracy of the calculation unit in the internal calculation memory, the subsequent internal memory will not be able to directly calculate the data to be processed through the calculation unit. The data to be processed can be split into multiple sub-data first, and the multiple sub-data are written into different rows or columns of the storage area for storage. The quantization accuracy of each sub-data is less than or equal to the calculation accuracy of the calculation unit, so that the internal memory can subsequently calculate each sub-data separately through the calculation unit. For calculations compatible with mixed quantization accuracy, it is only necessary to adjust the data splitting strategy according to different combinations of quantization accuracy and calculation accuracy, so that the calculation of multiple quantization accuracy parameters can be completed through a calculation circuit with one calculation accuracy. There is no need to specially design calculation circuits with corresponding calculation accuracy for data with different quantization accuracy, which not only saves the area overhead of the calculation circuit.

[0013] In one possible implementation, the storage rule specifically includes that the multiple sub-data are written into designated rows or designated columns as adjacent rows or columns; or, the Bank includes multiple partitions, and the storage rule specifically includes that the designated rows or designated columns into which the multiple sub-data are written are located in different partitions.

[0014] In an embodiment of the present application, the multiple sub-data after splitting can be stored in adjacent rows or columns, so that the calculation of the adjacent rows or columns of stored data can be completed through a single calculation instruction for subsequent calculations, which is suitable for scenarios where a small number of rows or columns of data need to be calculated; or, the multiple sub-data after splitting can be stored in rows or columns of multiple independent partitions, so that the calculation of multiple partitions of stored data can be completed through a single calculation instruction for subsequent calculations, which is suitable for scenarios where a large number of rows or columns of data need to be calculated.

[0015] In a possible implementation manner, the maximum amount of data that can be stored in the designated row or column is an integer multiple of the quantization accuracy of the sub-data.

[0016] In an embodiment of the present application, the maximum amount of data that can be stored in a row or column storing sub-data is an integer multiple of the sub-data quantization precision. For example, if the maximum amount of data that can be stored is 256 bits, the sub-data quantization precision can be 1 bit, 2 bits, 4 bits, etc., to ensure that the storage resources of the row or column can be fully utilized.

[0017] In a possible implementation, the method further includes:

[0018] Send a second memory access instruction; the second memory access instruction is used to instruct the internal memory to calculate the multiple sub-data.

[0019] In the embodiment of the present application, after instructing the internal memory to store the data to be processed, the processor can instruct the internal memory to use the internal computing unit to calculate the data to be processed through a memory access instruction.

[0020] In a possible implementation, the second memory access instruction is further used to indicate a quantization precision L of the data to be processed.

[0021] In the embodiment of the present application, the processor can enable the internal memory to perceive the quantization accuracy of the data to be processed through memory access instructions.

[0022] In a possible implementation, the internal memory further includes a mode register, and the mode register includes an indication field for indicating data quantization accuracy; and the method further includes:

[0023] Send a control instruction; the control instruction is used to indicate the quantization accuracy L of the data to be processed, and instruct the internal memory to save the quantization accuracy L of the data to be processed in the indication field of the mode register.

[0024] In the embodiment of the present application, the processor can configure the mode register in the internal memory so that the internal memory can sense the quantization accuracy of the data to be processed.

[0025] In a possible implementation, the method further includes:

[0026] When the quantization accuracy L of the data to be processed is less than or equal to the calculation accuracy K of the calculation unit, a third memory access instruction is sent; the third memory access instruction is used to instruct the internal memory to write the data to be processed into a specified row or column for storage.

[0027] In the embodiment of the present application, when the quantization accuracy of the data to be processed is less than or equal to the calculation accuracy of the calculation unit, the processor can directly write the data to be processed completely into the specified row / column for storage through the memory access instruction without the need to split the data. Accordingly, in the subsequent calculation process, the internal memory can calculate the data to be processed completely stored in the specified row / column through the calculation unit to directly obtain the final calculation result.

[0028] In a second aspect, the present application provides a control method, which can be applied to an internal memory, wherein the internal memory is coupled to a processor, and the internal memory includes at least one storage area Bank and at least one computing unit; each of the Banks includes M rows and N columns of storage units, and the computing precision of each of the computing units is K, and M, N, and K are all greater than 0; the method includes:

[0029] receiving a second memory access instruction sent by the processor; the second memory access instruction is used to instruct the internal memory to calculate a plurality of sub-data; the plurality of sub-data are a plurality of sub-data obtained by the processor performing data splitting on the data to be processed when the quantization accuracy L of the data to be processed is greater than the calculation accuracy K of the calculation unit; the quantization accuracy of the sub-data is less than or equal to the calculation accuracy K of the calculation unit;

[0030] Determine storage addresses of the plurality of sub-data according to the second memory access instruction and storage rule;

[0031] Reading the plurality of sub-data sequentially from the storage address in response to the second memory access instruction;

[0032] The plurality of sub-data are used to perform calculations to determine a final result.

[0033] In an embodiment of the present application, after the internal memory receives the calculation instruction sent by the processor, when the quantization accuracy of the data to be processed is greater than the calculation accuracy of the calculation unit in the internal calculation memory, the calculation unit can calculate the sub-data stored in multiple different rows / columns respectively to determine the final result, thereby achieving the purpose of completing high quantization accuracy data calculation through a calculation unit with low calculation accuracy, saving circuit area overhead.

[0034] In a possible implementation, the multiple sub-data include first sub-data and second sub-data, the first sub-data are one or more high-order data in the data to be processed, and the second sub-data are one or more low-order data in the data to be processed; and using the multiple sub-data to perform calculations to determine the final result includes:

[0035] Performing multiplication calculation on the first sub-data and then performing shift addition to obtain a first result;

[0036] performing multiplication and addition calculation on the second sub-data to obtain a second result;

[0037] The final result is determined based on the first result and the second result.

[0038] In an embodiment of the present application, some of the multiple sub-data are high-order data of the data to be processed, and some of the sub-data are low-order data of the data to be processed. After multiplication calculation is performed on the high-order data, shift and addition processing is required to obtain an intermediate result corresponding to the high-order data; multiplication and addition calculation is performed on the low-order data without shift processing to obtain an intermediate result corresponding to the low-order data, and then the final result is determined based on the two intermediate results to ensure the accuracy of the calculation.

[0039] In a possible implementation, the multiple sub-data include first sub-data and second sub-data, the first sub-data are one or more high-order data in the data to be processed, and the second sub-data are one or more low-order data in the data to be processed; and using the multiple sub-data to perform calculations to determine the final result includes:

[0040] Performing multiplication and addition calculations on the first sub-data and then performing shift processing to obtain a third result;

[0041] performing multiplication and addition calculation on the second sub-data to obtain a fourth result;

[0042] The final result is determined based on the third result and the fourth result.

[0043] In an embodiment of the present application, some of the multiple sub-data are high-order data of the data to be processed, and some of the sub-data are low-order data of the data to be processed. After the multiplication and addition calculations are performed on the high-order data, shift processing is required to obtain an intermediate result corresponding to the high-order data; after the multiplication and addition calculations are performed on the low-order data, no shift processing is required to obtain an intermediate result corresponding to the low-order data, and the final result is determined based on the two intermediate results to ensure the accuracy of the calculation.

[0044] In a possible implementation manner, the number of shift bits of each intermediate result in the multiple intermediate results corresponding to the multiple sub-data is determined based on an initial position of the corresponding sub-data in the data to be processed.

[0045] In an embodiment of the present application, when the internal memory shifts multiple intermediate results through a shift circuit, it can determine the initial position of the sub-data corresponding to each intermediate result in the original data (the data to be processed when not split) to ensure the accuracy of the final result.

[0046] In one possible implementation, the storage rule specifically includes that the specified rows or specified columns to which the multiple sub-data are written are adjacent rows or columns; or, the Bank includes multiple partitions, and the storage rule specifically includes that the specified rows or specified columns to which the multiple sub-data are written are located in different partitions.

[0047] In a possible implementation, the maximum amount of data that can be stored in a specified row or column where the sub-data is written is an integer multiple of the quantization accuracy of the sub-data.

[0048] In a possible implementation, the second memory access instruction is further used to indicate a quantization precision L of the data to be processed.

[0049] In a possible implementation, the internal memory further includes a mode register, and the mode register includes an indication field for indicating data quantization accuracy; and the method further includes:

[0050] Receive a control instruction sent by the processor; the control instruction is used to indicate the quantization precision L of the data to be processed, and instruct the internal memory to save the quantization precision L of the data to be processed in the indication field of the mode register.

[0051] In a possible implementation, the method further includes:

[0052] When the quantization accuracy L of the data to be processed is less than or equal to the calculation accuracy K of the calculation unit, a third memory access instruction sent by the processor is received; the third memory access instruction is used to instruct the internal memory to write the data to be processed into a specified row or column for storage.

[0053] In a possible implementation, when the calculation accuracy K of the calculation unit is J times the quantization accuracy L of the data to be processed, J is an integer greater than 1; the method further includes:

[0054] Dividing the computing resources of the computing unit into J sub-resources;

[0055] Receive J data to be processed;

[0056] The J data to be processed are processed in parallel by using the J sub-resources; the quantization accuracy of the data to be processed is L.

[0057] In an embodiment of the present application, when the quantization accuracy of the data to be processed is less than the calculation accuracy of the computing unit, and the calculation accuracy is an integer multiple of the quantization accuracy, the computing resources of the computing unit can be divided into multiple parts during calculation, so that multiple resources of the computing unit can be calculated simultaneously, thereby improving computing efficiency.

[0058] In a third aspect, the present application provides a control method, which can be applied to an internal memory, wherein the internal memory is coupled to a processor, and the internal memory includes at least one storage area Bank and at least one computing unit; each of the Banks includes M rows and N columns of storage units, and the computing accuracy of each of the computing units is K, and M, N, and K are all greater than 0; the method includes:

[0059] When the calculation accuracy K of the calculation unit is J times the quantization accuracy L of the data to be processed, the calculation resources of the calculation unit are divided into J sub-resources; J is an integer greater than 1;

[0060] Receive J data to be processed;

[0061] The J data to be processed are processed in parallel by using the J sub-resources; the quantization accuracy of the data to be processed is L.

[0062] In a fourth aspect, the present application provides a processor, characterized in that the processor is coupled to an internal memory, the internal memory includes at least one storage area Bank and at least one computing unit; each of the Banks includes M rows and N columns of storage units, the computing precision of each computing unit is K, and M, N, and K are all integers greater than 0; the processor is used to:

[0063] Acquire data to be processed; the quantization accuracy of the data to be processed is L;

[0064] When the quantization accuracy L of the data to be processed is greater than the calculation accuracy K of the calculation unit, the data to be processed is split into multiple sub-data; the quantization accuracy of the sub-data is less than or equal to the calculation accuracy K of the calculation unit;

[0065] The first memory access instructions are respectively sent for the multiple sub-data according to the storage rules; the first memory access instructions are used to instruct the internal memory to write the sub-data into a specified row or column, and the specified rows or columns of different first memory access instructions among the multiple first memory access instructions are different.

[0066] In a fifth aspect, the present application provides an internal memory, characterized in that the internal memory is coupled to a processor, and the internal memory includes at least one storage area Bank and at least one computing unit; each of the Banks includes M rows and N columns of storage units, and the computing precision of each of the computing units is K, and M, N, and K are all greater than 0; the method includes:

[0067] receiving a second memory access instruction sent by the processor; the second memory access instruction is used to instruct the internal memory to calculate a plurality of sub-data; the plurality of sub-data are a plurality of sub-data obtained by the processor performing data splitting on the data to be processed when the quantization accuracy L of the data to be processed is greater than the calculation accuracy K of the calculation unit; the quantization accuracy of the sub-data is less than or equal to the calculation accuracy K of the calculation unit;

[0068] Determine storage addresses of the plurality of sub-data according to the second memory access instruction and storage rule;

[0069] Reading the plurality of sub-data sequentially from the storage address in response to the second memory access instruction;

[0070] The plurality of sub-data are used to perform calculations to determine a final result.

[0071] In a sixth aspect, the present application provides an electronic device, characterized in that it includes one or more processors and one or more internal memories; the one or more processors are coupled to the one or more internal memories; the one or more processors are used to execute the method described in any possible implementation manner in the first aspect, and the one or more internal memories are used to execute the method described in any possible implementation manner in the second aspect or the third aspect.

[0072] In the seventh aspect, the present application provides a computer-readable storage medium, characterized in that a computer program or instructions are stored in the storage medium, and when the computer program or instructions are executed by a computer, the method described in any possible implementation manner in the first aspect, the second aspect or the third aspect is implemented.

[0073] In an eighth aspect, the present application provides a computer program, characterized in that the computer program includes instructions, and when the computer program is executed by a computer, it implements the method described in any possible implementation manner in the first aspect, the second aspect or the third aspect.

[0074] Optionally, the processor and internal memory may be physically independent units, or the memory may be integrated with the processor.

[0075] It should be noted that the technical effects achieved in the above-mentioned aspects can be referenced to each other or to the beneficial effects in the method embodiments shown below, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0076] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the background technology, the drawings required for use in the embodiments of the present application or the background technology will be described below.

[0077] Figure 1 It is a schematic diagram of the structure of an in-memory computing memory;

[0078] Figure 2A is a structural schematic diagram of an electronic device provided in an embodiment of the present application;

[0079] Figure 2B It is a schematic diagram of a memory structure using in-memory computing technology provided in an embodiment of the present application;

[0080] Figure 2C It is a schematic diagram of the structure of a PIM-Die provided in an embodiment of the present application;

[0081] Figure 2D It is a schematic diagram of the structure of a Bank provided in an embodiment of the present application;

[0082] Figure 2E is a schematic diagram of the structure of a computing unit provided in an embodiment of the present application;

[0083] Figure 2F is a schematic diagram of the structure of another computing unit provided in an embodiment of the present application;

[0084] Figure 3 It is a flow chart of a control method provided in an embodiment of the present application;

[0085] Figure 4 is a flow chart of another control method provided in an embodiment of the present application;

[0086] Figure 5 This is a schematic diagram of a data calculation process for a combination of INT2 calculation accuracy and INT2 quantization accuracy provided in an embodiment of the present application;

[0087] Figure 6 This is a schematic diagram of data splitting with INT8 quantization accuracy provided in an embodiment of the present application;

[0088] Figure 7A-7D It is a schematic diagram of a calculation process of data splitting provided in an embodiment of the present application;

[0089] Fig. 8A This is a schematic diagram of data splitting and storage of a combination of INT2 calculation accuracy and INT4 quantization accuracy provided in an embodiment of the present application;

[0090] Figure 8B This is a schematic diagram of a data calculation process for a combination of INT2 calculation accuracy and INT4 quantization accuracy provided in an embodiment of the present application;

[0091] Figure 8CThis is another data splitting and storage diagram of a combination of INT2 calculation precision and INT4 quantization precision provided in an embodiment of the present application;

[0092] Fig.8D This is a schematic diagram of a data calculation process of another combination of INT2 calculation accuracy and INT4 quantization accuracy provided in an embodiment of the present application;

[0093] Fig. 8E This is a schematic diagram of a data calculation process for another combination of INT2 calculation accuracy and INT4 quantization accuracy provided in an embodiment of the present application;

[0094] Fig.9A This is a schematic diagram of data splitting and storage of a combination of INT2 calculation accuracy and INT3 quantization accuracy provided in an embodiment of the present application;

[0095] Fig. 9B This is another data splitting and storage diagram of a combination of INT2 calculation precision and INT3 quantization precision provided in an embodiment of the present application;

[0096] Fig. 10A This is a schematic diagram of data splitting and storage of a combination of INT1 calculation accuracy and INT3 quantization accuracy provided in an embodiment of the present application;

[0097] Fig. 10B This is a data splitting and storage diagram of another combination of INT1 calculation precision and INT3 quantization precision provided in an embodiment of the present application. DETAILED DESCRIPTION

[0098] The technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application.

[0099] In the description of the present application, words such as "first" and "second" are only used to distinguish different objects, and do not limit the quantity and execution order, and words such as "first" and "second" do not necessarily limit them to be different. For example, the first memory access instruction and the second memory access instruction are only used to distinguish different memory access instructions, and do not limit their order. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not limited to the listed steps or units, but optionally also includes steps or units that are not listed, or optionally also includes other steps or units inherent to these processes, methods, products or devices.

[0100] In the description of this application, unless otherwise specified, " / " means "or", for example, A / B can mean A or B. "And / or" in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, "at least one (item)", "the following one (item) or more (items)" or similar expressions refer to any combination of these items, including any combination of single items (items) or plural items (items). For example, at least one item (item) of a, b, or c can mean: a, b, c; a and b; a and c; b and c; or a, b and c. Among them, a, b, c can be single or multiple.

[0101] In the description of this application, words such as "exemplary", "exemplarily" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described in this application as "exemplary", "for example" or "for example" should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary", "for example" or "for example" is intended to present related concepts in a concrete way.

[0102] It can be understood that in the description of this application, "when", "if" and "if" all mean that the device will take corresponding actions under certain objective circumstances, but do not limit the time, nor do they require that the device must have a judgment action when it is implemented, nor do they mean that there are other limitations.

[0103] The term “simultaneously” in the present application may be understood as at the same time point, within a period of time, or within the same cycle, and may be understood in conjunction with the context.

[0104] Elements used in the singular herein are intended to mean "one or more" rather than "one and only one" unless specifically stated otherwise.

[0105] It can be understood that in each embodiment of the present application, "A and B correspond" means that B is associated with A, and B can be determined according to A. However, it should also be understood that determining B according to A does not mean determining B only according to A, and B can also be determined according to A and / or other information.

[0106] It can be understood that in the embodiments of the present application, "used to indicate" and "indicate" can include direct indication and indirect indication, and can also include explicit indication and implicit indication. When describing "a certain indication information is used to indicate A" or "indication information of A", it can include that the indication information directly indicates A or indirectly indicates A, but it does not mean that the indication information must carry A. The information indicated by a certain information is called information to be indicated. In the specific implementation process, there are many ways to indicate the information to be indicated, such as but not limited to, the information to be indicated can be directly indicated, such as the information to be indicated itself or the index of the information to be indicated. The information to be indicated can also be indirectly indicated by indicating other information, wherein there is an association relationship between the other information and the information to be indicated. It is also possible to indicate only a part of the information to be indicated, while the other parts of the information to be indicated are known or agreed in advance. For example, the indication of specific information can also be realized by means of the arrangement order of each information agreed in advance (for example, stipulated by the protocol), thereby reducing the indication overhead to a certain extent. At the same time, the common parts of each information can also be identified and uniformly indicated to reduce the indication overhead caused by indicating the same information separately. In addition, the specific indication method can also be various existing indication methods, such as but not limited to the above-mentioned indication methods and various combinations thereof. The specific details of various indication methods can refer to the prior art, and will not be repeated herein. As described above, it can be seen that, for example, when it is necessary to indicate multiple information of the same type, different indication methods may be used for different information. In the specific implementation process, the required indication method can be selected according to specific needs. The embodiment of the present application does not limit the selected indication method. In this way, the indication method involved in the embodiment of the present application should be understood to cover various methods that can enable the party to be indicated to know the information to be indicated. The information to be indicated can be sent as a whole, or it can be divided into multiple sub-information and sent separately, and the sending period and / or sending time of these sub-information can be the same or different. The specific sending method is not limited in this application.

[0107] In order to better understand the technical solutions of the embodiments of the present application, several terms or nouns related to the present application are briefly introduced below to facilitate understanding by those skilled in the art.

[0108] (1) Dynamic random access memory (DRAM) generally includes multiple storage areas (Banks), and each Bank can include storage cells composed of multiple rows (rows) and multiple columns (cols). DRAM is usually used as memory. Accessing the storage cells of the memory requires two steps. The first step is to find the address of a row in the Bank. Specifically, the Bank and row address to be accessed can be specified through the row selection command; the second step is to find the address of a specific column in the selected row. Specifically, the col address to be accessed can be specified through the column selection command. Generally, the last operation on the row is the precharge operation, which is also called closing the open row. After completing this operation, the new row can be accessed.

[0109] (2) The basic commands commonly used to access memory include activation, read, write, precharge and refresh. Among them, the activation command can be used to select a row of storage cells in an idle state and put it into a state ready for reading / writing. The read and write commands can be used to read and write data to specific columns in the activated row respectively. The precharge command can be used to close the originally opened row if other rows need to be addressed after executing the read and write commands. The refresh command can be used to refresh all rows according to a fixed cycle to prevent long-term storage capacitor charge leakage, resulting in data loss.

[0110] (3) Compared with ordinary DRAM, in addition to the storage area for storing data and other peripheral circuits for reading and writing data, the PIM-DRAM also integrates an independent processing unit (PU). PIM-DRAM can be used as ordinary DRAM. During use, the calculation logic can be bypassed to directly read and write data; or by issuing calculation commands, the calculation unit in the PIM-DRAM can be used to perform in-memory calculations to reduce data movement. In the embodiment of the present application, when the data precision of the data to be processed is greater than the calculation precision of the calculation unit in the in-memory calculation memory, the data to be processed can be split into multiple sub-data, and the multiple sub-data can be written into different rows or columns in the storage area for storage, and the data precision of each sub-data is less than or equal to the calculation precision of the calculation unit. After receiving the calculation instruction, the multiple sub-data stored in multiple different rows / columns can be taken out, respectively input into the calculation unit for calculation and correspondingly obtain multiple intermediate results, and the multiple intermediate results are shifted by the shift circuit and then transmitted to the accumulator for calculation, thereby achieving the purpose of completing high-precision data calculation by the calculation unit with low calculation precision, saving circuit area overhead.

[0111] (4) Quantization technology is a method to reduce the model calculation and storage overhead by reducing the number of bits used to represent model parameters. Traditional deep learning models usually use floating-point numbers to represent parameters and activation values, which can achieve higher accuracy but also higher computational costs. The goal of quantization technology is to reduce the number of bits used to represent parameters while maintaining model performance as much as possible, thereby achieving higher efficiency in calculation and storage.

[0112] The core idea of ​​quantization technology is to map the parameters and activation values ​​in the model to a smaller value range, usually a limited integer or a low-bit floating-point number. This mapping can be achieved in many ways, such as fixed-point quantization, floating-point quantization, and symmetric quantization. In fixed-point quantization, parameters and activation values ​​are mapped to integers within a fixed range. By limiting the value range, fewer bits can be used to represent each value, thereby reducing storage and computational overhead. Floating-point quantization is to map parameters and activation values ​​to floating-point representations with lower bits. For example, half-precision floating-point numbers (16 bits) can be used instead of standard single-precision floating-point numbers (32 bits), thereby reducing the number of bits for parameter representation. Symmetric quantization is a special quantization method in which parameters and activation values ​​are symmetrical around the zero point, which can keep the quantized values ​​balanced in positive and negative values, thereby reducing information loss. However, quantization may lead to a loss of model accuracy. In order to reduce the model's demand for hardware resources while maintaining the model's capabilities, mixed quantization accuracy can be used to allow the model to quantize key parameters with higher accuracy and quantize other parts with lower accuracy.

[0113] First, the technical problems to be solved by this application are analyzed and proposed. Figure 1, the in-memory computing memory PIM-DRAM integrates computing units (processing units) near the internal banks. The weight parameters W of the large model can be stored in the banks, and the input vector V can be passed from the host (or processor) to the PIM-DRAM and can be stored in the global buffer. To complete the multiplication and accumulation operation of W*V, the computing unit may include a multiplication circuit (including multiple multipliers), an addition circuit (including multiple adders), and an accumulator. Taking the example that a column inside the in-memory computing memory can store 256-bit data, when the quantization accuracy of the weight parameter is integer 2 bits (Integer 2bit, INT2), a column can store 128 (i.e., 256 / 2) weight parameters; when the quantization accuracy of the weight parameter is integer 4 bits (Integer4bit, INT4), a column can store 64 (i.e., 256 / 4) weight parameters. Therefore, when PIM-DRAM needs to be compatible with the calculation of two mixed quantization accuracy of INT2 / INT4, the computing unit needs to include at least 128 computing circuits that support INT4. When the quantization accuracy of the weight parameter is INT2, 128 INT4 calculation circuits are required for calculation, but only 2 bits (such as high 2 bits or low 2 bits) of the 4 bits of each INT4 calculation circuit are used for calculation; when the quantization accuracy of the weight parameter is INT4, 64 INT4 calculation circuits are required for calculation, and 4 bits of each INT4 calculation circuit are used.

[0114] exist Figure 1 In the scheme shown, in order to be compatible with the calculation of mixed quantization precision (such as INT2 and INT4), it is necessary to design corresponding calculation circuits for different quantization precisions. For example, one column can store 128 INT2 parameters, and correspondingly at least 128 INT2 calculation circuits are required. One column can store 64 INT4s, and correspondingly at least 64 INT4 calculation circuits are required. Therefore, when realizing the compatible calculation of mixed quantization precision, it is necessary to take into account the requirements of INT2 precision for the number of calculation circuits and INT4 precision for the accuracy. The calculation unit needs to include at least 128 INT4 calculation circuits. However, in the actual calculation process, whether it is the calculation of INT2 parameters or the calculation of INT4 parameters, the 128 INT4 calculation circuits cannot be fully utilized, resulting in additional circuit area overhead. In addition, the above scheme also has the problem of poor flexibility. For example, after the design of the calculation unit is determined, the quantization precision that the storage area can support for storing parameters is also fixed, and the accuracy of the model is also fixed.

[0115] To this end, the embodiment of the present application provides a control method, a related device and a readable storage medium. When it is necessary to use the in-memory computing memory for calculation, if the quantization accuracy of the data to be processed is greater than the calculation accuracy of the calculation unit in the in-memory computing memory, the multi-bit data with higher quantization accuracy can be split into multiple sub-data with fewer bits, and the quantization accuracy of each sub-data with fewer bits is less than or equal to the calculation accuracy of the calculation unit, so as to ensure that the calculation unit can calculate each sub-data, and then perform internal shift processing on the calculation results corresponding to each sub-data and accumulate them, and finally complete the calculation of multi-bit data with higher quantization accuracy. For calculations compatible with mixed quantization accuracy, it is only necessary to adjust the data splitting strategy according to different combinations of quantization accuracy and calculation accuracy, so as to realize the calculation of multiple quantization accuracy parameters through a calculation circuit with one calculation accuracy, without the need to specially design calculation circuits with corresponding calculation accuracy for data with different quantization accuracy, which not only saves the area overhead of the calculation circuit, but also improves the flexibility of the quantization accuracy of the data stored in the in-memory computing memory, thereby ensuring that the accuracy of the model can be flexibly adjusted according to the needs of different scenarios.

[0116] Optionally, when the quantization accuracy of the data to be processed is greater than the calculation accuracy of the calculation unit in the in-memory calculation memory, the data to be processed can be split into multiple sub-data, and the multiple sub-data are respectively written into different rows or columns of the storage area for storage, and the quantization accuracy of each sub-data is less than or equal to the calculation accuracy of the calculation unit. After receiving the calculation instruction sent by the processor, the in-memory calculation memory can calculate the sub-data stored in multiple different rows / columns through the calculation unit to obtain the corresponding multiple intermediate results, and then shift the multiple intermediate results and pass them into the accumulator for calculation, so as to achieve the purpose of completing the calculation of high quantization accuracy data through the calculation unit with low calculation accuracy, saving circuit area overhead.

[0117] For ease of understanding, the technical solution provided by this application will be described below in conjunction with more drawings.

[0118] In this application, unless otherwise specified, the same or similar parts between various embodiments or implementations can refer to each other. In each embodiment of this application, and each implementation method / implementation method / implementation method in each embodiment, if there is no special explanation and logical conflict, the terms and / or descriptions between different embodiments and each implementation method / implementation method / implementation method in each embodiment are consistent and can be referenced to each other. The technical features in different embodiments and each implementation method / implementation method / implementation method in each embodiment can be combined to form new embodiments, implementation methods, implementation methods, or implementation methods according to their inherent logical relationships. The implementation methods of this application described below do not constitute a limitation on the scope of protection of this application.

[0119] The following is an exemplary description of the applicable scenarios of the embodiments of the present application. It is understandable that the structure and application scenarios of the electronic device described in the embodiments of the present application are for the purpose of more clearly illustrating the technical solutions of the embodiments of the present application, and do not constitute a limitation on the technical solutions provided by the embodiments of the present application.

[0120] See also Figure 2A , Figure 2A : is a structural diagram of an electronic device provided in an embodiment of the present application. The electronic device 01 provided in the present application may include one or more processors 11, one or more memory controllers 12 and one or more memories 13. The electronic device may also include a power management unit, an external memory, etc. (not shown in the figure). The electronic device 01 may be a subscriber unit (Subscriber Unit), a cellular phone (Cellular Phone), a smart phone (Smart Phone), a personal digital assistant (Personal Digital Assistant, PDA) computer, a tablet computer, a handheld device (Handset), a laptop computer (Laptop Computer), a machine type communication (Machine Type Communication, MTC) terminal, an automatic driving terminal, a smart wearable device, a smart speaker and other mobile smart terminals, or various types of non-mobile terminal devices (i.e., fixed terminals) that need to use memory, which are not specifically limited here.

[0121] The processor 11 may be a central processing unit (CPU), a graphics processing unit (GPU), a neural processing unit (NPU), a system on chip (SOC) and a microcontroller (MCU), or a chip with other functions. The processor 11 may exchange data with the memory 13 through the memory controller 12. For example, when the CPU executes a memory access instruction, it may first send a read / write request to the memory management unit (MMU, not shown in the figure). After converting the virtual address and the physical address, the memory management unit sends the memory access command to the bus. The bus then passes the memory access command to the memory controller 12. The memory controller 12 translates the address again to access the data of the corresponding row / column in the memory 13.

[0122] The memory controller 12 may be a dynamic memory controller (DMC), which is mainly used to control the data transmission between the processor 11 and the memory 13. The memory controller 12 may be deployed on the same chip as the processor 11, or on a separate chip, that is, the memory controller 12 may be integrated into the processor 11 (such as a CPU) chip, or may be a separate chip independent of the processor 11, which is not specifically limited here. Generally, the memory controller may include a command processing module, a data processing module, and a refresh module, wherein the command processing module is used to process the instructions issued by the processor 11; the data processing module is used to process the data read or written into the memory 13 by the processor 11; and the refresh module is used to refresh the memory 13 according to the refresh command. The processor 11 may send memory access instructions (such as read / write instructions), control instructions, etc. to the memory 13 through the memory controller 12.

[0123] The memory 13 may be an internal memory (referred to as an in-memory computing memory) using in-memory computing technology, including a storage part and a computing part. The structure of the storage part in the memory 13 may be a dynamic random access memory (DRAM) structure. Generally, the memory 13 is a place for temporarily storing processes and data, and may also be referred to as a main memory or an internal memory. It may be made of semiconductor devices, and is a storage space that can be directly addressed by the processor 11 or other external devices with various functions (timers, analog-to-digital converters, etc., not shown in the figure). In an embodiment of the present application, the memory 13 may be an in-memory computing memory, the storage part of which may be used to store the weight parameters of a large model, and the computing part of which may be used to perform calculations. For example, when the quantization accuracy of the weight parameter is greater than the computing accuracy of the computing part, the weight parameter may be split into a plurality of sub-data of smaller accuracy and stored in different rows / columns, and the data accuracy of each sub-data is less than or equal to the computing accuracy of the computing part, and the subsequent computing part may calculate each sub-data separately in units of rows / columns.

[0124] For ease of understanding, the following is an illustrative description of the structure of the in-memory computing memory. Figure 2B , Figure 2B This is a schematic diagram of a memory structure using in-memory computing technology provided by an embodiment of the present application. Memory 13 may include one or more ranks. A rank refers to a set of memory chips (Die). The same group of memory chips share the same address and control signals. Each rank is usually 64 bits wide, which means it can process 64 bits of data at the same time. For example, memory 13 may include two ranks, each of which can be accessed independently, thereby improving the bandwidth and performance of memory 13. Figure 2BAs shown, Rank1 can include multiple Dies, and Rank1 can be used independently to access data; Rank2 can include multiple PIM-Dies, and PIM-Die is a Die with a computing unit (PU) integrated inside. Rank2 can be used independently to access data and perform computing tasks. It should be noted that the same Rank can include multiple Dies with the same structure or multiple Dies with different structures, which is not specifically limited in this application.

[0125] For easier understanding, the following is an example of the structure of PIM-Die. Figure 2C , Figure 2C is a schematic diagram of a PIM-Die structure provided in an embodiment of the present application. Each PIM-Die may include multiple storage areas (Banks), such as Figure 2C BankA-BankJ shown in the figure. Bank is the basic unit for storing and managing data. Each Bank can be regarded as a storage area that can independently process data requests. Different Banks allow parallel access and operation, which means that the memory controller 12 can access multiple Banks at the same time, thereby improving data reading efficiency. For example, a PIM-Die includes 4 Banks, and data is stored in these 4 Banks. Then, when accessing data, the memory controller 12 can activate the data in these 4 Banks at the same time. In addition to the storage area, each PIM-Die also integrates multiple computing units (PUs), such as Figure 2C PU1-PU5 shown in FIG. PU is used to perform computing tasks so as to directly perform computing tasks in memory 13 instead of transferring data to an external processor 11 for processing, thereby improving computing efficiency. Each PU can correspond to one or more banks, such as Figure 2C The PU1 shown in the figure can correspond to BankA and BankF, that is, the data read from BankA and BankF can be directly transferred to PU1 for calculation. In addition, each PIM-Die can also integrate multiple mode registers (Mode Register, MR), such as Figure 2CMode registers 1, ..., mode register n shown in . MR can be used to configure the operating mode and function of memory 13, for example, setting parameters such as data width, delay, refresh rate, etc. The existence of MR enables the memory controller 12 to configure and optimize memory 13 according to system requirements, thereby improving performance and compatibility. By programming these registers, the system can adjust the behavior of memory 13 to adapt to different application scenarios and performance requirements. Each PIM-Die can also include a global buffer, which can be used as a cache to store frequently accessed data to reduce access delays and improve system response speed.

[0126] For ease of understanding, the following is an example of the Bank structure. Figure 2D , Figure 2D : This is a schematic diagram of the structure of a Bank provided by an embodiment of the present application. A Bank may include a storage array and a row buffer. The storage array is responsible for actual data storage. It is composed of multiple storage cells, usually organized in the form of rows and columns, that is, the storage array may be composed of M rows and N columns of storage cells, allowing efficient data access. The row buffer is used to store row data of a row read from the storage array.

[0127] For ease of understanding, the structure of the calculation part (calculation unit) in the memory 13 is also exemplarily described below. Figure 2E and Figure 2F , Figure 2E is a schematic diagram of the structure of a computing unit provided in an embodiment of the present application, Figure 2F : is a schematic diagram of the structure of another computing unit provided by an embodiment of the present application, wherein the computing unit may include a multiplication circuit, an addition circuit, a shift circuit and an accumulator, and is suitable for multiplication and accumulation operations. Among them, the multiplication circuit may include one or more multipliers, which can perform multiplication operations; the addition circuit may include one or more adders, which can perform addition operations; the shift circuit may include one or more shifters, which can perform shift processing on input data; and the accumulator can perform accumulation operations on input data. Exemplarily, the shift operation can be performed after the multiplication operation is completed, and is suitable for Figure 2E The structure in; the shift operation can be performed after the addition operation is completed, applicable Figure 2FThe structure in. It is understandable that the calculation part in the memory 13 can also be designed according to different calculation requirements to achieve the purpose of completing other calculations. If the calculation part in the memory 13 needs to complete other calculations, the memory 13 may include other circuits accordingly. The structure of the memory 13 is not specifically limited here. In the embodiment of the present application, the multiplication circuit and the addition circuit can be used to perform multiplication and accumulation calculations on multiple sub-data and input vectors stored in different rows / columns after splitting, respectively. The shift circuit is used to shift one or more intermediate results obtained after calculating the multiple sub-data and input vectors. The accumulator can be used to accumulate the one or more intermediate results after shifting to obtain the final data, thereby ensuring that the calculation unit with low calculation precision can complete the calculation of high-precision data.

[0128] It should be noted that the electronic device 01 may also include a power management unit, which can be used to power the processor 11, the memory controller 12 and the memory 13, and can also power other devices or modules in the electronic device 01. In addition, the electronic device 01 may also include one or more interfaces, which can be used to realize communication between the electronic device 01 and other devices. In some embodiments, the interface may include wires for coupling wired connections, or pins for coupling wireless transceivers, or chips and / or pins for wireless connections. In some embodiments, the interface may include a transmitter, a receiver, a transceiver and / or an antenna.

[0129] It is understandable that the embodiments of the present application can be applied to various computer system architectures. Figure 2A The architecture in the embodiment of the present application is only an exemplary implementation method and should not constitute a limitation on the present application. The architecture applicable to the embodiment of the present application includes but is not limited to the above architecture. It should be understood that the electronic device 01 may have more Figure 2A More or fewer units / modules may be shown, two or more units / modules may be combined, or there may be different unit / module configurations. Figure 2A The various units / modules shown in the figure may be implemented in hardware, software, or a combination of hardware and software including one or more signal processing and / or application-specific integrated circuits, and the embodiments of the present application do not specifically limit this.

[0130] In order to better understand the embodiments of the present application, the technical solutions provided by the embodiments of the present application will be described below in conjunction with more drawings.

[0131] The following takes the processing of the weight parameter from the storage stage to the calculation stage as an example to briefly describe the control method provided by the embodiment of the present application. First, the control method is briefly described from the processing process of the storage stage. In the storage stage, the control method may at least include the following steps: Figure 3Steps S300-S302 shown involve a processor and an internal memory. The processor and the internal memory are coupled, and the internal memory may include at least one storage area Bank and at least one computing unit; each Bank includes M rows and N columns of storage units, and the computing accuracy of each computing unit is K, and M, N, and K are all greater than 0. Steps S300-S302 are as follows:

[0132] S300: The processor obtains data to be processed.

[0133] Among them, the quantization accuracy of the data to be processed is L, the data to be processed can be the weight parameters of the model, the weight parameters can be input by the user when building the model, or can be read by the processor from the external memory, and no specific limitation is made here. Optionally, when building the model, the user will clarify the quantization accuracy of the parameters required by the model, and the user will input the weight parameters according to the quantization accuracy required by the model, so when the processor obtains the weight parameters, it can determine the quantization accuracy of the weight parameters.

[0134] S301: When the quantization accuracy L of the data to be processed is greater than the calculation accuracy K of the calculation unit, the processor splits the data to be processed into multiple sub-data.

[0135] The quantization accuracy of each sub-data in the plurality of sub-data is less than or equal to the calculation accuracy K of the calculation unit.

[0136] S302: The processor sends a first memory access instruction to the plurality of sub-data respectively according to the storage rule.

[0137] Among them, the first memory access instruction can be a write instruction, which is used to instruct the internal memory to write the sub-data corresponding to the instruction into a specified row or column for storage, and the rows or columns specified by different first memory access instructions in multiple first memory access instructions are different. In other words, the processor can instruct the internal memory to store multiple sub-data in different rows / columns through multiple memory access instructions. Accordingly, after receiving the first memory access instruction corresponding to each sub-data, the internal memory can write the corresponding sub-data into the specified row / column for storage according to the row / column address carried by each first memory access instruction, and different sub-data are stored in different rows / columns. Optionally, the above-mentioned different rows / columns can be different rows / columns in the same Bank, or different rows / columns in different Banks, which are not specifically limited here.

[0138] In an embodiment of the present application, in the storage stage, after the processor obtains the data to be processed, it can first determine whether the quantization accuracy of the data to be processed is greater than the calculation accuracy of the calculation unit in the internal memory. If the quantization accuracy of the data to be processed is greater than the calculation accuracy of the calculation unit, the subsequent internal memory will not be able to directly calculate the data to be processed through the calculation unit, so the processor can first split the data to be processed to obtain multiple sub-data with quantization accuracy less than or equal to the calculation accuracy, so that the internal memory can subsequently calculate each sub-data separately through the calculation unit. It should be noted that the calculation unit can generally perform calculations in units of one row / column of data, and store the multiple sub-data obtained after splitting in different rows / columns, which can ensure that the calculation unit can restore accurate calculation results through shift accumulation in the subsequent calculation process.

[0139] For ease of understanding, the storage process is briefly described by taking the case where the data to be processed includes 1 weight parameter as an example. For example, the quantization accuracy of the weight parameter is INT4, and the calculation accuracy of the calculation unit is INT2. The processor can split the weight parameter into 2 2-bit data, one of which is the high 2-bit data of the weight parameter, and the other is the low 2-bit data of the weight parameter, that is, splitting the weight parameter of 1 INT4 quantization accuracy into 2 sub-data of INT2 quantization accuracy. The processor then writes the 2 sub-data of INT2 quantization accuracy into different rows / columns for storage through 2 memory access instructions. In the subsequent calculation process, the processor can instruct the internal memory to input the 2 sub-data of INT2 quantization accuracy into the calculation unit for calculation in sequence through the calculation instruction, and then obtain 2 intermediate results, and then the final calculation result (i.e., the calculation result of the weight parameter with quantization accuracy of INT4) can be restored after shifting and accumulating the 2 intermediate results.

[0140] Optionally, the storage rule may specifically include that the designated rows or designated columns to which the multiple sub-data are written are adjacent rows or columns, that is, after the processor splits the data to be processed, it may instruct the internal memory to write the multiple sub-data obtained after the splitting into multiple adjacent rows / columns for storage. For example, after 1 data to be processed is split into 2 sub-data, the processor may instruct the internal memory to write one of the sub-data into a designated column (such as col.k) for storage through a memory access instruction, and instruct the internal memory to write another sub-data into a designated column (such as col.k+1) for storage through another memory access instruction, and col.k and col.k+1 are adjacent columns.

[0141] Optionally, the storage rule may specifically include that the designated rows or designated columns to which the multiple sub-data are written are located in different partitions. In other words, after the processor splits the data to be processed, it may instruct the internal memory to write the multiple sub-data obtained after the splitting into multiple different rows / columns for storage, and the interval between different rows / columns is greater than 1. In other words, the processor may instruct the internal memory to write the multiple sub-data into multiple non-adjacent rows / columns for storage. In other words, a storage area of ​​the internal memory may be divided into multiple different partitions, and the processor may instruct the internal memory to write the multiple sub-data into multiple different partitions, each of which may include multiple rows / columns. For example, a storage area of ​​the internal memory may include two partitions, partition 1 includes col.0~col31, and partition 2 includes col.32~col63. After one piece of data to be processed is split into two sub-data, the processor can instruct the internal memory to write one of the sub-data into a specified column (such as col.0) for storage through a memory access instruction, and instruct the internal memory to write the other sub-data into a specified column (such as col.32) for storage through another memory access instruction. Col.0 and col.32 belong to different partitions respectively, and the two are not adjacent columns.

[0142] In one possible implementation, when the quantization accuracy of the data to be processed is less than or equal to the calculation accuracy of the calculation unit, the processor can directly write the data to be processed completely into the specified row / column for storage through the memory access instruction without splitting the data. In the subsequent calculation process, the internal memory can calculate the data to be processed completely stored in the specified row / column through the calculation unit to directly obtain the final calculation result. For example, the quantization accuracy of the data to be processed is INT2, and the calculation accuracy of the calculation unit is INT2. The calculation unit can support direct calculation of the data to be processed. Therefore, there is no need to split the data to be processed with the quantization accuracy of INT2 in the storage stage, and the data to be processed can be directly written into the specified row / column for storage.

[0143] In a possible implementation, when the quantization accuracy L of the data to be processed is less than the calculation accuracy K of the calculation unit, and the calculation accuracy K is J times the quantization accuracy L (J is an integer greater than 1), the data to be processed can be directly stored in the specified row / column without data splitting in the storage stage. And, in the subsequent calculation process, the calculation unit can simultaneously calculate the data to be processed stored in at most J rows / columns. For example, the quantization accuracy of the data to be processed is INT2 (i.e., 2-bit data), the calculation accuracy of the calculation unit is INT4 (4-bit data can be calculated), and the calculation accuracy is twice the quantization accuracy. At this time, the calculation unit can support direct calculation of the data to be processed, and the calculation unit calculates INT2 data of 1 row / column using only half of the resources (i.e., only 2 bits of 4 bits are used, and 2 bits are left), so the calculation unit can calculate INT2 data in at most 2 rows / columns in the calculation stage. That is to say, the computing resources of the computing unit can be divided into J sub-resources, and the computing accuracy of each of the J sub-resources is the same as the quantization accuracy L of the data to be processed. Therefore, the computing unit can calculate J data to be processed at the same time, and the quantization accuracy of the data to be processed is L.

[0144] It should be noted that, with respect to quantization accuracy, the processor can determine the quantization accuracy of the data to be processed when it obtains the data to be processed; with respect to the calculation accuracy of the computing unit, in one possible implementation method, the processor can send instructions to the internal memory via the bus, instructing the internal memory to report the calculation accuracy of the computing unit, so that when the subsequent processor obtains the data to be processed, it can determine whether the quantization accuracy of the data to be processed is greater than the calculation accuracy of the computing unit.

[0145] The following briefly describes the control method provided by the embodiment of the present application from the processing process of the calculation stage. In the calculation stage, the control method may at least include the following steps: Figure 4 Steps S400-S403 are shown.

[0146] S400: The internal memory receives a second memory access instruction sent by the processor.

[0147] The second memory access instruction may be a calculation instruction, which is used to instruct the internal memory to calculate a plurality of sub-data. The plurality of sub-data are the plurality of sub-data obtained by the processor performing data splitting on the data to be processed when the quantization accuracy L of the data to be processed is greater than the calculation accuracy K of the calculation unit; the quantization accuracy of the sub-data is less than or equal to the calculation accuracy K of the calculation unit. Optionally, the second memory access instruction may also be used to indicate the quantization accuracy L of the data to be processed.

[0148] S401: The internal memory determines storage addresses of a plurality of sub-data according to the second memory access instruction and storage rules.

[0149] The second memory access instruction will carry the storage address of one of the multiple sub-data. After determining the storage address of this sub-data, the internal memory can determine the storage addresses of other sub-data according to the storage rule (storage by adjacent rows / columns, or storage by different partitions). For example, two sub-data are stored in col.0 and col.1 according to the adjacent column rule, and the column address carried by the second memory access instruction is col.0. After the internal memory determines that one sub-data is stored in col.0 through the second memory access instruction, it can determine that another sub-data is stored in col.1 according to the storage rule.

[0150] S402: the internal memory reads the plurality of sub-data from the storage address in sequence in response to a second memory access instruction.

[0151] S403: The internal memory uses the multiple sub-data to perform calculations to determine a final result.

[0152] Among them, in the storage stage, since the quantization accuracy of the weight parameters is greater than the calculation accuracy of the calculation unit, the weight parameters are split into multiple sub-data and stored in multiple different rows / columns respectively. Therefore, in the calculation stage, when the internal memory needs to calculate the processed data, it is necessary to read multiple sub-data from multiple different rows / columns and calculate each sub-data in turn through the calculation unit. The calculation of each sub-data can obtain an intermediate result corresponding to the sub-data (such as the first result, the second result, the third result or the fourth result).

[0153] Optionally, after calculating the intermediate results corresponding to each sub-data, the internal memory can determine the specific number of bits that need to be shifted for each intermediate result according to the position of each sub-data in the data to be processed, and then perform shift processing on each intermediate result through a shift circuit, and finally calculate the multiple shifted intermediate results to obtain the final result. In simple terms, the data to be processed includes high-order data and low-order data. When the sub-data after splitting is high-order data, the intermediate result corresponding to the sub-data needs to be shifted. When the sub-data after splitting is low-order data, the intermediate result corresponding to the sub-data does not need to be shifted.

[0154] It should be noted that, with respect to calculation accuracy, the calculation unit is one of the capabilities of the internal memory, and the calculation accuracy of the calculation unit is known to the internal memory; with respect to the perception of quantization accuracy, it may include the following implementation methods, the internal memory may perceive the quantization accuracy of the data to be processed according to the predefined calculation instructions issued by the processor, or the processor may configure the mode register in the internal memory to enable the internal memory to perceive the quantization accuracy of the data to be processed.

[0155] Optionally, the processor and the internal memory can perceive the quantization accuracy through predefined calculation instructions. That is to say, the processor can instruct the internal memory to process the quantization accuracy of the data by issuing predefined calculation instructions to the internal memory. Exemplarily, the predefined calculation instructions can be configured with reference to the example forms of the following instructions, such as COMP_INT1, COMP_INT2, COMP_INT3, COMP_INT4, COMP_INT8, COMP_INT16, COMP_AC_INT1, COMP_AC_INT2, COMP_AC_INT3, COMP_AC_INT4, COMP_AC_INT8, COMP_AC_INT16, ACT_INT1, ACT_INT2, ACT_INT4, ACT_INT8, ACT_INT16, etc.

[0156] Among them, the COMP_INT1 instruction can indicate that a calculation operation is performed on the data of the specified column address, and the quantization accuracy of the data stored in this column is integer 1 bit (INT1); the COMP_INT8 instruction can indicate that a calculation operation is performed on the data of the specified column address, and the quantization accuracy of the data stored in this column is INT8; the meaning of other COMP_INTx is basically the same as that of COMP_INT1 and COMP_INT8, indicating that the quantization accuracy of the stored data is different, which will not be repeated here.

[0157] The COMP_AC_INT1 instruction can instruct to perform calculation operations on the data stored in all columns of a row, and the quantization accuracy of the data stored in all columns of the row is INT1; similarly, COMP_AC_INT16 indicates that the quantization accuracy of the data stored in all columns of the row is INT16. In addition, the ACT_INT1 instruction can also instruct to perform calculation operations on the data stored in all columns of a row, and the quantization accuracy of the data stored in all columns of the row is INT1. Further optionally, the difference between the ACT_INT1 instruction and the COMP_AC_INT1 instruction is that before issuing the COMP_AC_INT1 instruction, it is necessary to use the ordinary activation instruction ACT to perform row selection, while ACT_INT1 itself is a row selection command, and there is no need to issue an additional conventional ACT instruction for row selection.

[0158] It should be noted that, in addition to indicating the above-mentioned integer data type, the predefined calculation instructions in the embodiment of the present application can also indicate other data types (such as floating-point type); and, in addition to indicating the above-mentioned quantization precision (including 1, 2, 3, 4, 8, 16), the predefined calculation instructions can also indicate other quantization precisions (such as 32), which are not specifically limited here.

[0159] Optionally, a mode register (MR) may be used between the processor and the internal memory to implement the configuration of quantization accuracy perception. Exemplarily, the mode register for implementing quantization accuracy perception may be configured with reference to the following Table 1.

[0160] Table 1 Quantization Precision Aware Mode Register

[0161]

[0162]

[0163] The MR# field may represent the index of the mode register.

[0164] The MA[6:0] field may represent the address of the mode register. Generally, the processor may find the mode register through the mode register index and address, and then adjust the configuration of the mode register.

[0165] Access: R / W, indicating that the mode register can be read and written. For example, the values ​​of OP[5:4] and OP[3:0] can be read through the read command to determine the number of banks activated simultaneously in the in-memory calculation memory and the quantization accuracy of the data stored in the in-memory calculation memory. For another example, the values ​​of OP[5:4] and OP[3:0] can be changed through the write command to adjust the number of banks activated simultaneously in the in-memory calculation memory and the quantization accuracy of the data stored in the in-memory calculation memory.

[0166] OP[7]: The length can be 1 bit. When the value is 1, it indicates the all columns mode, and the calculation operation of the data in all columns (such as col.0 to col.63) is automatically performed in sequence, similar to the COMP_AC_INTx instruction or ACT_INTx instruction indicating that the calculation operation is performed on the data stored in all columns of a row; when the value is 0, it indicates the normal mode, and the calculation operation of the data in the specified column is performed, similar to the COMP_INTx instruction indicating that the calculation operation is performed on the data stored in the specified column.

[0167] OP[6]: reserved field, the length can be 1 bit.

[0168] OP[5:4]: The field length can be 2 bits, which is used to configure the number of banks activated at the same time. For example, the value can be 1, 4, 8, 16, etc., or other values, such as 2, 6, 12, etc. Exemplarily, the value of this field can directly indicate the number of banks activated at the same time, such as configuring the value of OP[5:4] to binary 00, indicating that the number of banks activated at the same time is 1; configuring the value of OP[5:4] to binary 01, indicating that the number of banks activated at the same time is 4; configuring the value of OP[5:4] to binary 10, indicating that the number of banks activated at the same time is 8; configuring the value of OP[5:4] to binary 11, indicating that the number of banks activated at the same time is 16. It can be understood that the length of this field can be configured according to the maximum number of banks activated at the same time. The above field length and the value of the number of banks activated at the same time are only examples and should not constitute a specific limitation on the configuration of the mode register.

[0169] OP[3:0]: The field length can be 4 bits, which are used to configure the quantization precision, for example, it can be configured as INT1, INT2, INT3, INT4, INT8, INT16, FP16 or FP32. Exemplarily, the 4 bits of OP[3:0] can be used to indicate the type and quantization precision of the data, such as the highest bit of the 4 bits can be used to indicate that the data type is integer or floating point, and the remaining 3 bits can be used to indicate the quantization precision of the data. Optionally, when the highest bit is 0, it indicates that the data type is floating point, and when the highest bit is 1, it indicates that the data type is integer. For example, OP[3:0]=1111 can be considered that the configured quantization precision is INT8. Alternatively, each quantization precision may correspond to a unique OP[3:0] value, and a certain quantization precision may be directly indicated by configuring the OP[3:0] value, without a specific bit to distinguish the data type, for example, OP[3:0]=0000 corresponds to INT1, OP[3:0]=0001 corresponds to INT2, OP[3:0]=0010 corresponds to INT3, ..., OP[3:0]=1111 corresponds to FP32, etc. Understandably, the length of the field may be configured according to the maximum quantization precision to be configured, and the above field length and quantization precision values, as well as the indication method of the quantization precision are only examples and are not specifically limited here.

[0170] Optionally, for the method of configuring quantization precision perception through a mode register, the calculation instructions exchanged between the processor and the internal memory can only indicate the calculation of the data of the specified row / column, and the quantization precision of the data may not be indicated. For example, the COMP instruction can indicate the calculation operation of the data of the specified column address, and the quantization precision of the data stored in the column can be determined according to the OP[3:0] field in the mode register. This method is applicable to scenarios where the number of reserved instruction types is limited and the existing number is insufficient to indicate multiple different quantization precisions. For example, the number of reserved instruction types is only 3, and currently 4 different quantization precisions (such as INT1, INT2, INT4 and INT8) need to be indicated. The reserved number is only enough to indicate three of the quantization precisions, such as COMP_INT1, COMP_INT2, and COMP_INT4, which indicate quantization precisions INT1, INT2, and INT4 respectively, and COMP_INT8, which indicates quantization precision INT8, cannot be indicated due to insufficient reserved number.

[0171] For ease of understanding, based on the above-mentioned quantization precision perception method, the process of the processor instructing the in-memory computing memory to perform data calculation is now exemplified. Taking the case where a row in the in-memory computing memory has 64 columns, a column can store 256-bit data, and the actuarial precision of the computing unit is INT2 as an example, when the quantization precision of the weight parameter is INT2, a column can store 128 weight parameters with a quantization precision of INT2, and a row can store 64*128=8192 weight parameters with a quantization precision of INT2.

[0172] For example, for the method of performing quantization precision perception between the processor and the internal memory through predefined calculation instructions, the processor can issue a COMP_INT2 calculation instruction to instruct the internal memory to complete the calculation of 128 INT2 weight parameters in the specified column address. Figure 5 In the processing flow shown, the processor can first activate the specified Banka and the specified rowi in the Banka through the activation instruction (ACTIVATE); then issue 64 COMP_INT2 calculation instructions in sequence to calculate the weight parameters of 64 columns from col.0 to col.63 in sequence, so as to complete the calculation of 8192 weight parameters with a quantization accuracy of INT2 as shown in the following formula 1. Optionally, after completing the calculation of all weight parameters in the 64 columns of the row (rowi), the processor can issue a precharge instruction (PRECHARGE) to close the opened rowi and Banka.

[0173] Formula 1:

[0174] The input vector V may be transmitted from the host (or processor) to the in-memory computing memory, and the number of elements of the input vector V corresponds to the number of weight parameters.

[0175] Exemplarily, for the method of perceiving quantization accuracy between the processor and the internal memory by configuring a mode register, the OP[3:0] field in the mode register can be configured to be 0001, indicating that the weight parameters with a quantization accuracy of INT2 are stored in the internal calculation memory, and the processor can issue a COMP calculation instruction to complete the calculation of the specified column address, that is, the calculation of 128 INT2 weight parameters. Furthermore, when the processor needs to complete the calculation of the above 8192 weight parameters, the processor can sequentially issue 64 COMP calculation instructions to calculate the weight parameters of 64 columns from col.0 to col.63 in turn, thereby completing the calculation of 8192 weight parameters with a quantization accuracy of INT2 as shown in the above formula 1. It can be understood that the processing process also involves activation instructions and pre-charge instructions, which will not be elaborated here.

[0176] In one possible implementation, when the quantization accuracy of the weight parameters that need to be written to the in-memory computing memory is greater than the calculation accuracy of the computing unit, the multi-bit quantized data can be first split into multiple sub-data with fewer bits and stored in different rows / columns of the in-memory computing memory. During calculation, the multiple sub-data can be read into the computing unit in turn for calculation, and the calculation results of each sub-data can be shifted and accumulated through the shift circuit of the in-memory computing memory, thereby achieving the purpose of completing the calculation of multi-bit quantized data using a computing unit with low calculation accuracy.

[0177] Exemplarily, taking the calculation accuracy of the calculation unit as INT4, the quantization accuracy of the weight parameter as INT8, and splitting the INT8 quantized data of the weight parameter into two INT4 quantized data for calculation as an example, the calculation process is briefly described. First, for any INT8 quantized data X, it can be expressed as the high 4-bit data shifted left by 4 bits, plus its low 4-bit data, that is, X = (H<<4) + L. For example, X = 01110010, then the high 4-bit data of X H = 0111, the low 4-bit data of X L = 0010, the data after H shifted left by 4 bits H <<4 = 01110000, H <<4 (01110000) + L (0010) = X (01110010).

[0178] Based on this, the multiplication of INT8 quantized data X and another data Y can refer to Figure 6In the manner shown, the data precision of Y can be INT4, and the multiplication of X and Y can be understood as splitting the INT8 quantized data X into high 4 bits (indicated by H) and low 4 bits (indicated by L), and then multiplying the high 4 bits and low 4 bits with data Y respectively and adding them together, that is, X*Y=[(H<<4)+L]*Y=(H<<4)*Y+L*Y=[(H*Y)<<4]+L*Y. Reference Figure 7A-7D , the process of multiplying X and Y may include the following calculation steps:

[0179] (1) Fig. 7A As shown, the high 4 bits of X are multiplied by Y to obtain the intermediate result R1;

[0180] (2) Figure 7B As shown, the intermediate result R1 is shifted left by four bits to obtain the result RH;

[0181] (3) Figure 7C As shown, the lower 4 bits of X are multiplied by Y to obtain the intermediate result RL.

[0182] (4) Fig.7D As shown, RH and RL are added to obtain the final result R.

[0183] It should be noted that after calculating the multiple sub-data obtained by splitting, multiple intermediate results can be obtained. When calculating the final result, it is necessary to first shift the multiple intermediate results, and then calculate the multiple intermediate results after shifting to ensure the accuracy of the final result. Among them, the specific number of bits of each intermediate result shifted in the multiple intermediate results can be determined based on the position of the sub-data corresponding to the intermediate result in the original data (i.e., the data before splitting). For example, INT8 quantized data is split into high 4-bit data (i.e., the first sub-data) and low 4-bit data (i.e., the second sub-data). Since there are 4 bits of data after the high 4-bit data before splitting, the intermediate result obtained by calculating this part of the data needs to be shifted left by 4 bits; and there is no data after the low 4-bit data before splitting, so the intermediate result obtained by calculating this part of the data can be obtained without shifting. For another example, the INT3 quantized data is split into high 1-bit data (i.e., the first sub-data), middle 1-bit data, and low 1-bit data (i.e., the second sub-data). Since the high 1-bit data is followed by 2-bit data before splitting, the intermediate result obtained by calculating this part of the data needs to be shifted left by 2 bits; and the middle 1-bit data is followed by 1-bit data before splitting, so the intermediate result obtained by calculating this part of the data needs to be shifted left by 1 bit; and there is no data behind the low 1-bit data before splitting, so the intermediate result obtained by calculating this part of the data does not need to be shifted.

[0184] Optionally, the weight parameter of the quantization precision INT8 can be split into 4+4, 2+2+2+2, 3+3+2, 3+3+1+1, etc. When splitting data, the resource utilization of the INT4 computing unit and the maximum amount of data that can be stored in each column (such as 256 bits) can be comprehensively considered to select a specific splitting method. For example, if it is split into 2+2+2+2, the storage resources of each column can be fully utilized (when storing 128 data, 256 bits can be fully used), but each calculation only uses 2 bits of resources in the computing unit, and the resource utilization of each computing unit is only 50%. If you want to fully utilize the resources of the computing unit, you need to divide the resources of the computing unit first so that each computing unit can calculate 2 2-bit data. The calculation process is more complicated than the 4+4 method. If it is split into 3+3+2, the resource utilization of the computing unit is improved, but the storage resources in some columns cannot be fully utilized (256 cannot divide 3). If it is split into 4+4, not only can the resources of the computing unit be fully utilized, but also the storage resources of each column can be fully utilized (256 is an integer multiple of 4). Therefore, for the combination of INT4 computing precision and INT8 quantization precision, the 4+4 split method can be preferred.

[0185] For ease of understanding, the following first takes the calculation of INT4 quantized data by cooperating with the quantization-aware calculation instruction COMP_INT4 and the calculation unit with the calculation accuracy of INT2 as an example, and combines the processing process of the storage stage and the calculation stage to exemplarily illustrate the control method in the embodiment of the present application. That is, take the use of the INT2 calculation unit to complete the INT4 quantized data as an example.

[0186] For example, each row in the in-memory computing memory can contain 64 columns (col), and each col can store 256-bit data, that is, one col can store 128 2-bit data (data with quantization accuracy of INT2) or 64 4-bit data (data with quantization accuracy of INT4), so a row can store 8192 INT2 weight parameters or 4096 INT4 weight parameters. The calculation result of all 4096 INT4 data contained in a row and the input vector V is shown in Formula 2:

[0187] Formula 2:

[0188] For example, in the storage stage, since the quantization accuracy of the weight parameter is greater than the calculation accuracy of the calculation unit (INT4 is greater than INT2), each weight parameter can be split into two 2-bit data, and then the two 2-bit data are stored in two different cols, and these two cols can be adjacent cols. For example, for these 4096 INT4 weight parameters Wi,j You can refer to Fig. 8A The data storage rules shown are used for storage. Since a col can store 128 2-bit data, we can consider storing 128 W i,j The 4-bit data is split into high 2-bit data and low 2-bit data, and the 128 W are written into the memory through a write instruction (corresponding to the first memory access instruction). i,j The high 2 bits of data are stored in the kth column (represented by col.k, such as col.0), and these 128 W are written to the memory through another write instruction (corresponding to the first memory access instruction). i,j The lower 2 bits of data are stored in the k+1 column (represented by col.k+1, such as col.1). i,j Represents an INT4 weight parameter, whose data length is 4 bits, and its high 2 bits of data can be represented as W i,j H2b , its lower 2 bits can be expressed as W i,j L2b . Optionally, 128 W i,j The lower 2 bits of data can also be stored in the kth column (represented by col.k). These 128 W i,j The high 2 bits of data can be stored in the k+1th column (expressed as col.k+1), that is, the high bit data can be stored in the front column or in the back column. Correspondingly, the low bit data can also be stored in the front column or in the back column, and no specific limitation is made here.

[0189] Accordingly, the other 3968 (4096-128) INT4 weight parameters are calculated according to the above 128 W i,j The storage rules are used to store data, and finally 4096 W i,j The data are split and stored in different columns, with the same weight parameter W i,j The high 2-bit data and the low 2-bit data can be stored in adjacent columns. It should be noted that the above is an explanation based on the example of data access by column in the internal memory. It can be understood that when the internal memory can access data in rows, the weight parameters can also be stored in rows. In other words, the technical solution of splitting the weight parameters and storing them in different rows should also fall within the scope of protection of the embodiments of the present application. For ease of understanding, this application is exemplified by taking the example of data access by column in the internal memory.

[0190] For the above Fig. 8AThe data storage method shown in the figure, the calculation instruction execution process in the calculation stage can refer to the following implementation method: the processor can issue a COMP_INT4 instruction to specify the column address col.k where the high 2 bits of data are stored, and the internal calculation memory stores the 128 W i,j H2b The data (i.e., the first sub-data) is calculated, and the calculation result is sent to the internal shift circuit for left shift by 2 bits, and then transferred to the accumulator; the 128 W stored in col.k+1 are read continuously. i,j L2b The data (i.e., the second sub-data) is calculated, and then the calculation result is passed to the accumulator to be accumulated with the calculation result of the shifted high 2-bit data, so as to achieve the purpose of using the calculation circuit of INT2 calculation accuracy to complete the calculation of the weight parameters of INT4 quantization accuracy.

[0191] Optionally, the above implementation can be based on Figure 2E The calculation unit structure shown in is completed, the data in col.k (i.e., the first sub-data) is calculated by the multiplication circuit to obtain 128 multiplication results, and these 128 multiplication results are sent to the shift circuit for shifting, and the shifted results are sent to the addition circuit for addition calculation, and then the addition result (i.e., the first result) is passed to the accumulator; the data in col.k+1 (i.e., the second sub-data) is calculated by the multiplication circuit to obtain another 128 multiplication results, which are sent to the addition circuit for addition calculation, and then the addition result (i.e., the second result) is passed to the accumulator to be accumulated with the previous result. Optionally, the above implementation method can also be based on Figure 2F The calculation unit structure shown in is completed, the data in col.k (i.e., the first sub-data) is calculated by the multiplication circuit to obtain 128 multiplication results, which are sent to the addition circuit for addition calculation to obtain 1 addition result, and then the addition result is sent to the shift circuit for shifting, and the shifted result (i.e., the third result) is transmitted to the accumulator; the data in col.k+1 (i.e., the second sub-data) is calculated by the multiplication circuit to obtain another 128 multiplication results, which are sent to the addition circuit for addition calculation, and then the addition result (i.e., the fourth result) is transmitted to the accumulator for accumulation with the previous result.

[0192] Based on this, for the above 4096 INT4 weight parameters W i,j The calculation process can be referred to as Figure 8BIn the process shown, the processor can first issue an activation instruction to activate the specified row rowi in Banka, and then sequentially issue 32 COMP_INT4 calculation instructions to calculate the 64 columns of data included in rowi respectively. Each COMP_INT4 instruction calculates the data stored in the specified two columns. For example, the first COMP_INT4 can calculate the data stored in col.0 and col.1, and after shifting the calculation result of col.0 data, it is added to the calculation result of col.1 to complete the calculation of the first group of 128 INT4 weight parameters; the second COMP_INT4 can calculate the data stored in col.2 and col.3, and after shifting the calculation result of col.2 data, it is added to the calculation result of col.3 to complete the calculation of the second group of 128 INT4 weight parameters. , and can be accumulated with the calculation results of the previous set of INT4 weight parameters; the third COMP_INT4 can calculate the data stored in col.4 and col.5, and after shifting the calculation results of col.4 data, accumulate them with the calculation results of col.5, complete the calculation of the third group of 128 INT4 weight parameters, and can be accumulated with the calculation results of the previous multiple groups of INT4 weight parameters; and so on, the 32nd COMP_INT4 can calculate the data stored in col.62 and col.63, and after shifting the calculation results of col.62 data, accumulate them with the calculation results of col.63, complete the calculation of the 32nd group of 128 INT4 weight parameters, and can be accumulated with the calculation results of the previous multiple groups of INT4 weight parameters, so as to complete the calculation of 4096 INT4 weight parameters. Optionally, after completing the calculation of all data in rowi, the processor can issue a precharge instruction to close the rowi.

[0193] It should be noted that for the calculation unit with calculation accuracy of INT2, since the weight parameter with quantization accuracy of INT4 is split into two 2-bit data stored in different columns (col), a COMP_INT4 instruction will take out the data stored in 2 cols for calculation, and a COMP_INT2 instruction will take out the data in 1 col for calculation. Therefore, the latency of the COMP_INT4 instruction will be twice that of the COMP_INT2 instruction, but the number of COMP_INT4 instructions is more economical, which is half the number of COMP_INT2 instructions. It should also be noted that in addition to being able to perceive the quantization accuracy through the above-mentioned predefined COMP_INT4 instructions, the processor can also perceive the quantization accuracy through the configuration mode register. At this time, the execution process of the calculation instruction can refer to the above Figure 8B The execution flow of using the COMP_INT4 instruction only needs to replace the COMP_INT4 instruction with the COMP instruction, which will not be further explained here.

[0194] Optionally, in addition to the above Fig. 8A In addition to the data storage method of splitting the same weight parameter into adjacent columns, these 4096 INT4 weight parameters W i,j Also refer to Figure 8C The storage rules shown are used to store data in a partitioned manner. It is possible to consider storing 4096 W i,j The high 2 bits of data are stored in the area of ​​columns 0 to 31 (i.e., col.0 to col.31, a total of 32 columns) (for example, indicated by 32 write instructions (corresponding to multiple first memory access instructions)). i,j The lower 2 bits of data are stored in the area of ​​columns 32 to 63 (i.e., col.32 to col.63, a total of 32 columns) (for example, indicated by 32 write instructions (corresponding to multiple first memory access instructions)), W i,j Represents an INT4 weight parameter, the data length is 4 bits, and its high 2 bits can be represented as W i,j H2b , its lower 2 bits can be expressed as W i,j L2b .

[0195] For the above Figure 8C For the data storage method shown, the execution process of the calculation instructions in the calculation stage can refer to the following implementation method: the processor can issue the COMP_AC_INT4 or ACT_INT4 instruction, and the internal calculation memory will sequentially take out the data in each column starting from col.0 for calculation. When the data stored in col.0~col.31 are calculated, the accumulated result is first sent to the internal shift circuit to shift left by 2 bits, and then the calculation result after left shift is passed to the accumulator to wait for the next calculation; then the data of each column is taken out from col.32~col.63 in sequence to perform multiplication and accumulation calculation, and this part of the calculation result is also passed to the accumulator, and then accumulated with the above-mentioned data calculation result after left shift by 2 bits, so as to complete the calculation of 4096 INT4 weight parameters. It should be noted that there may be differences in the execution process of using COMP_AC_INT4 and ACT_INT4 calculation instructions, such as Fig.8D As shown, for the method based on the COMP_AC_INT4 instruction, before issuing the COMP_AC_INT4 instruction, it is necessary to first issue the activation instruction ACT to perform row selection, while for the method based on the ACT_INT4 instruction, such as Fig. 8EAs shown, since the ACT_INT4 instruction itself is a row selection command, there is no need to issue an additional ACT instruction for row selection. Optionally, after completing the calculation based on the COMP_AC_INT4 or ACT_INT4 calculation instruction, the processor can close the opened row by issuing a precharge instruction.

[0196] Optionally, the above implementation can be based on Figure 2E The calculation unit structure shown in is completed, the data in any column from col.0 to col.31 (i.e., the first sub-data) is calculated by the multiplication circuit to obtain 128 multiplication results, and these 128 multiplication results are sent to the shift circuit for shifting, and the shifted results are sent to the addition circuit for addition calculation, and then the addition result (i.e., the first result) is passed to the accumulator; the data in any column from col.32 to col.63 (i.e., the second sub-data) is calculated by the multiplication circuit to obtain another 128 multiplication results, which are sent to the addition circuit for addition calculation, and then the addition result (i.e., the second result) is passed to the accumulator to be accumulated with the previous result. Optionally, the above implementation method can also be based on Figure 2F The calculation unit structure shown in is completed, the data in any column of col.0~col.31 (i.e., the first sub-data) is calculated by the multiplication circuit to obtain 128 multiplication results, which are sent to the addition circuit for addition calculation to obtain 1 addition result, and then the addition result is sent to the shift circuit for shifting, and the shifted result (i.e., the third result) is transmitted to the accumulator; the data in any column of col.32~col.63 (i.e., the second sub-data) is calculated by the multiplication circuit to obtain another 128 multiplication results, which are sent to the addition circuit for addition calculation, and then the addition result (i.e., the fourth result) is transmitted to the accumulator for accumulation with the previous result.

[0197] It should be noted that the above Figure 8CIn the data storage method shown, a row of 64 columns is divided into 2 partitions (such as a partition of col.0 to col.31 and a partition of col.32 to col.63), and the high 2-bit data is stored in the partition of col.0 to col.31, and the low 2-bit data is stored in the partition of col.32 to col.63. Optionally, a row of 64 columns can be divided into more partitions, such as 4 partitions, including partition a of col.0 to col.15, partition b of col.16 to col.31, partition c of col.32 to col.47, and partition d of col.48 to col.63, wherein partition a can store 2048 high 2-bit data, partition b can store 2048 low 2-bit data, partition c can store other 2048 high 2-bit data, and partition d can store other 2048 low 2-bit data. It is understandable that the number of partitions of the storage part can also be other values, such as 8, 16, etc., which are not specifically limited here. Understandably, when a row of 64 columns is divided into multiple partitions, and the number of rows / columns contained in each partition is 1, the following can be obtained: Fig. 8A The data storage method shown.

[0198] The following is an example of using the quantization-aware computing instruction COMP_INT3 to cooperate with a computing unit with a computing accuracy of INT2 to complete the calculation of INT3 quantized data, and combines the processing process of the storage stage and the calculation stage to exemplify the control method in the embodiment of the present application. That is, taking the use of the INT2 computing unit to complete the INT3 quantized data as an example.

[0199] For example, each row in the in-memory computing memory can contain 64 columns (col), and each col can store 256-bit data. Since 256 and 64 are not divisible by 3, in order to ensure the utilization of each column storage resource, when splitting the INT3 weight parameters for storage, only 63 columns in a row can be used for storage, so a row can store 5376 (63*256 / 3) INT3 weight parameters. Then, the calculation result of all 5376 INT3 data contained in a row and the input vector V is shown in Formula 3:

[0200] Formula 3:

[0201] For example, in the storage stage, since the quantization accuracy of the weight parameter is greater than the calculation accuracy of the calculation unit (INT3 is greater than INT2), each weight parameter can be split into 1 2-bit data and 1 1-bit data, and then the two split data are stored in two different cols. For example, for these 5376 INT3 weight parameters W i,j You can refer to Fig.9AThe 5376 INT3 weight parameters are stored according to the data storage rules shown, and the 5376 INT3 weight parameters are grouped into groups of 256 weight parameters (a total of 21 groups), where each 256 INT3 weight parameters W i,j , the first 128 W can be written by a write instruction (corresponding to the first memory access instruction) i,j The high 2 bits of the 128 W are stored in the kth column (represented by col.k, such as col.0), and the last 128 W can be written to the memory through a write instruction (corresponding to the first memory access instruction). i,j The high 2 bits of are stored in the k+1 column (represented by col.k+1, such as col.1), and the 256 W can be written to the memory through a write instruction (corresponding to the first memory access instruction). i,j The lower 1 bit is stored in the k+2 column (represented by col.k+2, such as col.2). i,j Indicates an INT3 parameter, with the upper 2 bits represented by W i,j H2b , the lower 1 bit is represented by W i,j L1b Optionally, the first 128 W i,j The high 2 bits of data can also be stored in the k+1th column (or k+2th column), and the last 128 W i,j The high 2 bits of data can also be stored in the k+2th column (or the kth column), 256 W i,j The lower 2 bits of data can also be stored in the kth column (or the k+1th column), which is not specifically limited here.

[0202] Accordingly, the 256 INT3 weight parameters in each of the other 20 groups are calculated according to the 256 W parameters in the first group. i,j The storage rules are used to store data, and finally each group has 256 W i,j The data are split and stored in different columns, and the weight parameters W of the same group i,j The high 2 bits of data and the low 1 bit of data can be stored in three adjacent columns.

[0203] For the above Fig.9A The data storage method shown in the figure, the calculation instruction execution process in the calculation stage can refer to the following implementation method: the processor sends a COMP_INT3 instruction to specify the column address col.k where the upper 2 bits are stored, and the internal calculation memory stores the first 128 upper 2 bits of data W stored in the column i,j H2b Calculate and get the result Res H2b1 ; Continue to read the last 128 high 2 bits of data W stored in col.k+1 i,j H2b Calculate and get the result Res H2b2, and Res H2b1 After the accumulation, the accumulated result is sent to the internal shift circuit for left shift by 1 bit, and then the shifted result is passed to the accumulator; continue to read the 256 low 1-bit data W stored in col.k+2 i,j L1b The calculation is performed, and the result is passed to the accumulator and accumulated with the calculation result of the shifted high 2 bits of data. The calculation of 256 INT3 weight parameters is completed.

[0204] Optionally, the above implementation can be based on Figure 2E The calculation unit structure shown in is completed, the data in col.k (i.e., part of the first sub-data) is calculated by the multiplication circuit to obtain 128 multiplication results and sent to the shift circuit for shifting, the shifted results are sent to the addition circuit for addition calculation, and then the addition result is passed to the accumulator; the data in col.k+1 (i.e., part of the first sub-data) is calculated by the multiplication circuit to obtain another 128 multiplication results and sent to the shift circuit for shifting, the shifted results are sent to the addition circuit for calculation, and then the addition result is passed to the accumulator and added with the previous result (i.e., the first result is obtained); the data in col.k+2 (i.e., the second sub-data) is calculated by the multiplication circuit to obtain 256 multiplication results, which are sent to the addition circuit for addition calculation, and then the addition result (i.e., the second result) is passed to the accumulator and added with the previous result. Optionally, the above implementation method can also be based on Figure 2F The calculation unit structure shown in is completed, the data in col.k (i.e., part of the first sub-data) is calculated by the multiplication circuit to obtain 128 multiplication results, which are sent to the addition circuit for addition calculation to obtain 1 addition result, and then the addition result is sent to the shift circuit for shifting, and the shifted result is transmitted to the accumulator; the data in col.k+1 (i.e., part of the first sub-data) is calculated by the multiplication circuit to obtain another 128 multiplication results, which are sent to the addition circuit for addition calculation to obtain another addition result, and then the addition result is sent to the shift circuit for shifting, and the shifted result is transmitted to the accumulator to be accumulated with the previous result (i.e., the third result); the data in col.k+2 (i.e., the second sub-data) is calculated by the multiplication circuit to obtain 256 multiplication results, which are sent to the addition circuit for addition calculation, and then the addition result (i.e., the fourth result) is transmitted to the accumulator to be accumulated with the previous result.

[0205] Based on this, for the above 5376 INT3 weight parameters W i,jThe calculation process of can refer to the following process: the processor can first issue an activation instruction to activate the specified row rowi in Banka, and then issue 21 COMP_INT3 instructions in sequence to calculate the 63 columns of data included in rowi. Each COMP_INT3 instruction calculates the data stored in the specified 3 columns. Among them, the first COMP_INT3 can calculate the data stored in col.0, col.1 and col.2, and after accumulating and shifting the calculation results of col.0 data and the calculation results of col.1 data, they are accumulated with the calculation results of col.2 data to complete the calculation of the first group of 256 INT3 weight parameters; the second COMP_INT3 can calculate the data stored in col.3, col.4 and col.5, and after accumulating and shifting the calculation results of col.0 data and col.1 data, they are accumulated with the calculation results of col.2 data to complete the calculation of the first group of 256 INT3 weight parameters; After the calculation results of col.1.3 data and col.4 data are accumulated and shifted, they are accumulated with the calculation results of col.5 data to complete the calculation of the second group of 256 INT3 weight parameters, and can be accumulated with the calculation results of the previous group of INT3 weight parameters; and so on, the 21st COMP_INT3 can calculate the data stored in col.60, col.61 and col.62, and after the calculation results of col.60 data and col.61 data are accumulated and shifted, they are accumulated with the calculation results of col.62 data to complete the calculation of the 21st group of 256 INT3 weight parameters, and can be accumulated with the calculation results of the previous multiple groups of INT3 weight parameters, so as to complete the calculation of 5376 INT3 weight parameters. Optionally, after completing the calculation of all data in rowi, the processor can issue a precharge instruction to close the rowi.

[0206] It should be noted that for the calculation unit with calculation accuracy of INT2, since one COMP_INT3 instruction will take out data from 3 cols for calculation, and one COMP_INT2 instruction will take out data from 1 col for calculation, the COMP_INT3 instruction will take twice as long as the COMP_INT2 instruction, but the number of COMP_INT3 instructions is less than the number of COMP_INT2 instructions.

[0207] Optionally, in addition to the above Fig.9A In addition to the data storage method of splitting the same set of weight parameters into adjacent columns, these 5376 INT3 weight parameters W i,j Also refer to Fig. 9B The storage rule shown is to store data in a partitioned manner. It can be considered to store the 5376 INT3 weight parameters W in the area of ​​col.0 to col.41. i,jEach column (col) can store 128 2-bit data. In addition, the 5376 INT3 weight parameters W are stored in the area from col.42 to col.62. i,j Each column (col) can store 256 1-bit data.

[0208] For the above Fig. 9B The data storage method shown in the figure, the calculation instruction execution process in the calculation stage can refer to the following method a and method b:

[0209] Method a: The processor can issue a COMP_INT3 instruction. The COMP_INT3 instruction specifies the high 2-bit column address col.k. After the internal calculation memory sequentially performs the calculation of the first 256 high 2-bit data in the col.k and col.k+1 addresses, the calculation results of the two columns are accumulated and sent to the internal shift circuit for left shifting by 1 bit, and then the shifted result is passed to the accumulator; the first 256 low 1-bit data in the col.(k / 2+42) address are taken out for calculation, and the calculation result is passed to the accumulator after it is obtained, and accumulated with the high 2-bit calculation result, thus completing the calculation of 256 INT3 weight parameters. Correspondingly, the calculation process of the remaining 5120 (5376-256) INT3 weight parameters can refer to the calculation of the aforementioned 256 INT3 weight parameters. In addition, 20 COMP_INT3 instructions are used in sequence to complete the calculation of the remaining 20 groups (60col) of data, and finally the calculation results of 21 groups (63col) are accumulated to complete the calculation of 5376 INT3 weight parameters.

[0210] Method b: The processor can issue a COMP_AC_INT3 or ACT_INT3 instruction, and the internal calculation memory will sequentially take out the data in each column starting from col.0 for calculation. After the data in col.0~col.41 are calculated, the accumulated result is sent to the internal shift circuit and shifted left by 1 bit, and then passed to the accumulator; continue to sequentially execute the multiplication and accumulation calculation of the data in col.42~col.62, and pass this part of the calculation result to the accumulator, and then accumulate it with the above data calculation result after shifting left by 1 bit, so as to calculate 5376 INT3 weight parameters.

[0211] Optionally, both the above method a and method b can be based on Figure 2E or Figure 2F The calculation process of the calculation unit with different structures can refer to the above Fig. 8A , Figure 8C or Fig.9A The relevant descriptions in several implementation methods will not be repeated here.

[0212] The following is an example of using the quantization-aware calculation instruction COMP_INT3 to cooperate with the calculation unit with the calculation accuracy of INT1 to complete the calculation of INT3 quantized data, and combines the processing process of the storage stage and the calculation stage to exemplify the control method in the embodiment of the present application. That is, taking the use of the INT1 calculation unit to complete the INT3 quantized data as an example.

[0213] For example, each row in the in-memory computing memory can contain 64 columns (col), each col can store 256 bits of data, and 63 columns in a row are used for storage, so a row can store 5376 (63*256 / 3) INT3 weight parameters. Then, the calculation result of all 5376 INT3 data contained in a row and the input vector V is shown in the above formula 3.

[0214] Optionally, in the storage stage, since the quantization accuracy of the weight parameter is greater than the calculation accuracy of the calculation unit (INT3 is greater than INT1), each weight parameter can be split into three 1-bit data, and then the three 1-bit data are stored in three different cols. For example, for these 5376 INT3 weight parameters W i,j You can refer to Fig. 10A The 5376 INT3 weight parameters are stored according to the data storage rules shown, and the 5376 INT3 weight parameters are grouped into groups of 256 weight parameters (a total of 21 groups), where each 256 INT3 weight parameters W i,j , 256 W can be written into the memory through a write instruction (corresponding to the first memory access instruction). i,j The high 1 bit of is stored in the kth column (represented by col.k, such as col.0), and the 256 W can be written to the memory through a write instruction (corresponding to the first memory access instruction). i,j The middle 2 bits of the 256 W are stored in the k+1 column (represented by col.k+1, such as col.1), and a write instruction (corresponding to the first memory access instruction) can be used to write the 256 W i,j The lower 1 bit of W is stored in the k+2 column (represented by col.k+2, such as col.2). i,j Indicates an INT3 parameter, the high 1 bit is represented by W i,j H1b , where 1 bit is represented by W i,j M1b , the lower 1 bit is represented by W i,j L1b Optionally, the high 1-bit data may be stored in the k+1th column (or k+2th column), the middle 1-bit data may be stored in the k+2th column (or kth column), and the low 1-bit data may be stored in the kth column (or k+1th column), without specific limitation here.

[0215] Accordingly, the 256 INT3 weight parameters in each of the other 20 groups are calculated according to the 256 W parameters in the first group. i,j The storage rules are used to store data, and finally each group has 256 W i,j The data is split and stored in three different columns, which can be three adjacent columns.

[0216] For the above Fig. 10A The data storage method shown in the figure, the calculation instruction execution process in the calculation stage can refer to the following implementation method: the processor sends a COMP_INT3 instruction to specify the column address col.k where the high 1 bit is stored, and the internal calculation memory calculates the 256 high 1 bit data stored in the column, and passes the calculation result to the internal shift circuit to shift left by 1 bit to obtain Res H1b ; Automatically read the 1-bit data of the 256 stored in col.k+1, and get the calculation result and compare it with Res H1b Accumulate, shift the accumulated result left by 1 bit and pass it to the accumulator; then automatically read the 256 low 1-bit data stored in col.k+2 for calculation, pass the calculation result to the accumulator and accumulate it with the previous result, so as to complete the calculation of 256 INT3 weight parameters. Optionally, after the processor issues a COMP_INT3 instruction, the in-memory calculation memory can also directly shift the calculation result left by 2 bits through the shift circuit after obtaining the calculation result of the 256 high 1-bit data and pass it to the accumulator; the subsequent in-memory calculation memory can first shift the calculation result left by 1 bit through the shift circuit after obtaining the calculation result of the 256 middle 1-bit data and then pass it to the accumulator for accumulation with the shift result of the high 1-bit data; finally, the subsequent in-memory calculation memory calculates the 256 low 1-bit data, and passes the calculation result to the accumulator and accumulates it with the previous result, so as to complete the calculation of 256 INT3 weight parameters. Optionally, the above implementation method can be based on Figure 2E or Figure 2F The calculation process of the calculation unit with different structures can refer to the above Fig. 8A , Figure 8C or Fig.9A The relevant descriptions in several implementation methods will not be repeated here.

[0217] Based on this, for the above 5376 INT3 weight parameters W i,j The calculation process can refer to the following process: the processor can first issue an activation instruction to activate the specified row rowi in Banka, and then sequentially issue 21 COMP_INT3 instructions to calculate the 63 columns of data included in rowi. Each COMP_INT3 instruction calculates the data stored in the specified 3 columns.

[0218] Optionally, in addition to the above Fig. 10A In addition to the data arrangement method of splitting the same set of weight parameters and storing them in adjacent columns, these 5376 INT3 weight parameters W i,j Also refer to Fig. 10B The storage rule shown is to store data in a partitioned manner. It can be considered to store the 5376 INT3 weight parameters W in the area of ​​col.0 to col.20. i,j Each column (col) can store 256 1-bit data. The 5376 INT3 weight parameters W are stored in the area of ​​col.21 to col.41. i,j Each column (col) can store 256 1-bit data. In addition, the 5376 INT3 weight parameters W are stored in the area from col.42 to col.62. i,j Each column (col) can store 256 1-bit data.

[0219] For the above Fig. 10B The data storage method shown in the figure, the calculation instruction execution process in the calculation stage can refer to the following method c and method d:

[0220] Method c: The processor can issue a COMP_INT3 instruction. The COMP_INT3 instruction specifies the column address col.k with the higher 1 bit. After the internal calculation memory executes the calculation of the 256 higher 1-bit data in col.k, the calculation result is shifted through the shift circuit and the shifted result is transferred to the accumulator; the 256 middle 1-bit data in the col.k+21 address are automatically taken out for calculation, the calculation result is shifted through the shift circuit, and the shifted result is transferred to the accumulator and accumulated with the shifted result of the higher 1-bit data; the 256 lower 1-bit data in the col.k+42 address are automatically taken out for calculation, and the calculation result is transferred to the accumulator and accumulated with the calculation result of the first 2 bits of data. At this point, the calculation of 256 INT3 weight parameters is completed. Correspondingly, the calculation process of the remaining 5120 (5376-256) INT3 weight parameters can refer to the calculation of the aforementioned 256 INT3 weight parameters. The calculation of the remaining 20 groups (60col) of data is completed sequentially through 20 COMP_INT3 instructions, and finally the calculation results of 21 groups (63col) are accumulated to complete the calculation of 5376 INT3 weight parameters.

[0221] Mode d: The processor can issue a COMP_AC_INT3 or ACT_INT3 instruction, and the internal calculation memory sequentially takes out the 256 high 1-bit data in each column starting from col.0 for calculation. After the data of col.0 to col.20 are calculated, the accumulated result is sent to the internal shift circuit and shifted left by 1 bit, and then passed to the accumulator; the calculation of the 256 middle 1-bit data stored in each column of col.21 to col.41 is continued to be executed sequentially, and the calculation result of the middle 1-bit is accumulated with the shift result of the high 1-bit, and then shifted left by 1 bit, and then passed to the accumulator; the multiplication and addition calculation of the 256 low 1-bit data stored in each column of col.42 to col.62 is continued to be executed sequentially, and this part of the calculation result is also passed to the accumulator, and then accumulated with the previous shift result, thereby calculating 5376 INT3 weight parameters.

[0222] Optionally, both the above method c and method d can be based on Figure 2E or Figure 2F The calculation process of the calculation unit with different structures can refer to the above Fig. 8A , Figure 8C or Fig.9A The relevant descriptions in several implementation methods will not be repeated here.

[0223] It should be noted that in addition to the above-mentioned combinations of INT2 calculation unit and INT4 weight parameter, INT2 calculation unit and INT3 weight parameter, and INT1 calculation unit and INT3 weight parameter, there can be more combinations. The embodiment of the present application does not specifically limit the calculation accuracy value of the calculation unit and the quantization accuracy value of the weight parameter. For example, the combination of INT2 calculation unit and INT5 weight parameter, for this combination, the storage stage can split the 5-bit weight parameter into 2+2+1, 2+1+2 or 1+2+2, and write 3 rows / columns for storage. The calculation process of the calculation stage can refer to the calculation of the combination of INT1 calculation unit and INT3 weight parameter, which will not be expanded here.

[0224] The present application also provides a processor, the processor is coupled to an internal memory, the internal memory includes at least one storage area Bank and at least one computing unit; each of the Banks includes M rows*N columns of storage units, the computing precision of each computing unit is K, and M, N, and K are all greater than 0; the processor is used to:

[0225] Acquire data to be processed; the quantization accuracy of the data to be processed is L;

[0226] When the quantization accuracy L of the data to be processed is greater than the calculation accuracy K of the calculation unit, the data to be processed is split into multiple sub-data; the quantization accuracy of each sub-data in the multiple sub-data is less than or equal to the calculation accuracy K of the calculation unit;

[0227] The first memory access instructions are respectively sent for the multiple sub-data according to the storage rules; the first memory access instructions are used to instruct the internal memory to write the sub-data into a specified row or column, and the specified rows or columns of different first memory access instructions among the multiple first memory access instructions are different.

[0228] The present application also provides an internal memory, the internal memory is coupled to a processor, the internal memory includes at least one storage area Bank and at least one computing unit; each of the Banks includes M rows*N columns of storage units, the computing precision of each computing unit is K, and M, N, and K are all greater than 0; the method includes:

[0229] receiving a second memory access instruction sent by the processor; the second memory access instruction is used to instruct the internal memory to calculate a plurality of sub-data; the plurality of sub-data are a plurality of sub-data obtained by the processor performing data splitting on the data to be processed when the quantization accuracy L of the data to be processed is greater than the calculation accuracy K of the calculation unit; the quantization accuracy of the sub-data is less than or equal to the calculation accuracy K of the calculation unit;

[0230] Determine storage addresses of the plurality of sub-data according to the second memory access instruction and storage rule;

[0231] Reading the plurality of sub-data sequentially from the storage address in response to the second memory access instruction;

[0232] The plurality of sub-data are used to perform calculations to determine a final result.

[0233] In addition, the present application also provides an electronic device, characterized in that it includes one or more processors and one or more internal memories; the one or more processors are coupled to the one or more internal memories; the one or more processors are used to implement the functions involved in any of the above-mentioned embodiments, for example, splitting data and issuing memory access instructions; the one or more internal memories are used to implement the functions involved in any of the above-mentioned embodiments, for example, receiving memory access instructions and performing data storage, data calculation, etc. according to the memory access instructions.

[0234] The present application also provides a computer program, which is used to implement the operations and / or processing performed by any one of the processor and the internal memory in the method provided by the present application.

[0235] The present application also provides a readable storage medium, which stores computer code. When the computer code runs on a computer, the computer executes the operations and / or processes performed by any one of the processor and the internal memory in the method provided by the present application.

[0236] The present application also provides a computer program product, which includes a computer code or a computer program. When the computer code or the computer program runs on a computer, the operations and / or processing performed by any one of the processor and the internal memory in the method provided by the present application are executed.

[0237] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0238] It should be noted that, for the aforementioned method embodiments, for the sake of simplicity of description, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the present application is not limited by the described order of actions, because according to the present application, some steps may be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present application.

[0239] In several embodiments provided in the present application, the mutual coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection through some interfaces, devices or units, or may be an electrical, mechanical or other form of connection.

[0240] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A control method, characterized in that: Applied to a processor, the processor is coupled to an internal memory, the internal memory includes at least one storage area Bank and at least one computing unit; each Bank includes M rows and N columns of storage units, the computing precision of each computing unit is K, and M, N, K are all integers greater than 0; the method includes: Acquire data to be processed; the quantization accuracy of the data to be processed is L; When the quantization accuracy L of the data to be processed is greater than the calculation accuracy K of the calculation unit, the data to be processed is split into multiple sub-data; the quantization accuracy of the sub-data is less than or equal to the calculation accuracy K of the calculation unit; A first memory access instruction is sent for each of the plurality of sub-data according to a storage rule; the first memory access instruction is used to instruct the internal memory to write the sub-data into a specified row or a specified column.

2. The method according to claim 1, characterized in that The storage rule specifically includes that the designated rows or designated columns into which the multiple sub-data are written are adjacent rows or columns; or, the Bank includes multiple partitions, and the storage rule specifically includes that the designated rows or designated columns into which the multiple sub-data are written are located in different partitions.

3. The method according to any one of claims 1 to 2, characterized in that: The maximum amount of data that can be stored in the designated row or the designated column is an integer multiple of the quantization accuracy of the sub-data.

4. The method according to any one of claims 1 to 3, characterized in that The method further comprises: Send a second memory access instruction; the second memory access instruction is used to instruct the internal memory to calculate the multiple sub-data, and is used to indicate the quantization accuracy L of the data to be processed.

5. The method according to any one of claims 1 to 3, characterized in that The internal memory further includes a mode register, and the mode register includes an indication field for indicating data quantization accuracy; the method further includes: Send a control instruction; the control instruction is used to indicate the quantization accuracy L of the data to be processed, and instruct the internal memory to save the quantization accuracy L of the data to be processed in the indication field of the mode register.

6. A control method, characterized in that: Applied to an internal memory, the internal memory is coupled to a processor, the internal memory includes at least one storage area Bank and at least one computing unit; each of the Banks includes M rows and N columns of storage units, the computing precision of each of the computing units is K, and M, N, and K are all greater than 0; the method includes: receiving a second memory access instruction sent by the processor; the second memory access instruction is used to instruct the internal memory to calculate a plurality of sub-data; the plurality of sub-data are a plurality of sub-data obtained by the processor performing data splitting on the data to be processed when the quantization accuracy L of the data to be processed is greater than the calculation accuracy K of the calculation unit; the quantization accuracy of the sub-data is less than or equal to the calculation accuracy K of the calculation unit; Determine storage addresses of the plurality of sub-data according to the second memory access instruction and storage rule; Reading the plurality of sub-data sequentially from the storage address in response to the second memory access instruction; The plurality of sub-data are used to perform calculations to determine a final result.

7. The method according to claim 6, characterized in that The storage rule specifically includes that the designated rows or designated columns into which the multiple sub-data are written are adjacent rows or columns; or, the Bank includes multiple partitions, and the storage rule specifically includes that the designated rows or designated columns into which the multiple sub-data are written are located in different partitions.

8. The method according to claim 6 or 7, characterized in that The plurality of sub-data include first sub-data and second sub-data, the first sub-data being one or more high-order data in the data to be processed, and the second sub-data being one or more low-order data in the data to be processed; The using the plurality of sub-data to perform calculations and determine a final result comprises: Performing multiplication calculation on the first sub-data and then performing shift addition to obtain a first result; performing multiplication and addition calculation on the second sub-data to obtain a second result; The final result is determined based on the first result and the second result.

9. The method according to claim 6 or 7, characterized in that The plurality of sub-data include first sub-data and second sub-data, the first sub-data being one or more high-order data in the data to be processed, and the second sub-data being one or more low-order data in the data to be processed; The using the plurality of sub-data to perform calculations and determine a final result comprises: Performing multiplication and addition calculations on the first sub-data and then performing shift processing to obtain a third result; performing multiplication and addition calculation on the second sub-data to obtain a fourth result; The final result is determined based on the third result and the fourth result.

10. The method according to any one of claims 6 to 9, characterized in that The maximum amount of data that can be stored in a designated row or a designated column where the sub-data is written is an integer multiple of the quantization accuracy of the sub-data.

11. The method according to any one of claims 6 to 10, characterized in that: The second memory access instruction is also used to indicate the quantization accuracy L of the data to be processed.

12. The method according to any one of claims 6 to 11, characterized in that The internal memory further includes a mode register, and the mode register includes an indication field for indicating data quantization accuracy; the method further includes: Receive a control instruction sent by the processor; the control instruction is used to indicate the quantization precision L of the data to be processed, and instruct the internal memory to save the quantization precision L of the data to be processed in the indication field of the mode register.

13. The method according to any one of claims 6 to 12, characterized in that When the calculation accuracy K of the calculation unit is J times the quantization accuracy L of the data to be processed, J is an integer greater than 1; the method further includes: Dividing the computing resources of the computing unit into J sub-resources; Receive J data to be processed; The J data to be processed are processed in parallel by using the J sub-resources; the quantization accuracy of the data to be processed is L.

14. A processor, characterized in that: The method comprises a module or a unit for executing the method according to any one of claims 1 to 5.

15. An internal memory, characterized in that: The method comprises a module or a unit for executing the method according to any one of claims 6 to 13.

16. An electronic device, characterized in that: It comprises one or more processors and one or more internal memories; the one or more processors are coupled to the one or more internal memories; the one or more processors are used to execute the method as described in any one of claims 1-5, and the one or more internal memories are used to execute the method as described in any one of claims 6-13.

17. A computer-readable storage medium, characterized in that: The storage medium stores a computer program or instruction. When the computer program or instruction is executed by a computer, the method according to any one of claims 1 to 5 or 6 to 13 is implemented.

18. A computer program, characterized in that The computer program comprises instructions, and when the computer program is executed by a computer, the method according to any one of claims 1 to 5 or 6 to 13 is implemented.

Images