Method, processing module, storage medium and program product performed by an electronic device

Abstract

Embodiments of the present disclosure provide a method, processing module, storage medium and program product executed by an electronic device, relating to the field of artificial intelligence. The method comprises: determining, according to size information of input data, a plurality of first operation units and configuration information of each first operation unit in an operation unit contained in a first processing module; configuring each first operation unit according to the corresponding configuration information; and processing the input data based on an operation array composed of each first operation unit, wherein the size of the operation array corresponds to the size of the input data, and the configuration information comprises position-related information of the first operation unit in the operation array. Optionally, the above-mentioned method executed by the electronic device can be executed using an artificial intelligence model.

Description

Technical Field

[0001] This disclosure relates to the field of artificial intelligence technology, and more specifically, to a method, processing module, storage medium, and program product executed by an electronic device. Background Technology

[0002] The matrix operation unit includes a cube array structure of MAC (Multiply Accumulate) array units and an accumulator unit. AI models involve matrix operations of different sizes during computation. Existing NPU (Neural Processing Unit) matrix operation units preprocess the input matrices of different sizes, such as matrix partitioning, to adapt to the hardware structure. However, after performing multiplication, an accumulator is still needed to perform accumulation operations to obtain the final calculation result. This lengthy calculation process severely reduces the computational efficiency of the matrix operation unit, resulting in the inability to meet the real-time requirements of the output data. Summary of the Invention

[0003] The purpose of this disclosure is to at least solve one of the aforementioned technical defects. The technical solution provided by the embodiments of this disclosure is as follows: In a first aspect, embodiments of this disclosure provide a method performed by an electronic device, comprising: Based on the size information of the input data, multiple first processing units and configuration information of each first processing unit are determined in the processing units included in the first processing module. Configure each first processing unit according to the corresponding configuration information; The input data is processed based on the arithmetic array composed of each first arithmetic unit; The size of the computing array corresponds to the size of the input data, and the configuration information includes information related to the position of the first computing unit in the computing array.

[0004] In a feasible embodiment, each first processing unit is configured according to the corresponding configuration information, including: The configuration information of each first arithmetic unit is configured into the register of that first arithmetic unit.

[0005] In one feasible embodiment, the position-related information includes: first indication information indicating at least the first operation unit is located at the beginning and / or end of the operation row in the operation array.

[0006] In one feasible embodiment, the input data is processed accordingly, including: When the first instruction indicates that the first arithmetic unit is at the beginning of the arithmetic line, the first arithmetic unit outputs the result to the next connected first arithmetic unit; and / or When the first indication information indicates that the first arithmetic unit is at the end of the arithmetic line, the first arithmetic unit outputs the arithmetic result to the second processing module; and / or When the first instruction indicates that the first arithmetic unit is not located at the beginning or end of the arithmetic line, the first arithmetic unit outputs the arithmetic result to the next connected first arithmetic unit.

[0007] In one feasible embodiment, the arithmetic units in the first processing module are sequentially cascaded, and the output port of each arithmetic unit is connected to the second processing module.

[0008] In one feasible embodiment, determining the first indication information includes: The M×N input data and the L×H operation unit in the first processing module are respectively converted into one-dimensional row vectors, wherein M, N, L and H are not less than 1; The input data 1×N row vector is mapped to the row vector of the operation unit 1×(L×H) contained in the first processing module; The first indication information is determined based on the position of the corresponding arithmetic unit whose mapping length is a multiple of N.

[0009] In one feasible embodiment, determining the first indication information includes: Based on the comparison between the size of the input data and the size of the computing unit contained in the first processing module, the first indication information is determined.

[0010] In one feasible embodiment, determining the first indication information includes: When the number of rows and columns of the M×N input data is smaller than the number of rows and columns of the L×H operation units in the first processing module, the first indication information is determined based on the position of the (N-1)th operation unit corresponding to each row. When the logical row of the input data is larger than the physical row of the operation array, and the total size of the input data is smaller than the total size of the operation units contained in the first processing module, the first indication information is determined based on the position of the N×i-th ​​operation unit in each row, where i is an integer that increments sequentially from 1 to M. If the total size of the input data is greater than the total size of the computing units contained in the first processing module, a loop calculation is performed. At the end of the current calculation, the row vector of M-(L×H) / N is rearranged into the computing array, and the current calculation result is concatenated with the previous calculation result. The first indication information is determined based on the output position of each calculation result.

[0011] In one feasible embodiment, the cyclic calculation includes: In the first round of calculation, the number of rows of weighted data processed in the input data is (H×L) / N; In the next round of calculation, if (M - ((H×L) / N)>(H×L)), then perform the same operation as in the first round of calculation; otherwise, end the loop calculation.

[0012] In one feasible embodiment, the input data includes data to be processed and processing weight data; The configuration information also includes at least one of the following: A second indication information indicating whether the processing weight data corresponding to the first processing unit is zero; The processing weight data corresponding to the first processing unit; The location information of the first arithmetic unit in the first processing module.

[0013] In one feasible embodiment, the location information includes at least one of the following: The first position information is used to indicate the position of the first operation unit after the operation of each operation row ends; The second position information is used to indicate the position of the first operation unit at the start of each operation row.

[0014] In one feasible embodiment, processing the input data includes: When the second indication information indicates that the processing weight data corresponding to the first processing unit is zero, the first processing unit directly outputs the calculation result of the previous first processing unit to the next first processing unit or the second processing module.

[0015] In one feasible embodiment, it further includes at least one of the following: The data to be processed is mapped sequentially to a first sequence in column order, and the first operation unit in the operation matrix is ​​mapped sequentially to a second sequence. The data to be processed is then input to the first operation unit through the first sequence and the second sequence. Starting from the first position of each operation row in the operation array, the data to be processed is sequentially input into each first operation unit, column by column.

[0016] In one feasible embodiment, it further includes: The CPU receives a first threshold value, which is configured by the CPU based on the current calculation result of the input data. Determine the configuration information of each first arithmetic unit, including: Set the values ​​of elements in the processed weight data that are less than the first threshold to zero.

[0017] In one feasible embodiment, it further includes: Use a gated clock to shut down the arithmetic units in the first processing module that do not process the input data.

[0018] In one feasible embodiment, the arithmetic unit is a multiply-accumulate (MAC) unit.

[0019] In one feasible embodiment, the input data includes at least one of the following: communication data, image data, audio data, and text data; The communication data refers to the communication data received by the electronic device and to be pre-distorted, wherein the electronic device is a network node or a user equipment.

[0020] Secondly, embodiments of this disclosure provide a processing module, including a control logic unit and multiple arithmetic units; The control logic unit is configured to determine multiple first arithmetic units and configuration information of each first arithmetic unit in the arithmetic unit included in the first processing module according to the size information of the input data, and configure each first arithmetic unit according to the corresponding configuration information. The size of the arithmetic array composed of each first arithmetic unit corresponds to the size of the input data, and the configuration information includes position-related information of the first arithmetic unit in the arithmetic array. The processing unit is configured to process the input data.

[0021] In one feasible embodiment, the arithmetic unit includes a register; The control logic unit is configured to configure the configuration information of each first arithmetic unit into the register of that first arithmetic unit; The arithmetic unit is configured to process the input data according to the configuration information in the register.

[0022] The location-related information includes: first indication information indicating at least the first operation unit is located at the beginning and / or end of the operation row in the operation array.

[0023] In one feasible embodiment, the computing unit includes a first data path selector, which is configured to: When the first instruction indicates that the first arithmetic unit is at the beginning of the arithmetic line, the result of the arithmetic operation is output to the next connected first arithmetic unit; and / or When the first indication information indicates that the first arithmetic unit is at the end of the arithmetic line, the arithmetic result is output to the second processing module; and / or When the first instruction indicates that the first arithmetic unit is not located at the beginning or end of the arithmetic line, the arithmetic result is output to the next connected first arithmetic unit.

[0024] In one feasible embodiment, the arithmetic units in the first processing module are sequentially cascaded, and the output port of each arithmetic unit is connected to the second processing module.

[0025] In one feasible embodiment, the control logic unit is configured as follows: The M×N input data and the L×H operation unit in the first processing module are respectively converted into one-dimensional row vectors, wherein M, N, L and H are not less than 1; The input data 1×N row vector is mapped to the row vector of the operation unit 1×(L×H) contained in the first processing module; The first indication information is determined based on the position of the corresponding arithmetic unit whose mapping length is a multiple of N.

[0026] In one feasible embodiment, the control logic unit is configured as follows: Based on the comparison between the size of the input data and the size of the computing unit contained in the first processing module, the first indication information is determined.

[0027] In one feasible embodiment, the control logic unit is configured as follows: When the number of rows and columns of the M×N input data is smaller than the number of rows and columns of the L×H operation units in the first processing module, the first indication information is determined based on the position of the (N-1)th operation unit corresponding to each row. When the logical row of the input data is larger than the physical row of the operation array, and the total size of the input data is smaller than the total size of the operation units contained in the first processing module, the first indication information is determined based on the position of the N×i-th ​​operation unit in each row, where i is an integer that increments sequentially from 1 to M. If the total size of the input data is greater than the total size of the computing units contained in the first processing module, a loop calculation is performed. At the end of the current calculation, the row vector of M-(L×H) / N is rearranged into the computing array, and the current calculation result is concatenated with the previous calculation result. The first indication information is determined based on the output position of each calculation result.

[0028] In one feasible embodiment, the cyclic calculation includes: In the first round of calculation, the number of rows of weighted data processed in the input data is (H×L) / N; In the next round of calculation, if (M - ((H×L) / N)>(H×L)), then perform the same operation as in the first round of calculation; otherwise, end the loop calculation.

[0029] In one feasible embodiment, the input data includes data to be processed and processing weight data; The configuration information also includes at least one of the following, each configured in different registers: A second indication information indicating whether the processing weight data corresponding to the first processing unit is zero; The processing weight data corresponding to the first processing unit; The location information of the first arithmetic unit in the first processing module.

[0030] In one feasible embodiment, the location information includes at least one of the following: The first position information is used to indicate the position of the first operation unit after the operation of each operation row ends; The second position information is used to indicate the position of the first operation unit at the start of each operation row.

[0031] In one feasible embodiment, the computing unit includes a second data path selector, which is configured to: When the second indication information indicates that the processing weight data corresponding to the first processing unit is zero, the first processing unit directly outputs the calculation result of the previous first processing unit to the next first processing unit or the second processing module.

[0032] In one feasible embodiment, the arithmetic unit is further configured as follows: The data to be processed is mapped sequentially to a first sequence in column order, and the first operation unit in the operation matrix is ​​mapped sequentially to a second sequence. The data to be processed is then input to the first operation unit through the first sequence and the second sequence. And / or, starting from the first position of each operation row in the operation array, the data to be processed is sequentially input into each first operation unit, column by column.

[0033] In one feasible embodiment, the control logic unit is further configured to: The CPU receives a first threshold value, which is configured by the CPU based on the current calculation result of the input data. Set the values ​​of elements in the processed weight data that are less than the first threshold to zero.

[0034] In one feasible embodiment, the control logic unit is further configured to: Use a gated clock to shut down the arithmetic units in the first processing module that do not process the input data.

[0035] In one feasible embodiment, the arithmetic unit is a multiply-accumulate (MAC) unit.

[0036] In one feasible embodiment, the input data includes at least one of the following: communication data, image data, audio data, and text data; The communication data refers to the communication data received by the electronic device and to be pre-distorted, wherein the electronic device is a network node or a user equipment.

[0037] Thirdly, embodiments of this disclosure provide a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the methods described in the first aspect and any of the embodiments thereof.

[0038] Fourthly, embodiments of this disclosure provide a computer program product, including a computer program that, when executed by a processor, implements the methods described in the first aspect and any of its embodiments.

[0039] The beneficial effects of the technical solutions provided in this disclosure are: This disclosure provides a method executed by an electronic device. Specifically, multiple first arithmetic units and configuration information of each first arithmetic unit can be determined in the arithmetic units included in a first processing module based on the size information of the input data. Each first arithmetic unit is then configured according to the corresponding configuration information, enabling dynamic configuration of the arithmetic units in the first processing module. The input data can be processed based on an arithmetic array composed of the first arithmetic units. The size of the arithmetic array corresponds to the size of the input data, and the configuration information includes information related to the position of the first arithmetic units within the arithmetic array. This disclosure allows for adaptive adjustment of the arithmetic array structure based on the size information of the input data, enabling processing of input data of arbitrary size. It avoids matrix partitioning and merging operations, eliminates the need for accumulator units, improves computational efficiency, ensures real-time operation, and saves hardware resources.

[0040] This disclosure provides a processing module including a control logic unit and multiple arithmetic units. The control logic unit is configured to determine multiple first arithmetic units and configuration information for each first arithmetic unit within the arithmetic units of the first processing module based on the size information of the input data. The control logic unit then configures each first arithmetic unit according to the corresponding configuration information. The size of the arithmetic array formed by the first arithmetic units corresponds to the size of the input data, and the configuration information includes the position information of the first arithmetic units within the arithmetic array. The arithmetic units are configured to process the input data. The processing module provided by this disclosure can adaptively adjust the structure of the arithmetic array, avoiding matrix partitioning and merging operations, which is beneficial for improving computational efficiency, ensuring real-time operation, and enabling processing of input data of arbitrary size. Since this disclosure does not require preprocessing of the input data and no accumulator needs to be configured after the MAC unit, it helps save hardware resources. Attached Figure Description

[0041] To more clearly illustrate the technical solutions in the embodiments of this disclosure, the accompanying drawings used in the description of the embodiments of this disclosure will be briefly introduced below.

[0042] Figure 1 This is a structural diagram of a matrix operation unit in related technologies; Figure 2 This is a schematic diagram illustrating the computational principle of a matrix operation unit in related technologies; Figure 3 This is a comparative schematic diagram of the MAC array provided in the embodiments of this disclosure and the MAC array in related technologies; Figure 4a A schematic diagram of a dynamically configurable processing module provided in an embodiment of this disclosure; Figure 4b A schematic diagram of a matrix operation unit provided in an embodiment of this disclosure; Figure 4c Corresponding to Figure 4b A schematic diagram of the control logic unit shown; Figure 5a This is a schematic diagram of the MAC unit in related technologies; Figure 5b A schematic diagram of a MAC unit provided in an embodiment of this disclosure; Figure 5c A schematic diagram of the structure of a dynamically configurable processing module provided in an embodiment of this disclosure; Figure 6 A schematic diagram illustrating the configuration of a control logic unit provided in an embodiment of this disclosure; Figure 7 A schematic diagram of a size adaptation comparison logic unit provided in an embodiment of this disclosure; Figure 8 This is a comparative diagram of the embodiments of this disclosure and related technologies regarding matrix partitioning; Figure 9 A schematic diagram of a configuration information generation logic unit provided in an embodiment of this disclosure; Figure 10 This is a schematic diagram of the structure of a MAC array provided in an embodiment of the present disclosure; Figure 11 This is a comparative schematic diagram of the MAC unit between the embodiments of this disclosure and related technologies; Figure 12 A schematic diagram of the structure of a MAC unit without W bits provided in an embodiment of this disclosure; Figure 13 A schematic diagram of the structure of a MAC unit without a ZERO bit provided in an embodiment of this disclosure; Figure 14 This is a schematic diagram of the structure of a MAC unit without an END bit provided in an embodiment of the present disclosure; Figure 15 A schematic diagram illustrating zero-value control provided in an embodiment of this disclosure; Figure 16 A flowchart illustrating zero-value adaptive adjustment provided in this embodiment of the disclosure; Figure 17a A schematic diagram illustrating a change in a data propagation path, provided as an embodiment of this disclosure; Figure 17b This is a schematic diagram of a data path for W data provided in an embodiment of the present disclosure; Figure 18 This is a schematic diagram of a data path for X data provided in an embodiment of the present disclosure; Figure 19 This is a schematic diagram of another data path for X data provided in an embodiment of this disclosure; Figure 20 This is a schematic diagram illustrating the computational matrix effect of related technologies; Figure 21 This is a schematic diagram illustrating the effect of the computation matrix in an embodiment of this disclosure; Figure 22 A flowchart illustrating a method executed by an electronic device, as provided in this disclosure embodiment; Figure 23 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this disclosure. Detailed Implementation

[0043] The following description, with reference to the accompanying drawings, is provided to aid in a thorough understanding of the various embodiments of this disclosure as defined by the claims and their equivalents. This description includes various specific details to aid understanding but should be considered exemplary only. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the various embodiments described herein without departing from the scope and spirit of this disclosure. Furthermore, for clarity and brevity, descriptions of well-known functions and structures may be omitted.

[0044] The terms and phrases used in the following description and claims are not limited to their dictionary meanings, but are merely configured by the inventors to enable a clear and consistent understanding of this disclosure. Therefore, it will be apparent to those skilled in the art that the following description of various embodiments of this disclosure is configured for illustrative purposes only and not to limit the purpose of this disclosure as defined in the appended claims and their equivalents.

[0045] It should be understood that the singular forms of “a,” “an,” and “the” can also include plural references unless the context clearly indicates otherwise. Thus, for example, the reference to “component surface” includes referring to one or more such surfaces. When we say that an element is “connected” or “coupled” to another element, the element can be directly connected or coupled to the other element, or it can mean that the element and the other element are connected through an intermediate element. Furthermore, the use of “connected” or “coupled” herein can include wireless connections or wireless couplings.

[0046] The terms “comprising” or “may include” refer to the presence of a corresponding disclosed function, operation, or component that may be used in the various embodiments of this disclosure, rather than limiting the presence of one or more additional functions, operations, or features. Furthermore, the terms “comprising” or “having” may be interpreted as indicating certain characteristics, numbers, steps, operations, constituent elements, components, or combinations thereof, but should not be construed as excluding the possibility of the presence of one or more other characteristics, numbers, steps, operations, constituent elements, components, or combinations thereof.

[0047] The term "or" as used in the various embodiments of this disclosure includes any of the listed terms and all combinations thereof. For example, "A or B" may include A, may include B, or may include both A and B. When describing multiple (two or more) items, if the relationship between the multiple items is not explicitly defined, the multiple items may refer to one, more, or all of the multiple items. For example, the description "parameter A includes A1, A2, A3" can be implemented as parameter A includes A1 or A2 or A3, or it can be implemented as parameter A includes at least two of the three items A1, A2, and A3.

[0048] Unless otherwise defined, all terms used in this disclosure (including technical or scientific terms) have the same meaning as understood by one of those skilled in the art as described herein. Common terms as defined in dictionaries are to be interpreted as having a meaning consistent with the context in the relevant technical field and should not be interpreted ideally or overly formally unless expressly defined in this disclosure.

[0049] At least some of the functions of the device or electronic device provided in this disclosure embodiment can be implemented by an AI model, such as implementing at least one module of a plurality of modules of the device or electronic device by an AI model. AI-related functions can be executed by non-volatile memory, volatile memory, and a processor.

[0050] The processor may include one or more processors. In this case, the one or more processors may be general-purpose processors, such as central processing unit (CPU), application processor (AP), etc., or pure graphics processing unit, such as graphics processing unit (GPU), vision processing unit (VPU), and / or AI-specific processors, such as neural processing unit (NPU).

[0051] The one or more processors control the processing of input data based on predefined operating rules or artificial intelligence (AI) models stored in non-volatile and volatile memory. These predefined operating rules or AI models are provided through training or learning.

[0052] Here, providing through learning refers to obtaining predefined operating rules or an AI model with desired characteristics by configuring a learning algorithm with multiple learning data. This learning can be performed within the device or electronic device itself, in which the AI ​​is executed according to the embodiment, and / or can be implemented via a separate server / system.

[0053] AI models can contain multiple neural network layers. Each layer has multiple weight values, and each layer performs neural network computations by calculating the input data of that layer (such as the computation results of the previous layer and / or the input data of the AI ​​model) and the multiple weight values ​​of the current layer. Examples of neural networks include, but are not limited to, convolutional neural networks (CNNs), deep neural networks (DNNs), recurrent neural networks (RNNs), restricted Boltzmann machines (RBMs), deep belief networks (DBNs), bidirectional recurrent deep neural networks (BRDNNs), generative adversarial networks (GANs), and deep Q-networks.

[0054] A learning algorithm is a method of training a predetermined target device (e.g., a robot) using multiple learning data sets to enable, allow, or control the target device to make determinations or predictions. Examples of such learning algorithms include, but are not limited to, supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning.

[0055] The methods provided in this disclosure may relate to one or more fields in the technical fields of speech, language, image, video, or data intelligence.

[0056] Optionally, in the context of speech or language, according to this disclosure, in a method performed by an electronic device, a matrix computation method may receive a speech signal as an analog signal via a speech acquisition device (e.g., a microphone) and convert the speech portion into computer-readable text using an Automatic Speech Recognition (ASR) model. The user's utterance intent can be obtained by interpreting the converted text using a Natural Language Understanding (NLU) model. The ASR model or NLU model can be an artificial intelligence (AI) model. The AI ​​model can be processed by a dedicated AI processor designed in a hardware architecture specified for processing the AI ​​model. The AI ​​model can be acquired through training. Here, "acquired through training" means obtaining a predefined operating rule or AI model configured to perform desired features (or purposes) by training a basic AI model with multiple training data using a training algorithm. Language understanding is a technique for recognizing and applying / processing human language / text, including, for example, natural language processing, machine translation, dialogue systems, question answering, or speech recognition / synthesis.

[0057] Optionally, in the context of images or videos, according to this disclosure, in a method executed in an electronic device, a matrix computation method can obtain output data for recognizing an image or a feature map in an image by using image data as input data for an artificial intelligence model. The artificial intelligence model can be obtained through training. Here, "obtained through training" means obtaining a predefined operating rule or artificial intelligence model configured to perform desired features (or purposes) by training a basic artificial intelligence model with multiple training data using a training algorithm. The methods of this disclosure can relate to the field of visual understanding in artificial intelligence technology, which is a technology for recognizing and processing things like human vision, and includes, for example, object recognition, object tracking, image retrieval, human recognition, scene recognition, 3D reconstruction / localization, or image enhancement.

[0058] Optionally, in the context of data intelligence processing, according to this disclosure, in methods executed in electronic devices, methods for matrix calculations can be performed using artificial intelligence models. The processor of the electronic device can perform preprocessing operations on the data to transform it into a form suitable for use as input to an artificial intelligence model. The artificial intelligence model can be obtained through training. Here, "obtained through training" means obtaining predefined operating rules or an artificial intelligence model configured to perform desired features (or purposes) by training a basic artificial intelligence model with multiple training data using a training algorithm. Inference and prediction are techniques for logical reasoning and prediction based on determined information, including, for example, knowledge-based reasoning, optimization prediction, preference-based planning, or recommendation.

[0059] The term "based on" as used in the various embodiments of this disclosure can be interpreted as meaning that the premises, conditions, or information upon which it is based are not unique, but at least one or a part of them. That is, it indicates that at least one explicit basis exists, and does not exclude other possible basis.

[0060] The following description of several optional embodiments illustrates the technical solutions of this disclosure and the technical effects produced by these solutions. It should be noted that the following embodiments can be referenced, learned from, or combined with each other. Identical terms, similar features, and similar implementation steps in different embodiments will not be repeated.

[0061] In related technologies, the matrix operation unit includes a cube-shaped MAC (Multiply Accumulate) array unit and an accumulator unit. The cube-shaped MAC array unit performs multiplication operations on the input matrix. Due to its special structure (i.e., 16 16×16 MAC array cores, each of which can perform parallel multiplication of a row of matrix A with the entire matrix B), the 16 MAC array cores can complete two matrix multiplications A×B, each with a maximum size of 16×16, in one clock cycle; for example... Figure 1 As shown.

[0062] However, the matrix operation units provided by related technologies cannot meet the requirements of some real-time scenarios (such as AI communication scenarios and autonomous driving scenarios) where the output data needs to be updated in every clock cycle. With the explosive growth of network data, AI models need larger matrices to accommodate and process this data. Furthermore, as network throughput increases in the future, the matrix size will become even larger, resulting in matrix data sizes far exceeding the size of the MAC array core (16×16). For example, the size of the matrix data (e.g., 16×36) may be larger than the size of the MAC array (e.g., 16×16). Moreover, because the structure of the MAC array is fixed in related technologies, it cannot be changed to adapt to the input matrix size. Therefore, preprocessing of the input matrix is ​​necessary—matrix partitioning (dividing large matrices into smaller matrices suitable for the MAC array) and zero-padding (filling small matrices with zeros to make them suitable for the MAC array size)—to make it suitable for the hardware's operation unit size before computation. Additionally, after multiplying the partitioned matrices, an accumulator is needed to sum these parts to obtain the final result. This lengthy computation process severely reduces the computational efficiency of the matrix operation unit, resulting in the inability to meet the real-time requirements of the output data. Figure 2 As shown.

[0063] To address at least one technical problem in related technologies, this disclosure provides an intelligent matrix operation architecture based on dynamically reconfigurable MAC arrays, such as a matrix operation accelerator. Starting with the shape (rows × columns) of the hardware MAC array and combining it with matrix operation rules, the structure of the matrix operation units (such as the MAC array) can be dynamically adjusted to adapt to matrices of different sizes, ensuring that their (rows × columns) size is greater than or equal to the size of the input matrix. This eliminates the need for matrix preprocessing, avoids matrix partitioning and zero-padding, and thus eliminates the need for accumulators after the MAC units. This fully utilizes hardware resources and improves computational efficiency, thereby meeting the requirements for real-time computation. Figure 3 As shown.

[0064] In one example, the matrix size used in the AI ​​communication module's calculations is mostly less than 100. For instance, the matrix data size of AIDPD is 16×36, which, if... Figure 3 If the related technical solutions (such as CUBE) are used to meet the real-time output data, then the size of the MAC array should be 36×36. Therefore, the three-dimensional size of the related technology is 36×36×36. It can be seen that this size requires 46656 MAC units, and the resource utilization rate is 12 times that of the related technology. Therefore, from the perspective of cost, the feasibility of commercial use is very low.

[0065] The solution provided in this disclosure requires only a 16×36 MAC array (576 MAC units) to meet the real-time requirements of the output data. In terms of resource utilization, it saves a significant amount of resources compared to the aforementioned related technical solutions. Using a 16×36 MAC array in the NPU solution can effectively improve the feasibility of commercial use.

[0066] The solution provided in this disclosure can dynamically change the structure of the MAC array by using configurable MAC units, thereby achieving adaptive input matrix size and avoiding matrix segmentation.

[0067] like Figure 4a As shown, AI model computation involves matrix operations of varying sizes. Traditional NPU computation units preprocess (matrix partitioning) input matrices of different sizes to adapt to the hardware architecture. However, the solution provided in this disclosure dynamically adjusts the structure of the matrix computation unit to accommodate matrices of different sizes without requiring matrix preprocessing. This function is implemented by a MAC (Multiply-Accumulate) unit that can dynamically change its row and column structure and a configuration logic unit (also known as a control logic unit). The configuration logic unit is responsible for adjusting the structure of the MAC unit according to the size of the input matrix. The MAC unit receives configuration information to change the connection structure and perform multiplication and addition calculations. The MAC unit and control logic unit are located in the matrix computation module of the NPU and are responsible for accelerating matrix computation functions.

[0068] The matrix operation unit is located at the core of the NPU, including control logic and a MAC array. Compared with NPU matrix operation units in related technologies, the solution provided in this disclosure forms cascaded data paths between MAC units in the MAC array, and there is also cascading between rows, which is not limited by the number of physical rows in the MAC array. Each MAC unit can output the final calculation result and can accommodate matrices of any size for calculation, such as... Figure 4b and Figure 4c As shown.

[0069] The core of the NPU is the matrix operation unit, which interacts with external data caches to exchange matrix and vector data and exchange configuration instructions with external CPUs.

[0070] The MAC array unit contains H×L MAC units, which have configurable registers and multiple data path selectors (MUX).

[0071] The registers in each MAC unit can control a variety of functions, such as storing data in the W matrix to reduce the number of times the weight matrix W data is read, and controlling the multiplication and addition operations to be turned off based on the 0 elements in the W matrix, so as to improve efficiency and reduce power consumption.

[0072] The cascading of data paths between MAC units is not limited by the number of rows in the MAC array. Each MAC unit can output the final calculation result and can accommodate matrices of any size for computation. It also has certain advantages in terms of hardware resource utilization, because no preprocessing of the input matrix is ​​required, thus eliminating the need for an accumulator after the MAC unit.

[0073] This disclosure provides a processing module (such as a matrix operation unit) including a control logic unit and multiple operation units (such as a multiply-accumulate MAC unit).

[0074] The control logic unit is configured to determine multiple first arithmetic units and configuration information of each first arithmetic unit in the arithmetic unit included in the first processing module according to the size information of the input data, and configure each first arithmetic unit according to the corresponding configuration information. The size of the arithmetic array corresponds to the size of the input data, and the configuration information includes information related to the position of the first arithmetic unit in the arithmetic array.

[0075] The arithmetic unit is configured to process the input data.

[0076] Optionally, the input data (also known as the input matrix) can be the raw or processed data received by the AI ​​model, represented in matrix form. The input matrix can be a two-dimensional array, where rows represent samples or data points, and columns represent features or variables. The size information of the input matrix can be the number of rows M and the number of columns N. In matrix operations, the input matrix can be divided into a weight matrix (also called the processed weight data) and an input data matrix (also called the data to be processed). The weight matrix, represented by W, is a matrix used to store model parameters. The input data matrix, represented by X, is a matrix used to carry the data to be processed. For example, in an image classification task, the weight matrix can be the parameter set of a CNN convolutional kernel, and the input data matrix can be an arrangement of image pixel values.

[0077] Optionally, the computational array composed of the first computational units is also called a MAC array (Multiply-Accumulate Array), a hardware structure used in neural network accelerators (NPUs). A MAC array consists of a large number of MAC units, each capable of performing multiplication and addition operations simultaneously. MAC units are typically arranged in a two-dimensional array to enable parallel computation, with data flowing through the rows and columns of the array. In tasks such as image classification and object detection, MAC arrays can be used to improve inference speed.

[0078] In this embodiment, considering that AI models involve matrix operations of different sizes during computation, dynamically adjusting the structure of the matrix operation unit to adapt to input matrices of different sizes eliminates the need for matrix preprocessing, thus improving computational efficiency. The matrix operation unit provided in this disclosure is located in the NPU and includes a control logic unit and an operation unit. It interacts with an external data cache for matrix and vector data and with an external instruction unit for CPU-configured instruction data. The control logic unit configures the structure of the MAC unit according to the size of the input matrix. After receiving the configuration information and changing the connection structure, the operation unit can perform multiplication and addition calculations. When the size of the input matrix changes with the AI ​​model, the matrix operation unit adaptively changes the structure of the MAC array, thereby avoiding matrix partitioning and merging operations, improving computational efficiency, and ensuring real-time operation. Figure 4c As shown.

[0079] In one feasible embodiment, the arithmetic unit includes a register.

[0080] Optionally, the control logic unit is configured to configure the configuration information of each first arithmetic unit into the register of that first arithmetic unit. The arithmetic unit is configured to process the input data according to the configuration information in the register.

[0081] In relevant technical solutions, such as Figure 5a As shown, the connections between each MAC unit in the MAC array are fixed. When a MAC unit is at the first position in each row of the array, its input should be connected to the W and X data buffers, and its output should be connected to the next MAC unit. When a MAC unit is at the end of each row of the array, its input should be connected to the W and X data buffers and the output of the previous MAC unit, and its output should be connected to the next stage adder. When a MAC unit is in the middle of each row of the array, its input should be connected to the W and X data buffers and the output of the previous MAC unit, and its output should be connected to the next stage MAC unit.

[0082] In the embodiments disclosed herein, such as Figure 5bAs shown, the logical rows of the MAC array are not fixed; each MAC unit is cascaded. The inputs of a MAC unit are connected to the X data cache, the output of the previous MAC unit, and the configuration interface. The inputs are no longer connected to the W data cache, as the W data cache is stored in registers within the MAC unit. The outputs are connected to the next-level MAC unit and the computation module. When the MAC unit is at the last position of a logical row, the output is connected to the next-level computation module; otherwise, the output is connected to the next-level MAC unit. The first MAC unit in the MAC array has an input of 0, and its other interfaces are the same as described above. The last MAC unit in the MAC array has its output connected only to the next-level computation module, with the same interface as described above.

[0083] The MAC array provided in this disclosure cascades data paths between MAC units, and also cascades between physical rows, without being limited by the number of physical rows in the MAC array. Each MAC unit can output the final calculation result, thus accommodating matrices of arbitrary size for calculation; for example... Figure 5c As shown. Optionally, the MAC matrix has the following properties: The MAC array unit contains H×L MAC units, each with configurable register units and multiple data path selectors.

[0084] The register units in each MAC unit can control multiple functions, such as storing data of the W matrix to reduce the number of times the W data is read, and controlling the multiplication and addition operations to be turned off based on the 0 elements in the W matrix to improve efficiency and reduce power consumption.

[0085] The MAC units in the MAC array are structurally optimized so that their outputs connect to the next MAC unit and the next level of computing modules. When a MAC unit is located at the last position of a logical row in the MAC array, its output connects to the next level of computing modules; otherwise, its output connects to the next level of MAC units.

[0086] When the MAC unit is the first MAC unit in the array, the input is 0; the other interfaces are the same as described above.

[0087] When the MAC unit is the last MAC unit in the array, its output is only connected to the next level computing module, and the other interfaces are the same as described above.

[0088] Since no preprocessing of the input matrix is ​​required, the accumulator after the MAC unit is no longer needed, thus saving hardware resources.

[0089] In one feasible embodiment, the control logic unit determines the number of logical rows and columns of the MAC units participating in the calculation based on the size of the input matrix, generates configuration information (such as a register map) based on this size, and sends it to the register units of the MAC units to guide their calculation behavior, such as... Figure 6 As shown. Optionally, the control logic unit may also receive configuration instructions from the CPU, such as controlling the sparsity of the weight matrix W.

[0090] Among them, the control logic is the core unit of dynamic configuration. It generates configuration information (register mapping) based on the size of the input matrix to configure the MAC unit and guide its adaptation and operation behavior.

[0091] Optionally, the control logic unit includes a first logic unit (also known as a size adaptation comparison logic unit) and a second logic unit (also known as a configuration information generation logic unit).

[0092] Optionally, the first logic unit is configured to generate position information of the MAC unit related to the output result of each logic row based on the size information of the input matrix and the structure information of the MAC array; the second logic unit is configured to generate configuration information for the MAC unit used in the MAC array based on the position information and the weight value of the input matrix.

[0093] The size adaptation comparison logic unit calculates the fit of the hardware MAC array to the calculation size of the input matrix based on the hardware MAC array structure (L×H) (from CPU configuration instructions) and the input matrix size (M×N).

[0094] The configuration information generation logic unit generates instructions for the configuration register based on the size adaptation position information generated by the size adaptation comparison logic unit, in order to configure the register map of the MAC array. It is also responsible for configuring the weight parameter W into the MAC unit.

[0095] The register map represents the register configuration table for the entire MAC array. Each cell in the table corresponds to a register group in a MAC unit, with a total of (L×H). The default initial value for all registers is 0. It modifies the values ​​of certain registers based on the configuration register instructions generated by the adaptation logic unit, thereby altering information such as the output path and starting position of the calculation results.

[0096] Unused MAC units are turned off by a clock gate to reduce power consumption.

[0097] In one feasible embodiment, the position-related information includes: first indication information indicating at least the first operation unit is located at the beginning and / or end of the operation row in the operation array.

[0098] Optionally, the first indication information can be used to indicate the position information of the first arithmetic unit in the arithmetic array, which can be determined by the size adaptation comparison logic part in the control logic unit.

[0099] In one feasible embodiment, the first logic unit is configured as follows: The M×N input data and the L×H operation unit in the first processing module are respectively converted into one-dimensional row vectors, where M, N, L and H are not less than 1; The input data 1×N row vector is mapped to the row vector of the 1×(L×H) operation unit contained in the first processing module; The first indication information is determined based on the position of the corresponding arithmetic unit that is a multiple of the mapping length N.

[0100] Optionally, the weight matrix in the M×N input matrix and the L×H MAC array are converted into one-dimensional row vectors, where M, N, L, and H are not less than 1. The row vectors of the 1×N input matrix are mapped to the row vectors of the 1×(L×H) MAC array, and the first position information of the MAC unit corresponding to the mapping length being a multiple of N and the second position information of the next adjacent MAC unit are recorded. The first position information indicates the position of the MAC unit where the current logical row operation ends, and the second position information indicates the position of the MAC unit where the next logical row operation begins. The first position information and the second position information are then mapped inversely to the two-dimensional coordinates of the MAC array and sent to the second logic unit.

[0101] Optionally, other position information of the first arithmetic unit in the arithmetic array can be determined by the first position information and / or the second position information.

[0102] Optionally, the size-adaptive comparison logic unit can be implemented by expanding the input two-dimensional matrix and MAC array into their respective one-dimensional vectors, and mapping the input matrix vectors to the positions of the MAC vectors to obtain the size of the logical rows and columns, such as... Figure 7 As shown. The specific principle of its execution is as follows: To enable the MAC array to adaptively match the size of the input matrix, a vector mapping strategy is employed, mapping the input matrix W (M / N) to the hardware MAC array (L×H). The input matrix W and the MAC array are flattened into two one-dimensional row vectors. Then, the row vector (1×N) of the input matrix is ​​mapped to the entire row vector of the MAC array (1×(L×H)).

[0103] When flattening the MAC array, since the MAC array is transformed from two-dimensional to one-dimensional, it is necessary to convert the coordinates of the MAC array into a one-dimensional sequential sequence.

[0104] Based on the row vector of the MAC, when mapping to a length that is a multiple of N, the position information of that MAC cell and the position information of the next adjacent MAC cell are recorded. These two pieces of position information will serve as the basis for the position of the subsequent configuration registers "Start STA" and "End END". Figure 5c If the MAC unit at position (1,1) is a MAC unit corresponding to a mapping length N times the length of N, then the position information (1,1) and (1,2) are recorded. The MAC unit at position (1,1) is the end MAC unit of the logical row, and the MAC unit at position (1,2) is the beginning MAC unit of the logical row. Accordingly, when subsequently identifying the first and second registers in the MAC units, the value of the END register in the MAC unit at position (1,1) is configured to "1" (or other preset value), and the value of the STA register in the MAC unit at position (1,2) is configured to "1" (or other preset value), so as to control the end and start of the operation through the register values.

[0105] The recorded location information is mapped back to the two-dimensional coordinates of the MAC array and sent to the configuration information generation unit.

[0106] In one feasible embodiment, the configuration information further includes at least one of the following: A second indication message indicating whether the processing weight data corresponding to the first processing unit is zero; The processing weight data corresponding to the first processing unit; The location information of the first arithmetic unit in the first processing module.

[0107] Optionally, the location information includes at least one of the following: The first position information is used to indicate the position of the first operation unit after the operation of each operation row ends; The second position information is used to indicate the position of the first operation unit at the start of each operation row.

[0108] The first position information is mapped to the first register value of the corresponding MAC unit, and the first register is used to indicate the end of the operation; the second position information is mapped to the second register value of the corresponding MAC unit, and the second register is used to indicate the start of the operation.

[0109] Optionally, the configuration information generation logic unit, based on the position information generated by the size adaptation comparison logic unit, disables unused MAC units and generates and configures register maps for used MAC units, thereby achieving adaptive functionality for logical rows and columns. Figure 9 As shown.

[0110] The configuration method of this embodiment adopts MAP form, with the initial value defaulting to 0. Simply write the value "1" (or other preset value) in the appropriate position to save configuration time.

[0111] Optionally, the configuration information generation logic unit receives the matrix size and position information output by the size adaptive logic unit and the weight values ​​of the input W matrix, and performs at least one of the following configuration operations: Based on the number of MAC units in use, unused MAC units are shut down via a gated clock to reduce power consumption. Configure the W weight values ​​into the W register of all used MAC units to reduce the number of times W data is read during computation. Based on whether the W value is "0", configure the ZERO register of the corresponding MAC unit by setting this bit to "1". If it is set to "1", the multiply-accumulate operation of that MAC unit is disabled to improve efficiency and reduce power consumption. Based on the "start" position information of the MAC unit, the "start" bit of the corresponding MAC is configured to "1". Similarly, the "end" bit is configured to "1" to determine the length of the MAC unit in each logical row (one row of the W matrix) and the output position of the calculation result of a logical row.

[0112] In one feasible embodiment, the arithmetic units in the first processing module are sequentially cascaded, and the output port of each arithmetic unit is connected to the second processing module. Optionally, the MAC array is constructed by sequentially cascading MAC units, and physical rows are also cascaded, breaking the constraint of logical rows on physical rows, thereby enabling more flexible matrix operations; such as Figure 10 As shown.

[0113] Implementations of this disclosure remove the physical row size limitation from the MAC matrix, allowing for longer (or shorter) logical rows to fit the size of the input matrix. It has the following properties: The operation of the MAC unit is controlled by its contained registers, the configuration values ​​of which come from the control logic unit. A MAC array physically consists of multiple ROW-structured MAC operation units, each of which is primarily responsible for performing a multiplication and an addition operation. The last MAC unit in each row is responsible for cascading between physical rows, passing data to the first MAC unit of the next physical row. The elements of matrix W are stored in the W register in MAC, and due to the characteristics of the weight parameters, they can remain stable for a certain period of time. Matrix X is the data to be processed in the operation. It is pushed into the MAC unit by the shift register (SR) with the clock pulse and multiplied with W.

[0114] In one feasible embodiment, each MAC unit includes a second data path selector and a first data path selector.

[0115] Optionally, a second data path selector is configured to determine whether the data input to the previous MAC unit should be multiplied and added in the current MAC unit based on the weight values ​​of the input matrix or the data to be processed.

[0116] In one feasible embodiment, the arithmetic unit includes a first data path selector, which is configured to: When the first instruction indicates that the first arithmetic unit is at the beginning of the arithmetic line, the result of the arithmetic operation is output to the next connected first arithmetic unit; and / or When the first indication information indicates that the first arithmetic unit is at the end of the arithmetic line, the arithmetic result is output to the second processing module; and / or When the first instruction indicates that the first arithmetic unit is not located at the beginning or end of the arithmetic line, the arithmetic result is output to the next connected first arithmetic unit.

[0117] Optionally, the first data path selector is configured to determine, based on location information, whether to output the calculation result of the current MAC unit to the next MAC unit or to output it as the final result of the current logic row.

[0118] The MAC unit contains registers and a data path selector (MUX). The connection relationship and operation behavior of the MAC unit are changed according to the value of the register, thereby achieving the effect of adaptive input matrix size and efficient calculation. In this embodiment, the MAC unit adds a MUX and a register configuration unit (also known as a register unit) to the existing multiplier and adder, thus enabling flexible implementation of various data transfer and computation behaviors; such as Figure 11 As shown.

[0119] In one feasible embodiment, the MAC unit further includes a register configuration unit, which includes at least one of the following: The weight register is used to configure the weight values ​​of the input matrix; The zero register is used to instruct the corresponding MAC unit not to perform a multiply-accumulate operation when the weight value is zero. The first register is used to indicate the end of the operation; The second register is used to indicate the start of the operation.

[0120] Optionally, each MAC unit includes a multiplier, an adder, two data path selectors (MUX), and a register configuration unit (RU).

[0121] Optional, such as Figure 11 As shown in the right figure, the RU contains the following registers, which receive configuration information from the control logic unit and change the values ​​of the registers accordingly, thereby controlling the MAC unit to perform the corresponding functions (such as controlling the changes in the MAC unit connection relationship by configuring different values): MN: The position information of this MAC unit in the physical array, represented by the values ​​of H and L (row and column). It can be configured by the CPU according to the structure of the MAC array during initialization and does not change during calculation. Its purpose is to guide the selection of the control position for adaptive size and the broadcast position.

[0122] W: The elements of the stored weight matrix W. The configuration logic unit will detect changes in the input W matrix. When it changes, it will update the new W value and the specifications of W (the values ​​of W register, STA, END, and ZERO will all change accordingly).

[0123] ZERO: Set this bit to "1" based on whether the value of the input W element is "0". After setting, it controls the previous input MUX1 to pass the data to be processed directly to the output, and at the same time, it turns off the multiplier.

[0124] STA: The label of the first MAC cell in each logical row of the MAC array, corresponding to the first element of each row of the input matrix W.

[0125] END: The label of the last MAC cell in each logical row of the MAC array, corresponding to the last element of each row of the input matrix W.

[0126] MN, STA, and END are responsible for marking the position of the MAC unit to determine the length of the logical row. W is responsible for storing the W matrix data, so the MAC unit no longer needs a W input interface. ZERO is responsible for shutting down the MAC unit based on whether W is 0, in order to improve efficiency.

[0127] Optionally, MUX1 is responsible for selecting whether the data input from the previous stage is passed through at this stage or subjected to normal multiplication and addition operations.

[0128] Optionally, MUX2 is responsible for choosing whether to output the calculation result of this MAC unit to the next MAC unit or as the final result of this logic line to the next level calculation module.

[0129] Optionally, considering that storing the weight matrix W in registers is relatively inefficient in some AI models where the weight matrix W needs to change frequently, the weight register W can be removed, and the W data can be read into the MAC unit using conventional methods. This reduces the use of register resources, decreasing the total number of bits from 51 to 19, thereby reducing the overall hardware resource consumption. Figure 12 As shown.

[0130] Optionally, when the number of zero elements in the input matrix is ​​extremely small, the zero-value register ZERO can be removed to simplify the register configuration process. For example... Figure 13 As shown.

[0131] Optionally, for scenarios where the W matrix changes infrequently, the MAC unit is located at the end of the array, or register resources are limited, the first register END can be removed. In this case, the position flag of the second register STA can be decremented by "1" to locate the original first register END. Figure 14 As shown.

[0132] Optionally, if the W matrix does not change frequently, the MAC unit is located at the end of the array, or register resources are limited, the second register STA can be eliminated. In this case, the original second register STA can be located by adding a "1" to the position flag of the first register END.

[0133] In this embodiment of the disclosure, based on the deletable nature of registers W, ZERO, STA, and END, these four registers can be arbitrarily combined with the MN register to adapt to suitable application scenarios.

[0134] In one feasible embodiment, the computing unit includes a second data path selector, and the first data path selector is configured such that when the second indication information indicates that the processing weight data corresponding to the first computing unit is zero, the first computing unit directly outputs the computing result of the previous first computing unit to the next first computing unit or the second processing module.

[0135] Optionally, the control terminal of the second data path selector is configured with a preset value when the weight value of the input matrix and / or the data to be processed in the input matrix is ​​zero, so as to determine that the data output by the previous MAC unit is directly transmitted to the next MAC unit.

[0136] In this embodiment, a zero-value control function is also provided. This function is an adaptive function included in the MAC unit. Based on the zero values ​​in the W matrix elements, it sets the ZERO register, controlling the front-end data to be processed to pass directly to the output, eliminating the need for multiplication and addition operations within the MAC unit. Similarly, when a zero element in the X matrix is ​​detected, the front-end data to be processed can also be directly controlled to pass directly to the output, thereby improving computational efficiency and reducing power consumption. Figure 15 As shown.

[0137] The control terminal of the front-end data path selector MUX1 is affected by two factors: first, the 0 value in the input matrix W; and second, the 0 value in the input matrix X. Regardless of which path contains a 0 value, the control terminal will be set to "1". For example... Figure 15 In As shown. When the control terminal of MUX1 is set to "1", MUX1 will directly transmit the data input from the previous stage to the output terminal. At the same time, the control terminal is connected to the enable terminal of the multiplier to turn off the multiplier. At this point, the data passes directly through the MAC unit, and the MAC operation is not initiated, thus reducing computational latency and power consumption. This effect is even more pronounced when dealing with sparse matrices, where most MAC units are in a data pass-through state.

[0138] In one feasible embodiment, the control logic unit is further configured to: The CPU receives the first threshold, which is configured by the CPU based on the current calculation result of the input matrix. Set the values ​​of elements in the weight matrix of the input matrix that are less than the first threshold to zero.

[0139] Optionally, the sparsity of the W matrix can also be dynamically adjusted based on the zero_th threshold (such as the first threshold) configured by the CPU: The execution results (such as the quality of the constellation diagram) output by the matrix operation device are fed back to the CPU monitoring end. The CPU dynamically adjusts the zero_th value according to the current data quality (such as accuracy, loss value, etc.) and sends it to the control logic unit.

[0140] The control logic unit adjusts the elements in the W matrix that are less than zero_th to 0, performs the next round of update calculations, and feeds back the results until the feedback received by the CPU achieves the expected effect. This achieves adaptive adjustment of the sparsity of W, such as... Figure 16 As shown. (DU stands for device in a wireless base station).

[0141] Optionally, the first threshold is used to determine whether the element values ​​in the weight matrix need to be set to zero. The higher the first threshold, the more elements in the weight matrix are set to zero, which can improve the sparsity of the weight matrix and thus reduce the amount of computation.

[0142] By adaptively adjusting the sparsity of the W matrix, we can improve computational efficiency, reduce device power consumption, and ensure computational accuracy.

[0143] In one feasible embodiment, the arithmetic unit is further configured as follows: Configure the weight values ​​of the input matrix into the weight register of the corresponding MAC unit; And / or, map the data to be processed sequentially to the first sequence in column order, map the first operation unit in the operation matrix sequentially to the second sequence, and input the data to be processed to the first operation unit through the first sequence and the second sequence; And / or, starting from the first position of each operation row in the operation array, the data to be processed is sequentially input into each first operation unit, column by column.

[0144] Optionally, because the MAC array can dynamically adapt to the size of the input matrix, the data propagation path changes. Therefore, the data paths for W and X need to be redesigned. Figure 17a As shown, after determining the MAC array adaptation structure, the logical rows of matrix W data no longer correspond one-to-one with the physical rows of the MAC array. The broadcast path of matrix X data is no longer limited to this column. Therefore, a simple and efficient data path needs to be designed to adapt to the functionality of the MAC array.

[0145] On the one hand, embodiments of this disclosure provide a data path for the W matrix.

[0146] The W matrix is ​​typically used as a weight matrix and has the property of remaining unchanged over a long period of time. Therefore, it can be temporarily stored in the register of each MAC unit.

[0147] Each MAC unit has a 32-bit space in its register (RU) to store the elements of the W matrix. Each MAC unit has one register, and the elements of the W matrix are directly written into this register by the control logic unit.

[0148] The W value written to the register will be stored in the corresponding MAC for a long time and multiplied with the data X to be processed. The register will only be updated with the new W matrix elements when a W weight update is needed. Figure 17b As shown.

[0149] This method effectively reduces the number of times the W matrix needs to be read, especially when W is used as the weight matrix, thus improving computational efficiency.

[0150] On the other hand, embodiments of this disclosure provide a data path for the X matrix.

[0151] Matrix X, as the data to be processed, is input to the corresponding MAC unit via broadcast. Since the logical rows of the MAC array no longer correspond to the physical rows, that is, one row of matrix W no longer strictly corresponds to a physical row of the MAC array, the broadcast path of X data has also changed. Therefore, it is necessary to design a simple and efficient broadcast path to achieve the correct broadcast of X data.

[0152] From a data connectivity perspective, the X data interface interconnects with the MAC array interface, similar to the routing principle of data ports in a router. Therefore, a broadcast control module is designed to achieve router-like functionality. By correctly connecting the X data ports and MAC unit data ports on both sides, broadcasting of X data can be achieved. Figure 18 As shown.

[0153] Optionally, during X data broadcasting, each shift register SR port in the X matrix containing the data to be processed in the input matrix is ​​mapped sequentially to a first data port in ascending order, and each data port in the MAC array is mapped sequentially to a second data port in ascending order, and the first data port and the second data port are connected.

[0154] The broadcast control logic maps the ports on both sides based on the configuration information (input matrix dimensions M / N and MAC array dimensions H / L): The X data corresponds to each logical column of the MAC array. Therefore, the ports of each SR in the X matrix are mapped to the sequence "TS" (sequentially increasing data ports) in column order, such as... Figure 18 The port sequence numbered 0 to 45 (including 46 TS ports) is called TS.

[0155] Map the data ports on the MAC array side sequentially to the sequentially increasing port "TD", such as... Figure 18 As shown, the range of MAC units used is (0,0) to (5,14), which are mapped to a port sequence numbered 0 to 91 (including 92 TD ports), referred to as TD.

[0156] Concatenate TS and TD according to the rule: TD%N=TS (where N is the dimension of the input matrix, and % is the modulo operation). For example... Figure 18 As shown, based on the value of TD, when TD is 0 and 46, its input port comes from TS 0. In this case, connect both of these ports to TS 0, and similarly connect the remaining ports.

[0157] Broadcasting X data using this routing principle can improve the efficiency of broadcast path decision-making, greatly simplify the design of X data broadcast circuits, and save more hardware implementation resources.

[0158] Optionally, the SR port of the data to be processed in the first column of the X matrix can be connected to the input port of all MAC units that indicate the start of the operation, and the SR port of the data to be processed in the next column can be connected to the input port of the next MAC unit after that MAC unit, until the SR ports of all data to be processed are connected.

[0159] Optionally, X data can be broadcast based on the values ​​of registers "STA" and "MN" in the MAC unit, such as... Figure 19 As shown: Connect the SR port of the X data in the first column to the X data input port of all MAC units with "STA=1". Connect the SR port of the X data in the second column to the X data input port of the MAC unit next to the MAC unit of “STA=1”.

[0160] Similarly, connect all the SR ports of X to the ports of the MAC unit.

[0161] Broadcasting X data using this register positioning method can greatly simplify the design of the X data broadcasting circuit.

[0162] This disclosure provides an intelligent acceleration scheme based on dynamic reconfiguration, including dynamically adaptive adjustment of the MAC array structure and a configurable MAC unit design. This disclosure provides a hardware implementation basis for real-time artificial intelligence computing scenarios, such as 5G communication baseband processing requiring dynamic input and multimodal autonomous driving perception systems.

[0163] Based on the same inventive concept, embodiments of this disclosure also provide a method performed by an electronic device.

[0164] In one feasible embodiment, such as Figure 22 As shown, specifically including S001 to S003: S001. Based on the size information of the input data, determine multiple first arithmetic units and the configuration information of each first arithmetic unit in the arithmetic units included in the first processing module.

[0165] S002. Configure each first arithmetic unit according to the corresponding configuration information.

[0166] S003. Process the input data based on the arithmetic array composed of each first arithmetic unit.

[0167] The size of the computing array corresponds to the size of the input data, and the configuration information includes information related to the position of the first computing unit in the computing array.

[0168] Optionally, the input data includes at least one of communication data, image data, audio data, and text data. The communication data is the communication data received by the electronic device and to be pre-distorted; the electronic device is a network node or user equipment.

[0169] Optionally, the arithmetic unit is a multiply-accumulate (MAC) unit, and the arithmetic array composed of each first arithmetic unit can be called a MAC array.

[0170] Furthermore, the method executed by the electronic device provided in this disclosure can be implemented by the processing module of the above embodiments. The relevant description can be referred to the relevant parts of the above embodiments, and will not be repeated here.

[0171] In a feasible embodiment, in S002, each first arithmetic unit is configured according to the corresponding configuration information, including: configuring the configuration information of each first arithmetic unit into the register of the first arithmetic unit respectively.

[0172] Optionally, the arithmetic unit includes registers. By configuring the configuration information of each first arithmetic unit into the registers of the corresponding first arithmetic unit, the values ​​of certain registers can be changed, thereby altering information such as the output path and starting position of the calculation result. For related operations concerning register configuration in the arithmetic unit, please refer to the relevant examples provided in the above embodiments; these will not be elaborated upon here.

[0173] In one feasible embodiment, the position-related information includes: first indication information indicating that the first arithmetic unit is located at the beginning and / or end of the arithmetic row in the arithmetic array.

[0174] Optionally, the first indication information is used to indicate the location information of the arithmetic unit, which can be determined by the size adaptation comparison logic part of the control logic unit in the above embodiment. The arithmetic behavior is obtained by sequentially cascading each arithmetic unit, such as... Figure 5b As shown.

[0175] Optionally, in S003, the input data is processed accordingly, including at least one of S003a to S003c: S003a. When the first instruction information indicates that the first arithmetic unit is at the beginning of the arithmetic line, the first arithmetic unit outputs the arithmetic result to the next connected first arithmetic unit.

[0176] S003b. When the first instruction information indicates that the first arithmetic unit is located at the end of the arithmetic line, the first arithmetic unit outputs the arithmetic result to the second processing module.

[0177] S003c. When the first indication information indicates that the first arithmetic unit is not located at the beginning or end of the arithmetic line, the first arithmetic unit outputs the arithmetic result to the next connected first arithmetic unit.

[0178] Optional, such as Figure 5b As shown, the first operation unit located at the beginning position (the first one) can output the operation result to the next first operation unit, the first operation unit located in the middle position (neither the beginning nor the end position) can output the operation result to the next cascaded first operation unit, and the first operation unit located at the end position (the last one) can output the operation result to the second processing module.

[0179] Optionally, the arithmetic units in the first processing module are cascaded sequentially, and the output port of each arithmetic unit is connected to the second processing module.

[0180] In this embodiment of the disclosure, the computational array, which is adaptively adjusted according to the size information of the input data, is not limited by the number of physical rows, and each computational unit can output the final calculation result, thereby realizing more flexible matrix operations and improving computational efficiency.

[0181] In one feasible embodiment, the input data includes data to be processed and processing weight data. Optionally, the data to be processed is X data, and the processing weight data is W data.

[0182] The configuration information also includes at least one of the following: A second indication message indicating whether the processing weight data corresponding to the first processing unit is zero.

[0183] The processing weight data corresponding to the first processing unit; The location information of the first arithmetic unit in the first processing module.

[0184] Optionally, the second indication information can be used to indicate the zero value situation of the weighted data. It is adapted to the configuration in the zero value register. When a zero value is input into the zero value register, it indicates that the corresponding first arithmetic unit does not perform multiplication and addition operations, that is, the corresponding multiplier is turned off.

[0185] Optionally, the processing weight data corresponding to the first arithmetic unit included in the configuration information can be configured using the weight register shown in the above embodiment.

[0186] Optionally, the position information of the first arithmetic unit in the first processing module included in the configuration information can be used to configure registers STA and / or END to indicate the position of the first arithmetic unit in the arithmetic row.

[0187] Optionally, the location information includes at least one of the following: The first position information is used to indicate the position of the first operation unit after the operation of each operation row ends; The second position information is used to indicate the position of the first operation unit at the start of each operation row.

[0188] Optionally, the configuration information shown in the above embodiments can be used to configure the register group in the MAC unit to control the calculation.

[0189] Optionally, the generation of configuration information can be performed by the second logic unit in the above embodiment to generate a register mapping diagram to complete the configuration of the register group in the MAC unit.

[0190] Optionally, the weight values ​​of the input matrix can be configured into the weight register of the corresponding MAC unit. Configuring the weight values ​​of the input matrix into the W register of the corresponding MAC unit establishes a data path for the W matrix. The W values ​​written to the register are stored in the MAC unit for an extended period and multiplied with the data X to be processed until a weight value update is needed. Then, the control logic unit updates the W matrix elements to the W register of the MAC unit, effectively reducing the number of times the W matrix is ​​read and improving computational efficiency.

[0191] In a feasible embodiment, processing the input data in S003 includes: when the second indication information indicates that the processing weight data corresponding to the first operation unit is zero, the first operation unit directly outputs the operation result of the previous first operation unit to the next first operation unit or the second processing module.

[0192] Optionally, if the weight value of the input matrix or the data to be processed is zero, the data output from the previous MAC unit to the current MAC unit can be passed to the next MAC unit or the next processing module.

[0193] Optionally, this disclosure provides a zero-value control scheme, which can determine whether the calculation result output by the previous first calculation unit is directly transmitted to the next first calculation unit or the second processing module based on the zero value in the W data. Specifically, it can be implemented by the second data path selector MUX1. For details, please refer to the zero-value processing embodiment of the W data in the input data in the above embodiment. This disclosure will not be repeated here.

[0194] In one feasible embodiment, the method provided by this disclosure further includes: using a gated clock to shut down each arithmetic unit in the first processing module that does not process input data.

[0195] Optional, such as Figure 6 As shown, a gated clock can also be used to shut down unused processing units in the first processing module, reducing power consumption.

[0196] In one feasible embodiment, the determination of the first indication information includes step A1: determining the first indication information based on a comparison between the size of the input data and the size of the computing unit contained in the first processing module.

[0197] Optionally, based on the size information of the input matrix and the structural information of the MAC array, the fitting degree between the MAC array and the input matrix is ​​calculated, and the position information in the MAC array that matches the size of the input matrix is ​​determined.

[0198] Optionally, the fit is the degree of matching between the input matrix and the MAC array. For example, the utilization rate of the MAC array can be determined by calculating the ratio of the number of MAC units actually used in the MAC array to the total number of MAC units. For example, if the size of the input matrix is ​​32×32 and the size of the MAC array is also 32×32, then the utilization rate is 100%, and the input matrix and the MAC array are perfectly matched.

[0199] Optionally, the determination of the first indication information can be configured to be implemented in the control logic unit.

[0200] Optionally, the determination of the first indication information in step A1 includes steps A11 to A13: Step A11: When the rows and columns of the M×N input data are both smaller than the rows and columns of the L×H operation units in the first processing module, the M to (L-1) rows and N to (H-1) columns in the MAC array are turned off by configuring the gate clock, and the first indication information is determined based on the position of the N-1th operation unit corresponding to each row, such as marking the N-1th MAC unit of each row as the end MAC unit, and determining its first position information.

[0201] Step A12: When the rows and columns of the M×N input data are both smaller than the rows and columns of the L×H operation units in the first processing module, the (M×N / H)-(L-1) rows and (M×N%H)-(H-1) columns in the MAC array are turned off by configuring the gate clock, and the first indication information is determined based on the position of the (N-1)th operation unit corresponding to each row. For example, the N×ith MAC unit of each row is marked as the end MAC unit (determining its first position information), and the N×i+1th MAC unit of each row is marked as the start MAC unit (determining its second position information), where i is an integer that increments sequentially from 1 to M.

[0202] Step A13: If the total size of the input data is greater than the total size of the computing units contained in the first processing module, perform loop calculation. At the end of this calculation, rearrange the row vectors of M-(L×H) / N into the computing array, and concatenate the current calculation result with the previous calculation result. Determine the first indication information based on the output position of each calculation result, such as marking the end MAC unit of each round (determining its first position information) and the start MAC unit of the next round (determining its second position information) based on the output position of each calculation result.

[0203] Optionally, the calculation logic for matrix and MAC array size adaptation in various cases is shown in Table 1: Table 1. Calculation results for adapting the input matrix to the MAC array size.

[0204] When N ≤ H and M < L, it means that the rows and columns of the input matrix are smaller than the rows and columns of the hardware MAC array. In this case, the M-(L-1)th row and N-(H-1)th column of the MAC array are turned off by configuring the gating clock. In addition, the (N-1)th MAC unit in each row is marked as the "end" MAC.

[0205] If M > L and M×N ≤ L×H, it means that the logical rows of the input matrix are larger than the physical rows of the MAC array, but the total size is smaller than the total resources of the hardware MAC array. Therefore, it occupies a total of M×N MAC resources.

[0206] By configuring the gated clock, the (M×N / H)-(L-1) row and (M×N%H)-(H-1) column of the MAC array are turned off (% is the remainder operation).

[0207] To fully utilize MAC resources and enable the computation of larger matrices, the input matrix for the next row continues from the "end" MAC cell of the current row. Therefore, for a row vector (N elements) with M cycles, the (N×i)th MAC cell of each row is marked as the "end" MAC, and the (N×i+1)th MAC cell of each row is marked as the "start" MAC (where i is an integer increasing sequentially from 1 to M).

[0208] If M×N>L×H, it means that the size of the input matrix is ​​greater than the total resources of the hardware MAC array. Therefore, iterative calculation is required: at the end of this calculation, the row vectors of M-(L×H) / N) are rearranged into the MAC array, and the result is concatenated with the previous result.

[0209] When N>H, the corresponding operations are similar to those in the case of N≤H, and will not be elaborated here.

[0210] When M×N>L×H, perform a loop calculation.

[0211] Optionally, the iterative calculation in step A13 includes: In the first round of calculation, the number of rows in the W weight matrix of the input matrix is ​​(H×L) / N; In the next round of calculation, if (M - ((H×L) / N)>(H×L)), then perform the same operation as in the first round; otherwise, end the loop calculation.

[0212] Optionally, when the size of the input matrix is ​​larger than the size of the MAC array (M×N>H×L), a loop calculation is performed, that is, the remaining logical rows in this round are calculated in the next round, and the calculation result is finally output. Since the size of the MAC array can be customized according to resources, it can be ensured that at least one row of the W weight matrix is ​​operated on.

[0213] Optionally, a cyclic computation matrix partitioning method uses logical rows as the basic elements for partitioning, ensuring maximum utilization of the MAC array in each round of computation. When the size of the input matrix is ​​larger than the size of the MAC array (M×N>H×L), the matrix is ​​partitioned according to logical rows to ensure maximum utilization of the MAC array and reduce the number of blocks. Therefore, compared with traditional matrix partitioning methods, this scheme significantly reduces the number of partitions and eliminates the need for accumulators in the computation. Figure 8 As shown.

[0214] The rule for matrix partitioning is that in the W matrix input in the first round, the number of rows in the matrix is ​​(H×L) / N. Before the second round of input, it is checked whether (M - ((H×L) / N)>(H×L)) is true. If it is true, the same operation as in the first round is performed, and this operation is continuously repeated until the comparison result is false, at which point the partitioning ends; otherwise, the normal calculation mode is entered.

[0215] In one feasible embodiment, the method performed by the electronic device further includes: Based on the configuration information, the X matrix of the input matrix is ​​mapped to the port of the MAC array using a broadcast method.

[0216] Optionally, the X data of the input matrix can be broadcast to the corresponding MAC unit. Specifically, the broadcast control logic maps the ports of the input matrix and the MAC array according to the configuration information, as described in the above embodiments regarding the data path of the X data.

[0217] Optionally, it may also include at least one of the following steps B31 to B32: Step B31: Map the data to be processed sequentially to the first sequence according to column order, map the first operation unit in the operation matrix sequentially to the second sequence, and input the data to be processed into the first operation unit through the first and second sequences.

[0218] Optionally, the X data corresponds to each logical column of the MAC array, therefore the ports of each SR in the X matrix are mapped to the sequence "TS" (sequentially increasing data ports) in column order, such as... Figure 18 The 46 TS ports are numbered from 0 to 45. The data ports on the MAC array side are mapped sequentially to the sequentially increasing port numbers "TD", such as... Figure 18 As shown, the MAC units used range from (0,0) to (5,14), and they are mapped to 92 TD ports numbered 0 to 91. TS and TD are concatenated according to the rule: TD%N=TS (where N is the dimension of the input matrix, and % is the modulo operation). Figure 18As shown, based on the value of TD, when TD is 0 and 46, its input port comes from TS 0. In this case, connect both of these ports to TS 0, and similarly connect the remaining ports.

[0219] This embodiment of the disclosure achieves the broadcasting of X data through this routing principle, which can improve the efficiency of broadcast path decision-making, greatly simplify the broadcast circuit design of X data, and save more hardware implementation resources.

[0220] Step B32: Starting from the first position of each operation row in the operation array, input the data to be processed into each first operation unit in sequence, column by column.

[0221] Optionally, X data can be broadcast based on the values ​​of registers "STA" and "MN" in the MAC unit, such as... Figure 19 As shown: Connect the SR port of the X data in the first column to the X data input port of all MAC units with "STA=1". Connect the SR port of the X data in the second column to the X data input port of the next MAC unit after the MAC unit with "STA=1". And so on, connecting the SR ports of all X data to the ports of the MAC units.

[0222] The embodiments of this disclosure achieve broadcasting of X data through this register positioning, which can greatly simplify the design of broadcasting circuits for X data.

[0223] In one feasible embodiment, the provided method further includes (which can be implemented by configuring the control logic unit): The first threshold sent by the CPU is configured by the CPU based on the current calculation result of the input data. Determining the configuration information of each first processing unit includes setting the values ​​of elements in the processed weight data that are less than the first threshold to zero.

[0224] Optionally, the first threshold is configured by the CPU and can be a dynamically adjusted parameter used to determine whether elements in the weight matrix need to be set to zero. If the value of an element in the weight matrix is ​​less than the first threshold, it is set to zero. It is understandable that in controlling the sparsity of the weight matrix, a higher first threshold results in more elements being set to zero, which can improve the sparsity of the weight matrix and reduce computational load.

[0225] Optionally, the content related to the dynamic adjustment of the first threshold can be referred to the relevant description of the sparsity of the W matrix in the above embodiments, and will not be repeated here in this disclosure.

[0226] In this embodiment, by dynamically adjusting the structure of the processing module to adapt to matrices of different sizes, preprocessing of the matrix can be eliminated. The processing module, located within the NPU, includes a control logic unit and a MAC array unit. The control logic unit configures the MAC unit's structure based on the size of the input matrix. The MAC array unit receives the configuration information, changes its connection structure, and performs multiplication and addition calculations. When the size of the input matrix changes with the AI ​​model, the processing module can adaptively change the structure of the MAC array, avoiding matrix partitioning and merging operations, improving computational efficiency, ensuring real-time operation, and saving hardware resources by eliminating the need for accumulators. Furthermore, the MAC units in the MAC array form cascaded data paths, and physical rows are also cascaded. This structure is not limited by the number of physical rows in the MAC array; each MAC unit can output the final calculation result, enabling computation of input matrices of arbitrary size.

[0227] The following application example further illustrates the embodiments of this disclosure.

[0228] When using a certain AI network to implement a certain function in communication, the input matrix for matrix operations is W(2×46)×X(46×1).

[0229] In related technologies, the matrix computation unit (16×16×16 MAC + 16×16 accumulator) of the NPU needs to first divide the two matrices into blocks and zero-padded them, and then perform calculations according to the matrix operation rules "W0×X0 + W1×X1 + W2×X2". Matrix multiplication requires one clock cycle to complete, but due to the block-based matrix operation, a pipelined operation cannot be formed, and the final output data cannot achieve a real-time, continuous effect. Figure 20 As shown.

[0230] In this embodiment, the matrix computation unit (10×10 MAC) concatenates the MACs into two logical rows according to the size of W. X data is broadcast to these two rows respectively. Parallel multiplication requires one clock cycle, followed by concatenated accumulation. The second X input also undergoes this operation, creating a pipelined effect and achieving real-time computation. Figure 21 As shown.

[0231] Furthermore, in terms of resource usage, in order to achieve the same computing effect, the hardware resources used in this embodiment are 10×10 MAC + 0 ACC, which is significantly more resource-efficient than the 16×16×16 MAC + 16×16 ACC used in traditional solutions.

[0232] This disclosure also provides an electronic device including a processor, and optionally, a transceiver and / or memory coupled to the processor, the processor being configured to perform the steps of the method provided in any optional embodiment of this disclosure.

[0233] Figure 23 The diagram shows a structural schematic of an electronic device to which an embodiment of the present invention applies, such as... Figure 23 As shown, Figure 23 The illustrated electronic device 4000 includes a processor 4001 and a memory 4003. The processor 4001 and memory 4003 are connected, for example, via a bus 4002. Optionally, the electronic device 4000 may further include a transceiver 4004, which can be configured for data interaction between the electronic device and other electronic devices, such as sending and / or receiving data. It should be noted that in practical applications, the transceiver 4004 is not limited to one, and the structure of the electronic device 4000 does not constitute a limitation on the embodiments of this disclosure. Optionally, the electronic device may be a node in a wireless communication system.

[0234] Processor 4001 may be a CPU (Central Processing Unit), a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. It may implement or execute the various exemplary logic blocks, modules, and circuits described in conjunction with this disclosure. Processor 4001 may also be a combination that implements computational functions, such as including one or more microprocessor combinations, a combination of a DSP and a microprocessor, etc.

[0235] Bus 4002 may include a pathway for transmitting information between the aforementioned components. Bus 4002 may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus, etc. Bus 4002 can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 23 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0236] The memory 4003 may be ROM (Read Only Memory) or other types of static storage devices capable of storing static information and instructions, RAM (Random Access Memory) or other types of dynamic storage devices capable of storing information and instructions, or EEPROM (Electrically Erasable Programmable Read Only Memory), CD-ROM (Compact Disc Read Only Memory) or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital universal optical discs, Blu-ray discs, etc.), magnetic disk storage media, other magnetic storage devices, or any other medium that can be configured to carry or store computer programs and can be read by a computer, without limitation herein.

[0237] The memory 4003 is configured to store a computer program that executes embodiments of the present disclosure, and its execution is controlled by the processor 4001. The processor 4001 is configured to execute the computer program stored in the memory 4003 to implement the steps shown in the foregoing method embodiments.

[0238] This disclosure provides a computer-readable storage medium storing a computer program, which, when executed by a processor, can implement the steps and corresponding content of the aforementioned method embodiments.

[0239] This disclosure also provides a computer program product, including a computer program that, when executed by a processor, can implement the steps and corresponding content of the aforementioned method embodiments.

[0240] The terms “first,” “second,” “third,” “fourth,” “1,” “2,” etc. (if present) in this disclosure, claims, and accompanying drawings are configured to distinguish similar objects and are not necessarily configured to describe a particular order or sequence. It should be understood that such data can be interchanged where appropriate so that embodiments of this disclosure described herein can be implemented in a sequence other than that shown in the figures or text.

[0241] It should be understood that although arrows indicate various operation steps in the flowcharts of the embodiments of this disclosure, the order in which these steps are implemented is not limited to the order indicated by the arrows. Unless explicitly stated herein, in some implementation scenarios of the embodiments of this disclosure, the implementation steps in each flowchart can be executed in other orders as required. Furthermore, some or all of the steps in each flowchart may include multiple sub-steps or multiple stages based on the actual implementation scenario. Some or all of these sub-steps or stages can be executed at the same time, and each sub-step or stage can also be executed at different times. In scenarios where execution times differ, the execution order of these sub-steps or stages can be flexibly configured as required, and the embodiments of this disclosure do not limit this.

[0242] The above text and accompanying drawings are provided as examples only to help the reader understand this disclosure. They are not intended and should not be construed as limiting the scope of this disclosure in any way. Although certain embodiments and examples have been provided, it will be apparent to those skilled in the art, based on the content disclosed herein, that changes can be made to the illustrated embodiments and examples, and other similar implementations based on the technical concept of this disclosure can be adopted without departing from the scope of this disclosure, and these modifications and modifications are also within the protection scope of the embodiments of this disclosure.

Claims

1. A method performed by an electronic device, comprising: Based on the size information of the input data, multiple first processing units and configuration information of each first processing unit are determined in the processing units included in the first processing module. Configure each first processing unit according to the corresponding configuration information; The input data is processed based on the arithmetic array composed of each first arithmetic unit; The size of the computing array corresponds to the size of the input data, and the configuration information includes information related to the position of the first computing unit in the computing array.

2. The method according to claim 1, wherein, Configure each first processing unit according to the corresponding configuration information, including: The configuration information of each first arithmetic unit is configured into the register of that first arithmetic unit.

3. The method according to claim 1, wherein, The location-related information includes: first indication information indicating at least the first operation unit is located at the beginning and / or end of the operation row in the operation array.

4. The method according to claim 3, wherein, The input data is processed accordingly, including: When the first instruction indicates that the first arithmetic unit is at the beginning of the arithmetic line, the first arithmetic unit outputs the result to the next connected first arithmetic unit; and / or When the first indication information indicates that the first arithmetic unit is at the end of the arithmetic line, the first arithmetic unit outputs the arithmetic result to the second processing module; and / or When the first instruction indicates that the first arithmetic unit is not located at the beginning or end of the arithmetic line, the first arithmetic unit outputs the arithmetic result to the next connected first arithmetic unit.

5. The method according to claim 4, wherein, The arithmetic units in the first processing module are cascaded sequentially, and the output port of each arithmetic unit is connected to the second processing module.

6. The method according to claim 4, wherein, The determination of the first indication information includes: The M×N input data and the L×H operation unit in the first processing module are respectively converted into one-dimensional row vectors, wherein M, N, L and H are not less than 1; The input data 1×N row vector is mapped to the row vector of the operation unit 1×(L×H) contained in the first processing module; The first indication information is determined based on the position of the corresponding arithmetic unit whose mapping length is a multiple of N.

7. The method according to claim 4, wherein, The determination of the first indication information includes: Based on the comparison between the size of the input data and the size of the computing unit contained in the first processing module, the first indication information is determined.

8. The method according to claim 7, wherein, The determination of the first indication information includes: When the number of rows and columns of the M×N input data is smaller than the number of rows and columns of the L×H operation units in the first processing module, the first indication information is determined based on the position of the (N-1)th operation unit corresponding to each row. When the logical row of the input data is larger than the physical row of the operation array, and the total size of the input data is smaller than the total size of the operation units contained in the first processing module, the first indication information is determined based on the position of the N×i-th ​​operation unit in each row, where i is an integer that increments sequentially from 1 to M. If the total size of the input data is greater than the total size of the computing units contained in the first processing module, a loop calculation is performed. At the end of the current calculation, the row vector of M-(L×H) / N is rearranged into the computing array, and the current calculation result is concatenated with the previous calculation result. The first indication information is determined based on the output position of each calculation result.

9. The method according to claim 8, wherein, The loop calculation includes: In the first round of calculation, the number of rows of weighted data processed in the input data is (H×L) / N; In the next round of calculation, if (M - ((H×L) / N) > (H×L)), then perform the same operation as in the first round of calculation; otherwise, end the loop calculation.

10. The method according to claim 1, wherein, The input data includes data to be processed and processing weight data; The configuration information also includes at least one of the following: A second indication information indicating whether the processing weight data corresponding to the first processing unit is zero; The processing weight data corresponding to the first processing unit; The location information of the first arithmetic unit in the first processing module.

11. The method according to claim 10, wherein, The location information includes at least one of the following: The first position information is used to indicate the position of the first operation unit after the operation of each operation row ends; The second position information is used to indicate the position of the first operation unit at the start of each operation row.

12. The method according to claim 10, wherein, The processing of the input data includes: When the second indication information indicates that the processing weight data corresponding to the first processing unit is zero, the first processing unit directly outputs the calculation result of the previous first processing unit to the next first processing unit or the second processing module.

13. The method according to claim 10, wherein, It also includes at least one of the following: The data to be processed is mapped sequentially to a first sequence in column order, and the first operation unit in the operation matrix is ​​mapped sequentially to a second sequence. The data to be processed is then input to the first operation unit through the first sequence and the second sequence. Starting from the first position of each operation row in the operation array, the data to be processed is sequentially input into each first operation unit, column by column.

14. The method of claim 10, wherein, Also includes: The CPU receives a first threshold value, which is configured by the CPU based on the current calculation result of the input data. Determine the configuration information of each first arithmetic unit, including: Set the values ​​of elements in the processed weight data that are less than the first threshold to zero.

15. The method according to claim 1, wherein, Also includes: Use a gated clock to shut down the arithmetic units in the first processing module that do not process the input data.

16. The method according to claim 1, wherein, The arithmetic unit is a multiply-accumulate (MAC) unit.

17. The method according to claim 1, wherein, The input data includes at least one of the following: communication data, image data, audio data, and text data; The communication data refers to the communication data received by the electronic device and to be pre-distorted, wherein the electronic device is a network node or a user equipment.

18. A processing module, comprising a control logic unit and multiple arithmetic units; The control logic unit is configured to determine multiple first arithmetic units and configuration information of each first arithmetic unit from the arithmetic units included in the first processing module based on the size information of the input data, and to configure each first arithmetic unit according to the corresponding configuration information. The size of the computing array composed of each first computing unit corresponds to the size of the input data, and the configuration information includes information related to the position of the first computing unit in the computing array; The processing unit is configured to process the input data.

19. The processing module according to claim 18, wherein, The arithmetic unit includes registers; The control logic unit is configured to configure the configuration information of each first arithmetic unit into the register of that first arithmetic unit; The arithmetic unit is configured to process the input data according to the configuration information in the register.

20. The processing module according to claim 18, wherein, The location-related information includes: first indication information indicating at least the first operation unit is located at the beginning and / or end of the operation row in the operation array.

21. The processing module according to claim 20, wherein, The arithmetic unit includes a first data path selector, which is configured to: When the first instruction indicates that the first arithmetic unit is at the beginning of the arithmetic line, the result of the arithmetic operation is output to the next connected first arithmetic unit; and / or When the first indication information indicates that the first arithmetic unit is at the end of the arithmetic line, the arithmetic result is output to the second processing module; and / or When the first instruction indicates that the first arithmetic unit is not located at the beginning or end of the arithmetic line, the arithmetic result is output to the next connected first arithmetic unit.

22. The processing module according to claim 21, wherein, The arithmetic units in the first processing module are cascaded sequentially, and the output port of each arithmetic unit is connected to the second processing module.

23. The processing module according to claim 21, wherein, The control logic unit is configured as follows: The M×N input data and the L×H operation unit in the first processing module are respectively converted into one-dimensional row vectors, wherein M, N, L and H are not less than 1; The input data 1×N row vector is mapped to the row vector of the operation unit 1×(L×H) contained in the first processing module; The first indication information is determined based on the position of the corresponding arithmetic unit whose mapping length is a multiple of N.

24. The processing module according to claim 21, wherein, The control logic unit is configured as follows: Based on the comparison between the size of the input data and the size of the computing unit contained in the first processing module, the first indication information is determined.

25. The processing module according to claim 24, wherein, The control logic unit is configured as follows: When the number of rows and columns of the M×N input data is smaller than the number of rows and columns of the L×H operation units in the first processing module, the first indication information is determined based on the position of the (N-1)th operation unit corresponding to each row. When the logical row of the input data is larger than the physical row of the operation array, and the total size of the input data is smaller than the total size of the operation units contained in the first processing module, the first indication information is determined based on the position of the N×i-th ​​operation unit in each row, where i is an integer that increments sequentially from 1 to M. If the total size of the input data is greater than the total size of the computing units contained in the first processing module, a loop calculation is performed. At the end of the current calculation, the row vector of M-(L×H) / N is rearranged into the computing array, and the current calculation result is concatenated with the previous calculation result. The first indication information is determined based on the output position of each calculation result.

26. The processing module according to claim 25, wherein, The loop calculation includes: In the first round of calculation, the number of rows of weighted data processed in the input data is (H×L) / N; In the next round of calculation, if (M - ((H×L) / N) > (H×L)), then perform the same operation as in the first round of calculation; otherwise, end the loop calculation.

27. The processing module according to claim 19, wherein, The input data includes data to be processed and processing weight data; The configuration information also includes at least one of the following, each configured in different registers: A second indication information indicating whether the processing weight data corresponding to the first processing unit is zero; The processing weight data corresponding to the first processing unit; The location information of the first arithmetic unit in the first processing module.

28. The processing module according to claim 27, wherein, The location information includes at least one of the following: The first position information is used to indicate the position of the first operation unit after the operation of each operation row ends; The second position information is used to indicate the position of the first operation unit at the start of each operation row.

29. The processing module according to claim 27, wherein, The processing unit includes a second data path selector, and the first data path selector is configured to: When the second indication information indicates that the processing weight data corresponding to the first processing unit is zero, the first processing unit directly outputs the calculation result of the previous first processing unit to the next first processing unit or the second processing module.

30. The processing module according to claim 27, wherein, The arithmetic unit is further configured to: The data to be processed is mapped sequentially to a first sequence in column order, and the first operation unit in the operation matrix is ​​mapped sequentially to a second sequence. The data to be processed is then input to the first operation unit through the first sequence and the second sequence. And / or, starting from the first position of each operation row in the operation array, the data to be processed is sequentially input into each first operation unit, column by column.

31. The processing module according to claim 27, wherein, The control logic unit is further configured to: The CPU receives a first threshold value, which is configured by the CPU based on the current calculation result of the input data. Set the values ​​of elements in the processed weight data that are less than the first threshold to zero.

32. The processing module according to claim 18, wherein, The control logic unit is further configured to: Use a gated clock to shut down the arithmetic units in the first processing module that do not process the input data.

33. The processing module according to claim 18, wherein, The arithmetic unit is a multiply-accumulate (MAC) unit.

34. The processing module according to claim 18, wherein, The input data includes at least one of the following: communication data, image data, audio data, and text data; The communication data refers to the communication data received by the electronic device and to be pre-distorted, wherein the electronic device is a network node or a user equipment.

35. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method of any one of claims 1 to 17.

36. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the method of any one of claims 1 to 17.

Images