Coprocessor and control method thereof, system-on-chip and control method thereof, storage medium and electronic equipment

Abstract

The invention discloses a coprocessor and a control method thereof, a system-on-chip and a control method thereof, a storage medium and electronic equipment, and the coprocessor comprises a control unit, a first arithmetic unit, a private memory unit, a direct access memory unit and a second-level cache interface, the control unit is used for configuring a calculation mode and a calculation process of the first operation unit based on an R-type extension instruction under the condition that the RISC-V processor executes a target operator in a neural network program; and the control unit is also used for reading parameter data of the target operator from the RISC-V processor based on the R-type extension instruction, storing the parameter data of the target operator and a calculation result in the private memory unit, and transferring the calculation result to the direct access memory unit, so that the RISC-V processor reads the calculation result. According to the coprocessor, the pause cost of an assembly line can be reduced, the cache hit rate is increased, and a higher energy efficiency ratio and reasoning speed are obtained in the neural network reasoning process.

Description

Technical Field

[0001] The present application relates to the field of processor technology, and in particular to a coprocessor, a coprocessor control method, a system on chip, a system on chip control method, a computer-readable storage medium, and an electronic device. Background Art

[0002] In today's artificial intelligence field, neural network reasoning is a crucial technology that plays a key role in various applications. Neural network reasoning refers to the process of using a trained neural network model to predict or classify new data. This process usually requires a large number of matrix operations, such as GEMM (General Matrix Multiplication) and convolution.

[0003] With the rapid development of deep learning technology, the complexity and scale of neural network models continue to increase, resulting in a sharp increase in computing requirements during neural network reasoning. Traditional general-purpose processors are often inefficient when processing these large-scale neural networks, and GPUs (Graphic Process Units) have high energy consumption. Whether it is cloud computing or edge computing, it is difficult to meet the stringent requirements for low power consumption and high energy efficiency. To solve this problem, researchers have proposed various optimization solutions, including specially designed neural network reasoning accelerators, customized processor architectures, and optimized computing methods. These technologies aim to improve the efficiency of neural network reasoning, reduce energy consumption, and accelerate processing speed to promote the widespread application of artificial intelligence technology in multiple fields.

[0004] In recent years, with the rise of RISC-V architecture and the application of custom instruction sets, more and more research has begun to explore ways to accelerate neural network reasoning on RISC-V processors. Without using dedicated parallel computing chips for neural network reasoning, adding hardware data-level parallel support to the processor requires the introduction of SIMD (Single Instruction Multiple Data) architecture: In 2015, RISC-V first introduced the "V" vector instruction set extension, which supports vectorized data operations and has been continuously improved in the following years. Subsequently, RISC-V processor design manufacturers have gradually implemented vector coprocessor architectures in their respective processor products.

[0005] However, on the one hand, a vector instruction requires a longer execution cycle than a scalar instruction. If a data risk pipeline is generated, there will be a longer pause cycle. At the same time, a large number of vector registers need to be configured to implement SIMD technology, but the high storage cost leads to diminishing returns on unit performance improvement. The current investment in further research on CPU architecture is not proportional to the progress achieved. On the other hand, neural network reasoning often involves the transmission of a large amount of data, including input / output feature mapping, network weights and bias parameters, and is a memory-intensive application. With the slowdown of Moore's Law, the gap between memory speed and processor speed is increasing, and model optimization measures such as sparsification and quantization at the software level are no longer sufficient to fully make up for this gap. Summary of the invention

[0006] The present application aims to solve one of the technical problems in the related art at least to a certain extent. To this end, the first purpose of the present application is to propose a coprocessor, a private memory unit is used to store the parameter data of the target operator in the neural network program, and the control unit is used to configure the calculation mode and calculation process of the first operation unit based on the R-type extension instruction when the RISC-V processor executes the target operator in the neural network program. The first operation unit is used to calculate the parameter data based on the calculation mode and the calculation process to obtain the calculation result, and the direct access memory unit is used to store the calculation result. The control unit is also used to read the parameter data of the target operator from the RISC-V processor based on the R-type extension instruction, and store the parameter data and calculation result of the target operator in the private memory unit, and transfer the calculation result to the direct access memory unit so that the RISC-V processor can read the calculation result. As a result, more calculation operations can be completed in fewer instruction cycles, the pipeline pause cost can be reduced, the cache hit rate can be improved, and higher energy efficiency and reasoning speed can be obtained in the process of neural network reasoning.

[0007] The second objective of the present application is to provide a control method for a coprocessor.

[0008] The third objective of the present application is to provide a system on chip.

[0009] The fourth objective of the present application is to provide a control method for a system on chip.

[0010] A fifth objective of the present application is to provide a computer-readable storage medium.

[0011] The sixth objective of the present application is to provide an electronic device.

[0012] To achieve the above-mentioned purpose, the first aspect of the present application proposes a coprocessor, comprising: a control unit, a first operation unit, a private memory unit, a direct access memory unit and a secondary cache interface, wherein the control unit is communicated with a RISC-V processor through the secondary cache interface, and the RISC-V processor is used to implement the RISC-V instruction set, and the RISC-V instruction set includes multiple R-type extension instructions, wherein the private memory unit is used to store parameter data of a target operator in a neural network program; the control unit is used to configure a calculation mode and a calculation process of the first operation unit based on the R-type extension instruction when the RISC-V processor executes the target operator in the neural network program; the first operation unit is used to calculate the parameter data based on the calculation mode and the calculation process to obtain a calculation result; the direct access memory unit is used to store the calculation result; the control unit is also used to read the parameter data of the target operator from the RISC-V processor based on the R-type extension instruction, and store the parameter data of the target operator and the calculation result in the private memory unit, and transfer the calculation result to the direct access memory unit so that the RISC-V processor can read the calculation result.

[0013] According to the coprocessor of the embodiment of the present application, the private memory unit is used to store the parameter data of the target operator in the neural network program, the control unit is used to configure the calculation mode and calculation process of the first operation unit based on the R-type extension instruction when the RISC-V processor executes the target operator in the neural network program, the first operation unit is used to calculate the parameter data based on the calculation mode and calculation process to obtain the calculation result, the direct access memory unit is used to store the calculation result, and the control unit is also used to read the parameter data of the target operator from the RISC-V processor based on the R-type extension instruction, and store the parameter data and calculation result of the target operator in the private memory unit, and transfer the calculation result to the direct access memory unit so that the RISC-V processor can read the calculation result. As a result, the coprocessor can complete more calculation operations in fewer instruction cycles, reduce the pipeline pause cost, improve the cache hit rate, and obtain higher energy efficiency and reasoning speed in the process of neural network reasoning.

[0014] In addition, the coprocessor according to the above embodiment of the present application may also have the following additional technical features:

[0015] According to one embodiment of the present application, the multiple R-type extension instructions include: a configuration calculation mode instruction, a load data instruction, a configuration data mode and start calculation instruction, and a read result instruction, wherein the control unit is configured to configure the calculation mode and calculation process of the first operation unit based on the configuration calculation mode instruction, and read the parameter data of the target operator based on the load data instruction, and transfer the calculation result to the direct access memory unit based on the read result instruction; the first operation unit is configured to calculate the parameter data based on the configuration data mode and start calculation instruction.

[0016] According to one embodiment of the present application, the coprocessor also includes: a second computing unit, used to implement the calculation of other operators except the target operator, and the other operators include one or more of activation functions, accelerated convolution operations, accumulation operations, and transposition operations.

[0017] According to one embodiment of the present application, the coprocessor also includes: a pipeline interface, the control unit is connected to the reorder cache unit of the RISC-V processor through the pipeline interface, and the reorder cache unit sends retired instructions and their decoding information to the control unit through the pipeline interface.

[0018] According to one embodiment of the present application, the first operation unit includes a multiplication-addition unit array, which is used to obtain the calculation result based on the target operator.

[0019] To achieve the above-mentioned purpose, the second aspect embodiment of the present application proposes a control method for a coprocessor, wherein the coprocessor includes: a control unit, a first operation unit, a private memory unit, a direct access memory unit and a secondary cache interface, wherein the control unit is communicated with a RISC-V processor through the secondary cache interface, and the RISC-V processor is used to implement the RISC-V instruction set, and the RISC-V instruction set includes multiple R-type extension instructions, wherein the method includes: when the RISC-V processor executes a target operator in a neural network program, controlling the control unit to configure a calculation mode and a calculation process of the first operation unit based on the R-type extension instruction, and reading parameter data of the target operator from the RISC-V processor and storing it in the private memory unit; controlling the first operation unit to calculate the parameter data based on the calculation mode and calculation process to obtain a calculation result; controlling the control unit to store the calculation result in the private memory unit, and transferring the calculation result to the direct access memory unit so that the RISC-V processor can read the calculation result.

[0020] According to the control method of the coprocessor of the embodiment of the present application, when the RISC-V processor executes the target operator in the neural network program, the control unit configures the calculation mode and calculation process of the first operation unit based on the R-type extension instruction, and reads the parameter data of the target operator from the RISC-V processor and stores it in the private memory unit; the first operation unit is controlled to calculate the parameter data based on the calculation mode and calculation process to obtain the calculation result; the control unit stores the calculation result in the private memory unit, and transfers the calculation result to the direct access memory unit so that the RISC-V processor can read the calculation result. As a result, the method can complete more calculation operations in fewer instruction cycles, reduce the pipeline pause cost, improve the cache hit rate, and obtain higher energy efficiency and reasoning speed in the process of neural network reasoning.

[0021] To achieve the above-mentioned objectives, the third aspect embodiment of the present application proposes a system on chip, which includes: a RISC-V processor and the above-mentioned coprocessor, the RISC-V processor is used to implement the RISC-V instruction set, and the RISC-V instruction set includes multiple R-type extension instructions, wherein the RISC-V processor is configured to load and execute a neural network program; the coprocessor is configured to, when the RISC-V processor executes a target operator in the neural network program, execute the R-type extension instruction and obtain a calculation result based on the extension instruction; the RISC-V processor is also configured to read the calculation result from the coprocessor.

[0022] According to the system on chip of the embodiment of the present application, the RISC-V processor is configured to load and execute the neural network program; the coprocessor is configured to execute the R-type extension instruction when the RISC-V processor executes the target operator in the neural network program, and obtain the calculation result based on the extension instruction; the RISC-V processor is also configured to read the calculation result from the coprocessor. As a result, the system on chip can complete more calculation operations in fewer instruction cycles, reduce the pipeline pause cost, improve the cache hit rate, and obtain higher energy efficiency and reasoning speed in the process of neural network reasoning.

[0023] To achieve the above-mentioned purpose, the fourth aspect embodiment of the present application proposes a control method for a system on chip, wherein the system on chip includes: a RISC-V processor and a coprocessor, wherein the RISC-V processor is used to implement the RISC-V instruction set, and the RISC-V instruction set includes multiple R-type extended instructions. The method includes: controlling the RISC-V processor to load and execute a neural network program; when the RISC-V processor executes a target operator in the neural network program, controlling the coprocessor to execute the R-type extended instruction and obtain a calculation result based on the extended instruction; controlling the RISC-V processor to read the calculation result from the coprocessor.

[0024] According to the control method of the system on chip of the embodiment of the present application, the RISC-V processor is controlled to load and execute the neural network program. When the RISC-V processor executes the target operator in the neural network program, the coprocessor is controlled to execute the R-type extension instruction, and the calculation result is obtained based on the extension instruction, and the RISC-V processor is controlled to read the calculation result from the coprocessor. As a result, the method can complete more calculation operations in fewer instruction cycles, reduce the pipeline pause cost, improve the cache hit rate, and obtain higher energy efficiency and reasoning speed in the process of neural network reasoning.

[0025] To achieve the above objectives, the fifth aspect of the present application proposes a computer-readable storage medium on which a program is stored, which, when executed by a processor, implements the above-mentioned system-on-chip control method or the above-mentioned coprocessor control method.

[0026] According to the computer-readable storage medium of the embodiment of the present application, by implementing the control method of the above-mentioned system on chip or the control method of the above-mentioned coprocessor during execution, more computing operations can be completed in fewer instruction cycles, the pipeline pause cost can be reduced, the cache hit rate can be improved, and higher energy efficiency and reasoning speed can be obtained during the neural network reasoning process.

[0027] In order to achieve the above-mentioned purpose, an electronic device proposed in the sixth aspect of the present application includes the above-mentioned system on chip or the above-mentioned coprocessor.

[0028] According to the electronic device of the embodiment of the present application, by including the above-mentioned system on chip or including the above-mentioned coprocessor, it is possible to complete more computing operations in fewer instruction cycles, reduce pipeline pause costs, improve cache hit rates, and obtain higher energy efficiency and reasoning speed during neural network reasoning.

[0029] Additional aspects and advantages of the present application will be given in part in the description below, and in part will become apparent from the description below, or will be learned through the practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 is a block diagram of a coprocessor according to an embodiment of the present application;

[0031] Figure 2 is a block diagram of a coprocessor according to a specific embodiment of the present application;

[0032] Figure 3 is a flowchart of a coprocessor control method according to an embodiment of the present application;

[0033] Figure 4 is a block diagram of a system on chip according to an embodiment of the present application;

[0034] Figure 5 is a schematic diagram of a system on chip according to a specific embodiment of the present application;

[0035] Figure 6 is a flow chart of a control method of a system on chip according to an embodiment of the present application;

[0036] Figure 7 is a block diagram of an electronic device according to an embodiment of the present application;

[0037] Figure 8 It is a block diagram of an electronic device according to another embodiment of the present application. DETAILED DESCRIPTION

[0038] Embodiments of the present application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present application, and should not be construed as limiting the present application.

[0039] The following describes a coprocessor, a control method for the coprocessor, a system on chip, a control method for the system on chip, a computer-readable storage medium, and an electronic device proposed in the embodiments of the present application with reference to the accompanying drawings.

[0040] Figure 1 is a block diagram of a coprocessor according to an embodiment of the present application.

[0041] like Figure 1 As shown, the coprocessor may include: a control unit 110 , a first operation unit 120 , a private memory unit 130 , a direct access memory unit 140 and a secondary cache interface 150 .

[0042] Among them, the control unit 110 is connected to the RISC-V processor 160 through the secondary cache interface 150, and the RISC-V processor 160 is used to implement the RISC-V instruction set, and the RISC-V instruction set includes multiple R-type extension instructions, wherein the private memory unit 130 is used to store the parameter data of the target operator in the neural network program. The control unit 110 is used to configure the calculation mode and calculation process of the first operation unit 120 based on the R-type extension instruction when the RISC-V processor 160 executes the target operator in the neural network program. The first operation unit 120 is used to calculate the parameter data based on the calculation mode and calculation process to obtain the calculation result. The direct access memory unit 140 is used to store the calculation result. The control unit 110 is also used to read the parameter data of the target operator from the RISC-V processor 160 based on the R-type extension instruction, and store the parameter data and calculation result of the target operator in the private memory unit 130, and transfer the calculation result to the direct access memory unit 140 so that the RISC-V processor 160 can read the calculation result.

[0043] Specifically, the control unit 110 can be connected to the RISC-V processor 160 through the secondary cache interface 150. The RISC-V processor 160 implements the RISC-V instruction set and may include multiple R-type extended instructions. Among them, the secondary cache interface 150 is used for the transmission of data and instructions to ensure efficient communication between the coprocessor 100 and the RISC-V processor 160. The secondary cache interface 150 implements two channels, a request channel and a response channel. The request channel includes: a valid-ready handshake signal for communication control, data transmission signals m_data1[63:0], m_data2[63:0], for transmitting custom instruction decoding information such as address, data block size, configuration information, and the coprocessor 100 operation code c_opcode[3:0], which is used to send to the internal state machine of the coprocessor 100 to achieve control and calculation. The response channel includes: a valid-ready handshake signal for communication control, and a data transmission signal s_data[63:0] for sending back the calculation results. Through this set of interfaces, tight coupling between the RISC-V CPU and the coprocessor 100 can be achieved, efficient communication can be realized, and bus resource usage can be reduced.

[0044] The private memory unit 130 is used to temporarily store the parameter data of the target operator in the neural network program. These parameter data are used by the first operation unit 120 during the calculation process. The parameter data of the target operator may include a convolution operator, a fully connected operator, an activation function operator, a batch normalization operator, etc. For example, the parameter data such as the convolution kernel weight of the convolution operator, the weight and bias of the fully connected layer are all stored in the private memory unit 130. The use of the private memory unit 130 can reduce the access to the main memory, increase the data access speed, and thus improve the overall computing efficiency. When the RISC-V processor 160 executes the target operator in the neural network program, the control unit 110 can configure the calculation mode and calculation process of the first operation unit 120 according to the R-type extension instruction. That is, after the control unit 110 receives the R-type extension instruction, it can parse the information such as the operation code (opcode), function code (funct3 and funct7), source register (rs1 and rs2) and target register (rd) in the instruction. According to the analysis result, the control unit 110 configures the computing resources inside the coprocessor 100, such as the size of the multiplication-addition unit array, the type of activation function, etc., and sets the state of the coprocessor 100 to prepare for subsequent computing tasks.

[0045] The first operation unit 120 can calculate the parameter data in the private memory unit 130 according to the calculation mode and calculation process configured by the control unit 110 to obtain a calculation result. For example, the first operation unit 120 includes an array of multiplication and addition units for efficiently performing calculation tasks such as matrix multiplication and convolution. These calculation tasks are key operations in neural network reasoning. The first operation unit 120 calculates the parameter data in the private memory unit 130 according to the configuration to obtain a calculation result. The direct access memory unit 140 is used to store the results calculated by the first operation unit 120. These results can be directly read by the RISC-V processor 160, thereby achieving efficient calculation and data transmission.

[0046] The control unit 110 also reads the parameter data of the target operator from the RISC-V processor 160 based on the R-type extension instruction, and stores the parameter data in the private memory unit 130. After the calculation is completed, the calculation result is transferred to the direct access memory unit 140 so that the RISC-V processor 160 can read the calculation result. That is, after the coprocessor 100 completes the calculation task, the control unit 110 will store the calculation result in the private memory unit 130. These calculation results include intermediate results and final results in neural network reasoning. For example, for a GEMM operation, after the multiplication and addition operation unit array of the coprocessor 100 completes the matrix multiplication, the control unit 110 will store the result in the private memory. In order to enable the RISC-V processor 160 to read these calculation results, the control unit 110 will also transfer the calculation results from the private memory unit 130 to the direct access memory unit 140 (such as L2 cache). This process is implemented by directly accessing the memory unit 140 without the direct intervention of the RISC-V processor 160, thereby reducing the burden on the CPU and improving the overall efficiency of the system.

[0047] As a result, more computing operations can be completed in fewer instruction cycles, reducing pipeline pause costs, improving cache hit rates, and achieving higher energy efficiency and inference speed during neural network inference.

[0048] According to one embodiment of the present application, multiple R-type extension instructions include: a configuration calculation mode instruction, a load data instruction, a configuration data mode and start calculation instruction, and a read result instruction, wherein the control unit 110 is configured to configure the calculation mode and calculation process of the first operation unit 120 based on the configuration calculation mode instruction, and read the parameter data of the target operator based on the load data instruction, and transfer the calculation result to the direct access memory unit 140 based on the read result instruction; the first operation unit 120 is configured to calculate the parameter data based on the configuration data mode and start calculation instruction.

[0049] Specifically, multiple R-type extended instructions may include: configure computing mode instructions, load data instructions, configure data mode and start computing instructions, and read result instructions. For example, the custom RISC-V extended instruction set designed by this application is as follows (operand 1 (x_rs1) and operand 2 (x_rs2) in the following description represent the values ​​in the registers corresponding to the rs1 and rs2 fields in the instruction). The calculation mode instruction is configured and represented by c_config. The 0-2 bits of operand 1 (x_rs1[2:0]) select the operation data type. 0x0 indicates fp8, 0x1 indicates fp16, 0x2 indicates int8, 0x3 indicates int16, and the rest are reserved. The 3-10 bits of operand 1 (x_rs1[10:3]) select the operation mode. 0x1 indicates matrix multiplication, 0x2 indicates img2col followed by matrix multiplication (i.e., convolution), 0x3 / 4 indicates matrix multiplication / convolution with ReLU activation, and 0x4 indicates matrix multiplication / convolution with Softmax activation. When the 11th bit of operand (x_rs1

[11] ) is 1, the row input matrix is ​​transposed. When the 12th bit of operand (x_rs1

[12] ) is 1, the column input matrix is ​​transposed. The other bits of operand 1 (x_rs1[63:13]) are reserved and can be extended with other functions. The instruction format is shown in Table 1:

[0050] Table 1

[0051]

[0052] The load data instruction is represented by c_load to load data from the private memory of the coprocessor 100 for use in calculating operators such as GEMM or convolution. Operand 1 (x_rs1) provides the address of the data in the global storage, the lower 32 bits of operand 2 (x_rs2[31:0]) provide the address of the data in the private memory of the coprocessor 100, and the upper 32 bits of operand 2 (x_rs2[63:32]) provide the size of the loaded data block. The instruction format is shown in Table 2:

[0053] Table 2

[0054]

[0055] Configure the data mode and start calculating instructions, which are represented by c_ex. That is, the lower 32 bits of operand 1 (x_rs1[31:0]) are calculated as the address of the matrix 1 data in the private memory of the coprocessor 100; the 32-47 bits of operand 1 (x_rs1[47:32]) are the number of rows of matrix 1; the 48-63 bits of operand 1 (x_rs1[63:48]) are the number of columns of matrix 1; the lower 32 bits of operand 2 (x_rs2[31:0]) are the address of the matrix 2 data in the private memory of the coprocessor 100; the 32-47 bits of operand 2 (x_rs2[47:32]) are the number of rows of matrix 2; the 48-63 bits of operand 2 (x_rs2[63:48]) are the number of columns of matrix 2. The instruction format is shown in Table 3:

[0056] Table 3

[0057]

[0058] The result read instruction is represented by c_store, that is, 2.c_store implements the CPU to read the data of the coprocessor 100 and obtain the calculated result. Operand 1 (x_rs1) provides the address of the data in the global storage, the lower 32 bits of operand 2 (x_rs2[31:0]) provide the address of the data in the private memory of the coprocessor 100, and the upper 32 bits of operand 2 (x_rs2[63:32]) provide the size of the loaded data block. Its instruction format is shown in Table 4:

[0059] Table 4

[0060]

[0061] In addition, the c_flush assembly instruction can be used to clear the internal pipeline and data of the coprocessor 100, so that the coprocessor 100 can be stopped and reset in time when a problem occurs in the calculation, thereby reducing unnecessary power consumption. The instruction format is shown in Table 5:

[0062] Table 5

[0063]

[0064] The c_reuse assembly instruction can be used to reuse the data of the last operation. That is, since the neural network reasoning will encounter the reuse of weights and inputs, the memory access frequency is reduced to reduce power consumption and increase speed. When rs1 and rs2 are the same as the objects of the last c_ex instruction, data reuse is performed. The instruction field is the same as c_ex. Its instruction format is shown in Table 6:

[0065] Table 6

[0066]

[0067] Thus, the control unit 110 can identify and parse the custom R-type extension instructions sent by the RISC-V processor 160. These instructions include the configuration calculation mode instruction c_config, the load data instruction c_load, the configuration data mode and start calculation instruction c_ex, and the read result instruction c_store. The control unit 110 can configure the calculation mode and calculation process of the first operation unit 120 based on the configuration calculation mode instruction. That is, when the RISC-V processor 160 executes the configuration calculation mode instruction (c_config), the control unit 110 configures the calculation mode and calculation process of the first operation unit 120 of the coprocessor 100 according to the parameters in the instruction. For example, configure the calculation mode of the multiplication and addition operation unit array and start the calculation task.

[0068] When the RISC-V processor 160 executes the load data instruction (c_load), the control unit 110 reads the parameter data of the target operator from the RISC-V processor 160. These parameter data are usually stored in the register or memory of the RISC-V processor 160. For example, the operand 1 (x_rs1) of the c_load instruction provides the address of the data in the global storage, the lower 32 bits of operand 2 (x_rs2[31:0]) provide the address of the data in the private memory of the coprocessor 100, and the upper 32 bits of operand 2 (x_rs2[63:32]) provide the size of the loaded data block. When the RISC-V processor 160 executes the read result instruction (c_store), the control unit 110 transfers the calculation result from the private memory unit 130 to the direct access memory unit 140 (such as L2 cache). This process is implemented through the direct memory access unit without the direct intervention of the RISC-V processor 160, thereby reducing the burden on the CPU and improving the overall efficiency of the system.

[0069] The first operation unit 120 is the core computing part of the coprocessor 100, responsible for performing specific computing tasks. When the RISC-V processor 160 executes the configuration data mode and starts calculating the instruction (c_ex), the control unit 110 configures the calculation mode of the first operation unit 120 and starts the calculation task. The first operation unit 120 calculates the parameter data according to the configured calculation mode and calculation process to obtain the calculation result. For example, for matrix multiplication, the first operation unit 120 performs multiplication and addition operations; for convolution operations, the first operation unit 120 performs matrix multiplication after img2col and processes according to the configured activation function (such as ReLU or Softmax).

[0070] Therefore, the control unit 110 and the first computing unit 120 play a key configuration and calculation role in the custom R-type extended instruction, ensuring the efficiency and accuracy of the neural network reasoning acceleration process.

[0071] According to one embodiment of the present application, Figure 2 The coprocessor 100 further includes: a second computing unit 170, which is used to implement the calculation of other operators except the target operator, and the other operators include one or more of activation functions, accelerated convolution operations, accumulation operations, and transposition operations.

[0072] Specifically, the coprocessor 100 may also include a second operation unit 170, which is used to implement the calculation of other operators except the target operator, and these other operators include one or more of activation functions, accelerated convolution operations, accumulation operations, and transposition operations. For example, the activation function is used in a neural network to map input to output, introduce nonlinear characteristics, and enable the network to learn complex patterns. Activation functions include: Sigmoid function, the output range is between 0 and 1, Tanh function, the output range is between -1 and 1, ReLU function: the output range is between 0 and positive infinity, the form is f(x) = max(0, x), the convergence speed is fast, and the accelerated convolution operation can be achieved by the following methods: img2col conversion, that is, converting the convolution operation into a matrix multiplication (GEMM) operation to improve resource reuse. The acceleration of the convolution operation can be achieved through a dedicated hardware unit (such as a multiplication and addition unit array) to reduce computing time and power consumption.

[0073] The accumulation operation is used in the neural network to add multiple input values. The implementation methods include: hardware accumulator, setting a special accumulator unit in the coprocessor 100 to quickly complete the accumulation operation. Software implementation, using loops or built-in functions (such as Python's sum() function) to perform accumulation operations. The transposition operation is used to swap the rows and columns of a matrix. Common implementation methods include: software implementation, using built-in functions in the programming language (such as Excel's TRANSPOSE function) to perform the transposition operation. Hardware implementation, setting a special transposition unit in the coprocessor 100 to quickly complete the matrix transposition operation.

[0074] These units are managed and scheduled by the control unit 110 and the state machine of the coprocessor 100 to ensure efficient data processing and calculation. Through these designs, the coprocessor 100 can efficiently process various complex operations in neural network reasoning and improve the performance and energy efficiency of the overall system.

[0075] According to one embodiment of the present application, Figure 2 As shown, the coprocessor 100 also includes: a pipeline interface 180, the control unit 110 is connected to the reorder cache unit 190 of the RISC-V processor 160 through the pipeline interface 180, and the reorder cache unit sends retired instructions and their decoding information to the control unit 110 through the pipeline interface 180.

[0076] Specifically, the coprocessor 100 also includes a pipeline interface 180, and the control unit 110 is connected to the reorder buffer unit 190 of the RISC-V processor 160 through the pipeline interface 180. The reorder buffer unit sends the retired instructions and their decoding information to the control unit 110 through the pipeline interface 180. That is, the reorder buffer unit (Re-Order Buffer, ROB) is an important component in the RISC-V processor 160 for supporting out-of-order execution. Its main functions include: recording instruction status: each instruction entering the pipeline is assigned a unique table entry in the ROB, recording the initial state of the instruction, destination register information, PC value, etc., instruction retirement: after the instruction is executed, the ROB is responsible for retiring the result of the instruction in program order and updating the architecture status. Exception handling: if an exception occurs during the execution of an instruction, the ROB can accurately handle the exception to ensure the consistency of the processor state.

[0077] The pipeline interface 180 is a communication bridge between the coprocessor 100 and the RISC-V processor 160. The pipeline interface 180 is responsible for transmitting the instructions and their decoding information in the RISC-V processor 160 to the control unit 110 of the coprocessor 100, and can ensure the correct transmission and synchronization of the instructions through handshake signals (such as the valid-ready mechanism). That is, when the instructions of the RISC-V processor 160 enter the pipeline, a table entry will be allocated in the ROB and the initial state of the instruction will be recorded. When the instruction is executed, the ROB sends the retired instruction and its decoding information to the control unit 110 of the coprocessor 100 through the pipeline interface 180. The control unit 110 receives the instruction and its decoding information, parses the opcode and operand of the instruction, and allocates the corresponding computing resources. The coprocessor 100 executes the instruction according to the scheduling of the control unit 110, such as loading data, performing multiplication and addition operations, storing results, etc. After the calculation result is ready, the result is moved from the private memory of the coprocessor 100 to the main memory through the c_store instruction for reading by the RISC-V processor 160.

[0078] Therefore, through this design, the coprocessor 100 can efficiently work with the RISC-V processor 160 to accelerate neural network reasoning and improve the performance and energy efficiency of the overall system.

[0079] According to one embodiment of the present application, the first operation unit 120 includes a multiplication-addition unit array for obtaining a calculation result based on a target operator.

[0080] Specifically, the first operation unit 120 includes a multiplication and addition unit array, which can obtain a calculation result according to the target operator. For example, the first operation unit 120 is composed of a 16*16 multiplication and addition unit array of 256 units, which can be used to directly complete the GEMM operation of a length not greater than 16 in neural network reasoning (larger matrix multiplications are optimized to matrix multiplication operators less than 16 through compilation). Each multiplication and addition unit can simultaneously complete multiplication and addition operations, that is, after multiplying two input data, they are added to the previously accumulated result to achieve multiplication and addition operations. The target operator usually specifies the type of operation and operation rules to be performed, such as matrix multiplication, convolution operation, etc. The input data is preprocessed and arranged according to the requirements of the target operator. For example, when performing matrix multiplication, the data of matrix A and matrix B are input into the multiplication and addition unit array in a certain order. Taking matrix multiplication as an example, assume that a 4×4 systolic array is used to calculate a 4×4 matrix multiplication. First, the data of matrix B is loaded into the register inside the multiplication and addition array, and the data of matrix A is sequentially input to the left side of the array. In each clock cycle, each multiplication and addition unit in the array receives data from the left and above, performs multiplication, adds the result to the previously accumulated result, and passes the new accumulated result to the multiplication and addition unit below. As the clock cycle progresses, data continues to flow and calculate in the array, and finally the calculation result is output at the bottom of the array. After a series of multiplication and addition operations, the final calculation result is obtained at the output of the multiplication and addition unit array. For matrix multiplication, each column of the array outputs a matrix multiplication result element, and the output of all columns combined is the final matrix multiplication result.

[0081] The size of the multiply-add unit array can be flexibly adjusted according to actual needs. By increasing the number of rows or columns of the array, the computing power can be improved to adapt to larger-scale computing tasks. In the fields of deep learning and image processing, a large number of matrix multiplication and convolution operations are required. The structural characteristics of the multiply-add unit array enable it to well meet the needs of these large-scale parallel computing and provide powerful computing support for related applications.

[0082] In summary, as a specific example, the control unit 110 is the core part of the coprocessor 100, which is responsible for parsing the custom R-type extension instructions and configuring the calculation mode and calculation process of the coprocessor 100 according to these instructions. The control unit 110 identifies and parses the custom R-type extension instructions sent by the RISC-V processor 160. These instructions include c_load, c_ex, c_store, etc., which are used to instruct the coprocessor 100 to perform specific computing tasks. For example, the c_load instruction is used to load data, the c_ex instruction is used to perform calculations, and the c_store instruction is used to store calculation results. When the RISC-V processor 160 executes the target operator in the neural network program, the control unit 110 reads the parameter data of the target operator from the RISC-V processor 160. These parameter data are usually stored in registers or memory of the RISC-V processor 160. For example, operand 1 (x_rs1) of the c_load instruction provides the address of the data in the global storage, the lower 32 bits of operand 2 (x_rs2[31:0]) provide the address of the data in the private memory of the coprocessor 100, and the upper 32 bits of operand 2 (x_rs2[63:32]) provide the size of the loaded data block. The control unit 110 configures the calculation mode and calculation flow of the first operation unit 120 of the coprocessor 100 according to the parsing result. For example, for GEMM operation, the control unit 110 configures the calculation mode of the multiplication and addition operation unit array and starts the calculation task.

[0083] The control unit 110 stores the parameter data of the read target operator in the private memory unit 130 of the coprocessor 100. The private memory unit 130 is a high-speed cache area inside the coprocessor 100, which is used to temporarily store the data required during the calculation process. After the calculation is completed, the control unit 110 stores the calculation results in the private memory unit 130. In order to enable the RISC-V processor 160 to read these calculation results, the control unit 110 transfers the calculation results from the private memory unit 130 to the direct access memory unit 140 (such as L2 cache). This process is implemented by directly accessing the memory unit 140 without the direct intervention of the RISC-V processor 160, thereby reducing the burden on the CPU and improving the overall efficiency of the system.

[0084] Thus, the control unit 110 realizes efficient data reading and storage through the management of the custom instruction set and the private memory unit 130. This not only reduces the burden of the RISC-V processor 160, but also improves the speed and efficiency of data access. The tight coupling design of the coprocessor 100 and the L2 cache makes data transmission more efficient. The control unit 110 directly accesses the L2 cache through the DMA module, reduces memory access latency and power consumption, and improves the overall energy efficiency of the system. The custom R-type extended instruction set is concise and powerful, and can flexibly support various neural network reasoning tasks. The control unit 110 can flexibly configure the computing mode of the coprocessor 100 according to different instruction types and parameters to achieve efficient computing acceleration. Through the custom instruction set, the control unit 110 can reduce the cost of pipeline pauses. For example, the c_ex instruction can submit a computing task to the coprocessor 100, and the RISC-V processor 160 can continue to execute subsequent tasks, thereby improving the system processing speed.

[0085] In summary, according to the coprocessor of the embodiment of the present application, the private memory unit is used to store the parameter data of the target operator in the neural network program, the control unit is used to configure the calculation mode and calculation process of the first operation unit based on the R-type extension instruction when the RISC-V processor executes the target operator in the neural network program, the first operation unit is used to calculate the parameter data based on the calculation mode and calculation process to obtain the calculation result, the direct access memory unit is used to store the calculation result, and the control unit is also used to read the parameter data of the target operator from the RISC-V processor based on the R-type extension instruction, and store the parameter data and calculation result of the target operator in the private memory unit, and transfer the calculation result to the direct access memory unit so that the RISC-V processor can read the calculation result. As a result, the coprocessor can complete more calculation operations in fewer instruction cycles, reduce the pipeline pause cost, improve the cache hit rate, and obtain higher energy efficiency and reasoning speed in the process of neural network reasoning.

[0086] Corresponding to the above embodiment, the present application also proposes a control method for a coprocessor, wherein: Figure 1 The coprocessor 100 shown includes: a control unit 110, a first operation unit 120, a private memory unit 130, a direct access memory unit 140 and a secondary cache interface 150. The control unit 110 is communicatively connected to the RISC-V processor 160 through the secondary cache interface 150. The RISC-V processor 160 is used to implement the RISC-V instruction set, and the RISC-V instruction set includes multiple R-type extended instructions.

[0087] like Figure 3 As shown, in one embodiment, the control method of the coprocessor includes the following steps:

[0088] S1, when the RISC-V processor executes the target operator in the neural network program, the control unit configures the calculation mode and calculation process of the first operation unit based on the R-type extension instruction, and reads the parameter data of the target operator from the RISC-V processor and stores it in the private memory unit.

[0089] S2, controlling the first computing unit to calculate the parameter data based on the calculation mode and the calculation process to obtain a calculation result.

[0090] S3, controls the control unit to store the calculation result in the private memory unit, and transfers the calculation result to the direct access memory unit so that the RISC-V processor can read the calculation result.

[0091] It should be noted that for details not disclosed in the control method of the coprocessor in the embodiment of the present application, please refer to the details disclosed in the coprocessor in the embodiment of the present application, and the details will not be repeated here.

[0092] In summary, according to the control method of the coprocessor of the embodiment of the present application, when the RISC-V processor executes the target operator in the neural network program, the control unit configures the calculation mode and calculation process of the first operation unit based on the R-type extension instruction, and reads the parameter data of the target operator from the RISC-V processor and stores it in the private memory unit; the first operation unit is controlled to calculate the parameter data based on the calculation mode and calculation process to obtain the calculation result; the control unit stores the calculation result in the private memory unit, and transfers the calculation result to the direct access memory unit so that the RISC-V processor can read the calculation result. As a result, the method can complete more calculation operations in fewer instruction cycles, reduce the pipeline pause cost, improve the cache hit rate, and obtain higher energy efficiency and reasoning speed in the process of neural network reasoning.

[0093] Corresponding to the above embodiments, the present application also proposes a system on chip.

[0094] Figure 4 is a block diagram of a coprocessor according to an embodiment of the present application.

[0095] like Figure 4 As shown, the system on chip 200 may include: a RISC-V processor 160 and the above-mentioned coprocessor 100.

[0096] Among them, the RISC-V processor 160 is used to implement the RISC-V instruction set, which includes multiple R-type extended instructions. Among them, the RISC-V processor 160 is configured to load and execute the neural network program; the coprocessor 100 is configured to execute the R-type extended instructions when the RISC-V processor 160 executes the target operator in the neural network program, and obtain the calculation result based on the extended instruction; the RISC-V processor 160 is also configured to read the calculation result from the coprocessor 100.

[0097] Specifically, unlike traditional commercial processors and coprocessors, after the neural network inference program starts executing, operations such as GEMM and convolution in the system on chip 200 are not included in the CPU pipeline, but are submitted to the coprocessor 100 for processing through micro-architecture decoding of custom instructions. After submission, execution is handed over to the coprocessor, while the CPU can continue to execute subsequent tasks, thereby improving processing speed and reducing pipeline pause costs.

[0098] RISC-V processor 160 is one of the core components of system on chip 200 (SoC), responsible for executing RISC-V instruction set. RISC-V instruction set is an open source instruction set architecture with high scalability and flexibility, suitable for a variety of application scenarios, specifically, RISC-V processor 160 is configured to load and execute neural network program, that is, RISC-V processor 160 is responsible for loading neural network program and executing it in order of program instructions. These programs usually contain computing tasks of multiple neural network layers, such as convolution layer, fully connected layer, activation function, etc. For example, RISC-V processor 160 can load a pre-trained convolutional neural network model and start to execute the computing tasks of each layer in the model. And when RISC-V processor 160 executes the target operator in the neural network program, it will encounter a custom R-type extension instruction. These instructions are used to configure the computing mode and data processing mode of coprocessor 100. For example, c_config instruction is used to configure the computing mode, c_load instruction is used to load data, c_ex instruction is used to configure the data mode and start computing, and c_store instruction is used to store the calculation results. After executing the custom instructions, the RISC-V processor 160 also reads the calculation results from the coprocessor 100. These results are stored in the private memory unit 130 of the coprocessor 100, and transferred to the secondary cache interface 150 (L2 cache) through the direct memory access unit for the RISC-V processor 160 to read.

[0099] The coprocessor 100 is an auxiliary computing unit in the system on chip 200, which is used to accelerate the computing tasks in the neural network reasoning. The coprocessor 100 is configured to execute the R-type extension instruction when the RISC-V processor 160 executes the target operator in the neural network program, and obtain the calculation result based on the extension instruction. That is, when the RISC-V processor 160 executes the target operator in the neural network program, the custom R-type extension instruction is sent to the coprocessor 100. After receiving the instruction, the coprocessor 100 configures its own calculation mode and data processing method according to the parameters in the instruction. The coprocessor 100 calculates the parameter data according to the configured calculation mode and data processing method to obtain the calculation result. For example, for matrix multiplication, the coprocessor 100 performs multiplication and addition operations; for convolution operations, the coprocessor 100 performs matrix multiplication after img2col and processes according to the configured activation function (such as ReLU or Softmax). After the calculation is completed, the coprocessor 100 transfers the calculation result from the private memory unit 130 to the direct access memory unit 140 (such as L2 cache). This process is implemented through a direct memory access unit without the need for direct intervention by the RISC-V processor 160, thereby reducing the burden on the CPU and improving the overall efficiency of the system.

[0100] Thus, the RISC-V processor 160 and the coprocessor 100 work together in the system on chip to ensure the efficiency and accuracy of the neural network reasoning acceleration process.

[0101] In addition, refer to Figure 5 As shown, in the SoC (System on Chip), a RISC-V processor 160 (RV64GC processor core) that implements the RV64GC instruction set is included as the main control of the system, which includes a 16KB L1 instruction cache and a 16KB L1 data cache, which is then connected to a 512KB L2 cache. Unlike the traditional coprocessor 100 that moves data from DDR, the coprocessor 100 in this system is tightly coupled with the L2 cache. On the one hand, through the direct memory access unit and the private bus interface, the coprocessor 100 can directly access the L2 cache to obtain data, reducing the bus occupancy and the number of load / store (data loading and saving) instructions executed by the CPU, and alleviating the pressure caused by intensive data transmission; on the other hand, compared with accessing DDR, the coprocessor 100 accessing the cache can effectively reduce the memory access cycle and energy consumption, and reduce the time and energy consumption cost of data movement. In order to maintain data consistency, the private bus also supports the MESI protocol to prevent data errors and ensure correctness. In addition, some peripherals are mounted on the interconnect bus for communication with the outside world.

[0102] In addition, in the RISC-V processor 160, a Regfile is also included for storing and managing the registers of the processor. In the RISC-V processor 160, Regfile usually contains multiple general purpose registers (General Purpose Registers, GPRs) and floating point registers (Floating Point Registers, FPRs). These registers are used to store information such as intermediate calculation results, addresses, constants, etc., so that the processor can quickly access and operate. It also includes an FPU (Floating Point Unit). In the RISC-V processor 160, the FPU supports single-precision (32-bit) and double-precision (64-bit) floating-point operations, including addition, subtraction, multiplication, division, square root and other operations. And includes an ALU (Arithmetic Logic Unit) for performing integer and logical operations. In the RV64GC processor, the ALU supports a variety of integer operations, including addition, subtraction, multiplication, division, bit operations (such as AND, OR, NOT, XOR), etc. These components work together in the RISC-V processor 160, providing efficient computing power and flexible data management capabilities, supporting the high performance of the processor in various application scenarios.

[0103] In addition, the system-on-chip also includes a UART (Universal Asynchronous Receiver-Transmitter) serial communication interface for transmitting data between devices. UART supports asynchronous communication, that is, the sending and receiving devices do not need to share a clock signal. UART is usually used for low-speed communication, such as connection with serial ports, modems, GPS modules and other devices. In the system-on-chip, the UART module is usually integrated inside the MCU and connected to the CPU core through an internal bus. It includes GPIO (General Purpose Input / Output), which is a pin on the MCU that can be used as input or output according to the configuration. Each GPIO pin can be configured independently. It also includes JTAG (Joint Test Action Group), an interface standard for chip testing and debugging. The JTAG interface allows external debugging tools to communicate with the chip through a set of standard pins to implement read and write operations on the chip's internal registers and memory. JTAG is widely used in the debugging and testing of embedded systems and system-on-chips. It includes DDR (Double Data Rate Synchronous Dynamic Random-Access Memory), a high-speed dynamic random access memory used to store large amounts of data. DDR memory achieves double the data transfer rate by transmitting data on the rising and falling edges of the clock signal. DDR memory is usually used as the main memory of the system on chip and supports high-speed data read and write operations. Common DDR types include DDR2, DDR3, DDR4, etc. Including I2C (Inter-Integrated Circuit), a synchronous serial communication protocol used to connect microcontrollers and their peripherals. The I2C bus uses two signal lines (SDA and SCL) for communication and supports multiple masters and multiple slaves. The I2C bus has the advantages of few interface lines, simple control method, small device package form, and high communication rate. It is widely used in systems on chip to connect slave devices such as EEPROM, real-time clock, and A / D converter. These components work together in the system on chip to provide rich functions and efficient communication capabilities.

[0104] In summary, according to the system on chip of the embodiment of the present application, the RISC-V processor is configured to load and execute the neural network program; the coprocessor is configured to execute the R-type extension instruction when the RISC-V processor executes the target operator in the neural network program, and obtain the calculation result based on the extension instruction; the RISC-V processor is also configured to read the calculation result from the coprocessor. As a result, the system on chip can complete more computing operations in fewer instruction cycles, reduce the pipeline pause cost, improve the cache hit rate, and obtain higher energy efficiency and reasoning speed in the process of neural network reasoning.

[0105] Corresponding to the above embodiment, the present application also proposes a control method of a system on chip, wherein: Figure 4 As shown, the system on chip 200 includes: a RISC-V processor 160 and a coprocessor 100, the RISC-V processor 160 is used to implement the RISC-V instruction set, and the RISC-V instruction set includes multiple R-type extended instructions.

[0106] like Figure 6 As shown, in one embodiment, the control method of the system on chip includes the following steps:

[0107] S11, controlling the RISC-V processor to load and execute the neural network program.

[0108] S12, when the RISC-V processor executes the target operator in the neural network program, controls the coprocessor to execute the R-type extended instruction, and obtains a calculation result based on the extended instruction.

[0109] S13, control the RISC-V processor to read the calculation result from the coprocessor.

[0110] It should be noted that for details not disclosed in the control method of the on-chip system in the embodiment of the present application, please refer to the details disclosed in the on-chip system in the embodiment of the present application, and the details will not be repeated here.

[0111] In summary, according to the control method of the system on chip of the embodiment of the present application, the RISC-V processor is controlled to load and execute the neural network program. When the RISC-V processor executes the target operator in the neural network program, the coprocessor is controlled to execute the R-type extension instruction, and the calculation result is obtained based on the extension instruction, and the RISC-V processor is controlled to read the calculation result from the coprocessor. As a result, the method can complete more calculation operations in fewer instruction cycles, reduce the pipeline pause cost, improve the cache hit rate, and obtain higher energy efficiency and reasoning speed in the process of neural network reasoning.

[0112] Corresponding to the above embodiments, the present application also proposes a computer-readable storage medium.

[0113] The computer-readable storage medium of the embodiment of the present application stores a program thereon, and when the program is executed by a processor, the program implements the control method of the above-mentioned system on chip or the control method of the above-mentioned coprocessor.

[0114] According to the computer-readable storage medium of the embodiment of the present application, by executing the control method of the above-mentioned system on chip or executing the control method of the above-mentioned coprocessor, more computing operations can be completed in fewer instruction cycles, the pipeline pause cost can be reduced, the cache hit rate can be improved, and higher energy efficiency and reasoning speed can be obtained during the neural network reasoning process.

[0115] Corresponding to the above embodiment, the present application also proposes an electronic device.

[0116] like Figure 7 As shown, the electronic device 300 of the embodiment of the present application may include a system on chip 200, or as shown in FIG. Figure 8 As shown, the electronic device 300 of the embodiment of the present application may include a coprocessor 100 .

[0117] According to the electronic device of the embodiment of the present application, by including the above-mentioned system on chip or including the above-mentioned coprocessor, it is possible to complete more computing operations in fewer instruction cycles, reduce pipeline pause costs, improve cache hit rates, and obtain higher energy efficiency and reasoning speed during neural network reasoning.

[0118] It should be noted that the logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be specifically implemented in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute instructions), or in combination with these instruction execution systems, devices or apparatuses. For the purposes of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in combination with these instruction execution systems, devices or apparatuses. More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection portion with one or more wirings (electronic device), a portable computer disk box (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium and then editing, interpreting or processing in other suitable ways if necessary, and then stored in a computer memory.

[0119] It should be understood that the various parts of the present application can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, multiple steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0120] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.

[0121] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In the description of this application, the meaning of "plurality" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

[0122] In this application, unless otherwise clearly specified and limited, the terms "installed", "connected", "connected", "fixed" and the like should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium, it can be the internal connection of two elements or the interaction relationship between two elements, unless otherwise clearly defined. For ordinary technicians in this field, the specific meanings of the above terms in this application can be understood according to specific circumstances.

[0123] Although the embodiments of the present application have been shown and described above, it can be understood that the above embodiments are exemplary and cannot be understood as limitations on the present application. Ordinary technicians in this field can change, modify, replace and modify the above embodiments within the scope of the present application.

Claims

1. A coprocessor, characterized in that: include: A control unit, a first operation unit, a private memory unit, a direct access memory unit and a secondary cache interface, wherein the control unit is communicatively connected to a RISC-V processor through the secondary cache interface, and the RISC-V processor is used to implement a RISC-V instruction set, wherein the RISC-V instruction set includes a plurality of R-type extended instructions, wherein: The private memory unit is used to store parameter data of the target operator in the neural network program; The control unit is used to configure the calculation mode and calculation process of the first operation unit based on the R-type extended instruction when the RISC-V processor executes the target operator in the neural network program; The first computing unit is used to calculate the parameter data based on the calculation mode and calculation process to obtain a calculation result; The direct access memory unit is used to store the calculation result; The control unit is also used to, based on the R-type extended instruction, read the parameter data of the target operator from the RISC-V processor, store the parameter data of the target operator and the calculation result in the private memory unit, and transfer the calculation result to the direct access memory unit so that the RISC-V processor can read the calculation result.

2. The coprocessor according to claim 1, characterized in that: The plurality of R-type extension instructions include: a configuration calculation mode instruction, a load data instruction, a configuration data mode and start calculation instruction, and a read result instruction, wherein: The control unit is configured to configure the calculation mode and calculation process of the first operation unit based on the configuration calculation mode instruction, read the parameter data of the target operator based on the load data instruction, and transfer the calculation result to the direct access memory unit based on the read result instruction; The first operation unit is configured to calculate the parameter data based on the configuration data mode and start calculation instructions.

3. The coprocessor according to claim 2, characterized in that: The coprocessor also includes: The second operation unit is used to implement the calculation of other operators except the target operator, and the other operators include one or more of activation functions, accelerated convolution operations, accumulation operations, and transposition operations.

4. The coprocessor according to claim 2, characterized in that: The coprocessor also includes: a pipeline interface, the control unit is connected to the reorder cache unit of the RISC-V processor through the pipeline interface, and the reorder cache unit sends retired instructions and their decoding information to the control unit through the pipeline interface.

5. The coprocessor according to claim 1, characterized in that: The first operation unit includes a multiplication-addition unit array, which is used to obtain the calculation result based on the target operator.

6. A method for controlling a coprocessor, characterized in that: The coprocessor includes: a control unit, a first operation unit, a private memory unit, a direct access memory unit and a secondary cache interface, the control unit is communicatively connected with a RISC-V processor through the secondary cache interface, the RISC-V processor is used to implement a RISC-V instruction set, and the RISC-V instruction set includes a plurality of R-type extended instructions, wherein the method includes: When the RISC-V processor executes the target operator in the neural network program, control the control unit to configure the calculation mode and calculation process of the first operation unit based on the R-type extension instruction, and read the parameter data of the target operator from the RISC-V processor and store it in the private memory unit; Controlling the first computing unit to calculate the parameter data based on the calculation mode and calculation process to obtain a calculation result; Control the control unit to store the calculation result in the private memory unit, and transfer the calculation result to the direct access memory unit so that the RISC-V processor can read the calculation result.

7. A system on chip, characterized in that: The system on chip comprises: a RISC-V processor and a coprocessor as claimed in any one of claims 1 to 5, wherein the RISC-V processor is used to implement a RISC-V instruction set, wherein the RISC-V instruction set comprises a plurality of R-type extended instructions, wherein: The RISC-V processor is configured to load and execute a neural network program; The coprocessor is configured to, when the RISC-V processor executes a target operator in a neural network program, execute the R-type extension instruction and obtain a calculation result based on the extension instruction; The RISC-V processor is also configured to read the calculation result from the coprocessor.

8. A control method of a system on chip, characterized in that: The system on chip includes: a RISC-V processor and a coprocessor, the RISC-V processor is used to implement a RISC-V instruction set, the RISC-V instruction set includes a plurality of R-type extended instructions, and the method includes: Controlling the RISC-V processor to load and execute a neural network program; When the RISC-V processor executes the target operator in the neural network program, control the coprocessor to execute the R-type extended instruction, and obtain a calculation result based on the extended instruction; Control the RISC-V processor to read the calculation result from the coprocessor.

9. A computer-readable storage medium, characterized in that: A program is stored thereon, and when the program is executed by a processor, the control method of the system on chip according to claim 8 or the control method of the coprocessor according to claim 6 is implemented.

10. An electronic device, characterized in that: The system on chip comprises the system as claimed in claim 7, or comprises the coprocessor as claimed in any one of claims 1-5.

Images