Method, apparatus, device, medium and product based on running of artificial intelligence model

Abstract

The application discloses an operation method and device based on an artificial intelligence model, equipment, a medium and a product, and relates to the technical field of computers.The method comprises the following steps: acquiring source data of target operation; wherein the source data comprises a first matrix and a second matrix; judging whether the target operation is block calculation; if yes, determining a first block size of the first matrix and a second block size of the second matrix; storing the first matrix as a plurality of first block matrices according to the first block size, and storing the second matrix as a plurality of second block matrices according to the second block size; and in the process of executing the target operation, the plurality of first block matrices and the plurality of second block matrices are read in sequence to realize the target operation and obtain a target operation result.The application improves operation efficiency and saves hardware resources.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to an operation method, device, equipment, medium and product based on an artificial intelligence model. Background Art

[0002] In the field of artificial intelligence, matrix operations play a central role, especially in the inference and training of large models. As model size continues to grow, traditional matrix operations face numerous challenges. Large-scale matrix operations require processing massive amounts of data, placing high demands on hardware resources and resulting in low computational efficiency.

[0003] Therefore, how to improve the computational efficiency of artificial intelligence models is a technical problem that needs to be solved by those skilled in the art. Summary of the Invention

[0004] The present invention provides an operation method, device, electronic device, storage medium and computer program product based on an artificial intelligence model, which improves the computing efficiency of the artificial intelligence model and saves hardware resources.

[0005] The present invention provides an operation method based on an artificial intelligence model, comprising:

[0006] Obtaining source data of a target operation; wherein the source data includes a first matrix and a second matrix;

[0007] Determine whether the target operation is a block calculation; if so, determine a first block size of the first matrix and a second block size of the second matrix;

[0008] Divide the first matrix into blocks and store them into a plurality of first block matrices according to a first block size, and divide the second matrix into blocks and store them into a plurality of second block matrices according to a second block size;

[0009] During the process of executing the target operation, multiple first block matrices and multiple second block matrices are read in sequence to implement the target operation and obtain the target operation result.

[0010] The present invention also provides an operating device based on an artificial intelligence model, comprising:

[0011] An acquisition module, configured to acquire source data for a target operation; wherein the source data includes a first matrix and a second matrix;

[0012] a determination module, configured to determine whether the target operation is a block calculation; if so, determining a first block size of the first matrix and a second block size of the second matrix;

[0013] A block storage module is used to store the first matrix into multiple first block matrices according to a first block scale, and to store the second matrix into multiple second block matrices according to a second block scale;

[0014] The operation module is used to read multiple first block matrices and multiple second block matrices in sequence during the execution of the target operation to achieve the target operation and obtain the target operation result.

[0015] The present invention also provides an electronic device, comprising: a memory for storing a computer program; and a processor for implementing the steps of any of the above-mentioned operating methods based on an artificial intelligence model when executing the computer program.

[0016] The present invention also provides a computer-readable storage medium, in which a computer program is stored, wherein when the computer program is executed by a processor, the steps of any of the above-mentioned operating methods based on the artificial intelligence model are implemented.

[0017] The present invention also provides a computer program product, comprising a computer program, which, when executed by a processor, implements the steps of any of the above-mentioned operating methods based on the artificial intelligence model.

[0018] The beneficial effects of the present invention are as follows: the operation method based on the artificial intelligence model provided by the present invention stores the first matrix and the second matrix in blocks before executing the target operation, splitting the large matrix into multiple small matrices. This block storage strategy makes data reading more efficient during the calculation process, reduces address jump calculations, shortens the delay in obtaining data, thereby improving calculation efficiency and saving hardware resources. The present invention also discloses an operation device based on the artificial intelligence model, an electronic device, a computer-readable storage medium, and a computer program product, which can also achieve the above-mentioned technical effects.

[0019] It should be understood that the foregoing general description and the following detailed description are exemplary only and are not restrictive of the invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the embodiments of the present invention, the following is a brief introduction to the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0021] Figure 1 The present invention is a flowchart of an operation method based on an artificial intelligence model according to an exemplary embodiment.

[0022] Figure 2 The figure is a schematic diagram of block storage of a weight matrix according to an exemplary embodiment.

[0023] Figure 3The figure is a schematic diagram showing a direct memory read instruction according to an exemplary embodiment.

[0024] Figure 4 The figure is a schematic diagram showing an operation instruction according to an exemplary embodiment.

[0025] Figure 5 The figure is a schematic diagram showing a configuration register instruction according to an exemplary embodiment.

[0026] Figure 6 The figure is a schematic diagram showing a loop start instruction according to an exemplary embodiment.

[0027] Figure 7 The figure is a schematic diagram showing a loop end instruction according to an exemplary embodiment.

[0028] Figure 8 The figure is a schematic diagram showing a general register calculation instruction according to an exemplary embodiment.

[0029] Figure 9 The figure is a schematic diagram showing a general register condition judgment instruction according to an exemplary embodiment.

[0030] Figure 10 The present invention is a flowchart of another operation method based on an artificial intelligence model according to an exemplary embodiment.

[0031] Figure 11 The figure is a schematic diagram showing a first matrix partitioning method according to an exemplary embodiment.

[0032] Figure 12 is a schematic diagram showing a second matrix partitioning method according to an exemplary embodiment.

[0033] Figure 13 The figure is a structural diagram of an operating device based on an artificial intelligence model according to an exemplary embodiment.

[0034] Figure 14 The figure is a structural diagram of an electronic device according to an exemplary embodiment. DETAILED DESCRIPTION

[0035] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making any creative efforts shall fall within the scope of protection of the present invention.

[0036] It should be noted that, in the description of the present invention, the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, article, or apparatus comprising a series of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or apparatus. The terms "first," "second," etc., in the present invention are used to distinguish similar objects, and are not used to describe a particular order or precedence.

[0037] In order to enable those skilled in the art to better understand the solutions of the present invention, the present invention is further described in detail below with reference to the accompanying drawings and specific implementation methods.

[0038] An embodiment of the present invention provides an operation method based on an artificial intelligence model, and the method is described in detail in conjunction with the execution process of the operation method based on the artificial intelligence model.

[0039] See also Figure 1 , according to an exemplary embodiment, a flowchart of an operation method based on an artificial intelligence model is shown, such as Figure 1 Shown, including:

[0040] S101: Acquire source data of a target operation; wherein the source data includes a first matrix and a second matrix.

[0041] In practice, before performing a matrix operation, it is necessary to first obtain the source data required for the target operation. This source data typically consists of two matrices: a first matrix and a second matrix. These two matrices are the basic inputs for matrix operations. For example, during the inference process of an artificial intelligence model, the first matrix may represent the input feature data, while the second matrix may represent the model's weight parameters.

[0042] S102: Determine whether the target operation is block calculation; if so, proceed to S103.

[0043] In specific implementations, after obtaining the source data, it is necessary to determine whether the target operation requires block calculation. Block calculation is an optimization strategy used to handle large-scale matrix operations, which improves computational efficiency by decomposing the matrix into multiple small matrices. The basis for determining whether block calculation is needed is usually the size of the matrix and the limitations of hardware resources. If the matrix size is large and exceeds the capacity of the hardware cache, or the hardware resources are limited, then block calculation is particularly important. Through block calculation, the delay in data transmission can be reduced, the utilization of the cache can be improved, and the overall computing performance can be improved. If the target operation does not require block calculation, the overall operation can be performed directly; if block calculation is required, proceed to the next step to determine the block size.

[0044] S103: Determine a first block size of the first matrix and a second block size of the second matrix.

[0045] In specific implementations, after determining that the target operation requires block calculation, the next step is to determine the block size of the first matrix and the second matrix. The choice of block size has an important impact on the computing efficiency and the utilization of hardware resources. The determination of the block size requires comprehensive consideration of the size of the matrix, the capacity of the hardware cache, and the complexity of the calculation logic. For example, if the hardware cache is small, the block size should choose a smaller block matrix to ensure that each block matrix can be completely stored in the cache, thereby reducing the number of accesses to the main memory. At the same time, the block size should also be as close as possible to the cache size to make full use of the cache resources. In addition, the determination of the block size also needs to consider the complexity of the calculation logic to ensure that the matrix after block calculation can be efficiently calculated.

[0046] S104: Divide the first matrix into blocks and store them into a plurality of first block matrices according to a first block size, and divide the second matrix into blocks and store them into a plurality of second block matrices according to a second block size.

[0047] In a specific implementation, after the block size is determined, the first matrix is ​​stored in blocks as multiple first block matrices according to the determined first block size, and the second matrix is ​​stored in blocks as multiple second block matrices according to the second block size. The purpose of block storage is to decompose a large-scale matrix into multiple small-block matrices so that data can be read and processed more efficiently during the calculation process. Block storage can reduce the delay of data transmission, improve the utilization of the cache, and thus improve the overall computing performance. For example, in matrix multiplication operations, through block storage, the multiplication operation of a large-scale matrix can be decomposed into the multiplication operation of multiple small-block matrices, and the multiplication operation of each small-block matrix can be efficiently completed in the cache, thereby reducing the number of accesses to the main memory and improving the computing speed.

[0048] S105: During the process of executing the target operation, the plurality of first block matrices and the plurality of second block matrices are sequentially read to implement the target operation and obtain the target operation result.

[0049] In a specific implementation, during the execution of the target operation, multiple first block matrices and multiple second block matrices are read in sequence, and the operation is performed according to the predetermined calculation logic to finally obtain the target operation result. The purpose of reading the block matrices in sequence is to ensure that each block matrix can be efficiently operated in the cache, thereby reducing the delay in data transmission. By using block storage and on-demand reading, cache resources can be better utilized and the operation efficiency can be improved. For example, in a matrix multiplication operation, the first block matrix and the second block matrix after the block are read in sequence, the block matrix multiplication operation is performed, and then the results are accumulated to obtain the final result. This on-demand reading method not only improves the data reading efficiency, but also reduces the waste of hardware resources, making the entire operation process more efficient.

[0050] For example, when calculating feature matrix A×weight matrix B, weight matrix B can be stored in blocks in advance according to the block matrix size required by the actual calculation process. In this way, when reading data during calculation, there is no need to skip the jump data length of each row after reading part of the data in each row and read the data of the next row. The advance block storage of the weight matrix can make the data read continuously during the actual calculation, thereby reducing the address jump calculation during reading, thereby shortening the delay caused by obtaining data. Figure 2 As shown in , assuming that the weight data is stored continuously in the memory, then when the block calculation is performed, reading the second line of data B1 requires skipping Nn data and then reading the next line, which will increase the reading delay. If the data is stored in the block manner according to the calculation, as shown in Figure 2 As shown on the right side, the block data B1 is stored continuously in the memory, and there is no need to calculate the jump length, thereby reducing the delay in obtaining data. In actual calculations, the required block data can be stored in advance for calculation and acquisition in the next step. After the calculation of this step is completed, it can be decided whether block storage is needed based on the actual calculation of the next step. If the next step is still a block calculation, then the calculation result of this step can be stored according to the actual block scale; if the next step does not require block calculation, but needs to calculate the entire large matrix, such as softmax, layrenorm (layer normalization), activation function calculation, then the calculation result of this step is stored as the original matrix scale, such as Figure 2 shown on the left.

[0051] The AI ​​model-based operation method provided by the present invention blocks the first and second matrices before executing the target operation, splitting the large matrix into multiple smaller matrices. This block storage strategy makes data reading more efficient during the calculation process, reduces address jump calculations, shortens data acquisition latency, and thus improves computational efficiency and conserves hardware resources.

[0052] On the basis of the above embodiments, it also includes: constructing an instruction architecture to use the instructions in the instruction architecture to perform matrix multiplication and / or nonlinear function operations during the operation of the artificial intelligence model; wherein the instruction architecture includes direct memory access instructions and control instructions, and the control instructions include any one or a combination of any several of the operation instructions, configuration register instructions, loop instructions, general register calculation instructions, and general register condition judgment instructions.

[0053] The artificial intelligence model of this embodiment can be a Transformer model (a deep learning architecture based on the self-attention mechanism). During the reasoning process of the artificial intelligence model, the main computational blocks involved include multi-layer perceptrons, layer normalization, softmax, and activation function calculations. Finer-grained computations at the operator level include matrix-matrix multiplication, matrix-vector multiplication, matrix transposition, matrix bisection with the left and right halves swapped and the right part negated, vector summation, vector exp (exponential operation), and vector activation function calculation. The most basic operations at the lowest level include scalar reciprocal, scalar addition, product, square root, exp, and activation function calculations. This embodiment summarizes and analyzes the above-mentioned computational types, summarizes common characteristics, and constructs an instruction architecture to achieve the goal of efficient execution with simple instruction design.

[0054] For matrix and vector-related calculation types, they can be summarized as data acquisition, calculation, and result writing. When analyzing at the instruction level, the main steps are: reading data, completing data acquisition, performing calculations (involving loops and conditional judgments), completing calculations, writing back results, and completing data writing. Instructions are designed based on these steps to build an instruction architecture. The instruction architecture can include operation instructions, register configuration instructions, loop instructions, general register calculation instructions, general register conditional judgment instructions, etc. The instructions can be used to perform matrix multiplication operations and nonlinear function operations during the operation of artificial intelligence models, such as layernorm, softmax, gelu (Gaussian Error Linear Unit) and other function operations.

[0055] As a feasible implementation method, the direct memory access instruction includes a direct memory read instruction and a direct memory write instruction; the direct memory read instruction includes any one or a combination of any several of the following fields: an operator field, an address field, a data length field, a row length field, a jump length field, a data type field, and a maximum burst transfer length field; the operator field is used to indicate the direct memory access module used to read the data; the address field is used to indicate the starting address of the read data; the data length field is used to indicate the data length of the read data; the row length field is used to indicate the length of each row of data when reading data in blocks; the jump length field is used to indicate the length of data that needs to be skipped at the end of each row when reading data in blocks; the data type field is used to indicate the data type of the read data; and the maximum burst transfer length field is used to indicate the specified data size for a single burst operation during the data reading process. The maximum number of module units; the direct memory write instruction includes any one or a combination of any several of the operator field, address field, data length field, row length field, jump length field, direct memory access module management field, and maximum burst transfer length field. The operator field is used to indicate the direct memory access module used to write back the data, the address field is used to indicate the starting address of the write-back data, the data length field is used to indicate the data length of the write-back data, the row length field is used to indicate the length of each row of data when writing back the data in blocks, the jump length field is used to indicate the length of data that needs to be skipped at the end of each row when writing back the data in blocks, the direct memory access module management field is used to indicate the status of the direct memory access module used to write back the data, and the maximum burst transfer length field is used to indicate the maximum number of specified data scale units transmitted in a single burst operation during the write-back data process.

[0056] Direct memory read instructions are used to read data and write it to the cache. For example, they use direct memory read instructions to read the source data of the target operation. Direct memory write instructions are used to write results back to the cache or DDR (Double Data Rate SDRAM). For example, they use direct memory write instructions to write the target operation results to the main memory.

[0057] The direct memory read instruction contains multiple fields that together define the specific parameters and behavior of the data read operation. The operator field is used to indicate the direct memory access module to be used when reading data. It explicitly specifies which DMA (Direct Memory Access) module will perform the data read operation through a specific number or identifier. The address field indicates the starting address of the data to be read, which is the specific location of the data in the memory. The data length field defines the length of the data to be read. It specifies how many bytes or data units need to be read continuously starting from the starting address. The row length field is used when reading data in blocks. It indicates the length of each row of data, which is particularly important for processing matrices or two-dimensional data structures. The jump length field is used to indicate the length of data that needs to be skipped at the end of each row when reading data in blocks, which helps to process non-contiguously stored data. The data type field indicates the data type of the read data, which is critical to ensuring that the data is interpreted and processed correctly. The maximum burst transfer length field is used to indicate the maximum number of specified data size units transferred in a single burst operation during the data read process, which helps to optimize the efficiency of data transfer. For example, Figure 3 As shown, the direct memory read instruction includes a prefix field, an operator field, an address field, a data length field, a line length field, a jump length field, a data type field, and a maximum burst transfer length field.

[0058] Direct memory write instructions also contain similar fields, but with some specific differences. The operator field is used to indicate the direct memory access module to be used when writing back data. The address field indicates the starting address of the data to be written back. The data length field defines the length of the data to be written back. The row length field and the jump length field are used when writing back data in blocks. They indicate the length of each row of data and the length of data to be skipped at the end of each row, respectively. The direct memory access module management field is used to indicate the status of the direct memory access module used to write back data, which may include the current status of the module, availability, or other status information related to module management. The maximum burst transfer length field is used to indicate the maximum number of specified data size units transferred in a single burst operation during the data write back process, which helps to optimize the efficiency of data write back.

[0059] Through the combination of these fields, direct memory access instructions can precisely control data read and write operations, ensuring efficient and accurate data transfer. The design of these fields takes into account the complexity and diversity of data storage, enabling the DMA module to adapt to a variety of application scenarios and data structures.

[0060] As a feasible implementation method, the operation instruction includes any one or a combination of several of the operator field, the operation type field, and the identifier field. The operator field is used to indicate the operation, the operation type field is used to indicate the operation type, and the identifier field is used to indicate the operation attribute.

[0061] Operation instructions are used to start matrix multiplication, vector, and scalar operations, such as using operation instructions to perform target operations. Operation instructions may include an operator field, an operation type field, an identifier field, and the like. Among them, the operator field is used to indicate the operation operation. It clearly indicates that the current instruction is an operation instruction through a specific encoding or identifier, so that the hardware can recognize and execute the corresponding operation. The operation type field is used to indicate the specific type of operation, such as addition, subtraction, multiplication, or division, etc. It provides the hardware with detailed information on what operation to perform. The identifier field is used to indicate the operation attributes, which may include additional information such as the priority of the operation and whether special processing is required, so that the operation can be optimized or adjusted according to specific attributes. For example, Figure 4 As shown, the operation instruction includes a prefix field, an operator field, an operation type field, a flag field, and a reserved field.

[0062] As a feasible implementation method, the configuration register instruction includes any one or a combination of an operator field, a register address field, and a register data field. The operator field is used to indicate the configuration register operation, the register address field is used to indicate the address of the register to be configured, and the register data field is used to indicate the data to be written to the register.

[0063] The configuration register instruction is used to configure data for the specified register address, which is widely used in computing. For example, by configuring the register value, it indicates whether the data reading process is completed, whether the calculation process is completed, whether the data writing process is completed, etc. The configuration register instruction may include an operator field, a register address field, a register data field, etc. Among them, the operator field is used to indicate the configuration register operation. It clearly identifies the current instruction as a configuration register instruction, so that the hardware can recognize and execute the corresponding configuration operation. The register address field is used to indicate the address of the register that needs to be configured. It provides the location information of the specific register that needs to be configured to ensure that the data can be written to the correct register. The register data field is used to indicate the data that needs to be written to the register. It provides the specific data value to be stored in the register, thereby completing the configuration of the register. For example, Figure 5 As shown, the configuration register instruction includes a prefix field, an operator field, a register address field, a register data field, and a reserved field.

[0064] As a feasible implementation method, the loop instruction includes a loop start instruction and a loop end instruction; the loop start instruction includes an operator field, and the operator field is used to indicate the start of the loop; the loop end instruction includes any one or a combination of an operator field, a register address field, and a loop count field, and the operator field is used to indicate the end of the loop, the register address field is used to indicate the address of the loop control register, and the loop count field is used to indicate the number of iterations of the loop.

[0065] The loop start instruction includes an operator field, which is used to indicate the start of the loop. It notifies the hardware of the start of the loop operation through a specific code or identifier, so that the hardware can initialize the loop control logic and prepare to execute the instructions in the loop body. For example, Figure 6 As shown, the loop start instruction includes a prefix field, an operator field, and a reserved field. The loop end instruction includes any one or a combination of any several of the operator field, the register address field, and the loop count field. The operator field is used to indicate the end of the loop. It clearly identifies the end of the loop operation, allowing the hardware to terminate the loop control logic. The register address field is used to indicate the address of the loop control register, which provides the register location for storing loop control information (such as a loop counter). The loop count field is used to indicate the number of iterations of the loop. It provides the specific number of times the loop needs to be executed, so that the hardware can control the execution of the loop based on this number. For example, Figure 7 As shown, the loop end instruction includes a prefix field, an operator field, a register address field, a loop count field, and a reserved field.

[0066] As a feasible implementation method, a general register calculation instruction includes any one or a combination of any two of an operator field, a general register operator field, a general register address field, and an immediate field. The operator field is used to indicate a general register calculation operation, the general register operator field is used to indicate the type of calculation operation, the general register address field is used to indicate the address of the general register involved in the calculation, and the immediate field is used to indicate the immediate value used in the calculation.

[0067] In a specific implementation, in order to ensure consistency in the generated instructions in the loop calculation, it is necessary to configure the general registers for the changing values ​​in the loop, and design general register calculation instructions to obtain the specific numerical values ​​required in the loop. The general register calculation instruction may include an operator field, a general register operator field, a general register address field, an immediate field, etc. Among them, the operator field is used to indicate the general register calculation operation. It clearly identifies the current instruction as a general register calculation instruction, so that the hardware can recognize and execute the corresponding calculation operation. The general register operator field is used to indicate the type of calculation operation, such as addition, subtraction, displacement, etc. It provides the hardware with detailed information on what calculation operation to perform. The general register address field is used to indicate the operation on the register value of the source address (saddr0, saddr1) and write the data to the destination register address (daddr). The immediate field is used to indicate the immediate value used in the calculation. It provides the specific value used directly in the calculation process, so that the calculation can use the data in the register or the immediate value to operate as needed. For example, Figure 8 As shown, the general register calculation instruction includes a prefix field, an operator field, a general register operator field, a general register address field, and an immediate field.

[0068] As a feasible implementation method, a general register conditional judgment instruction includes any one or a combination of any several of an operator field, a general register operator field, a conditional symbol, a general register address field, and an immediate field. The operator field is used to indicate a general register conditional judgment operation, the general register operator field is used to indicate the operation type of the conditional judgment, the conditional symbol is used to indicate the conditional judgment type, the general register address field is used to indicate the address of the general register involved in the conditional judgment, and the immediate field is used to indicate the immediate value used in the conditional judgment.

[0069] In a specific implementation, the conditional logic in the calculation process can be implemented through general register conditional judgment instructions to implement different task executions when different conditions are met. The general register conditional judgment instruction may include an operator field, a general register operator field, a conditional symbol, a general register address field, an immediate field, etc. Among them, the operator field is used to indicate the general register conditional judgment operation, which clearly identifies the current instruction as a conditional judgment instruction, so that the hardware can recognize and execute the corresponding conditional judgment operation. The general register operator field is used to indicate the operation type of the conditional judgment, such as if, elseif, else, end, ifi, elseeifi, etc., which provides the hardware with detailed information on what conditional judgment operation to perform. The conditional symbol is used to indicate the conditional judgment type, such as equal to, greater than, less than, etc., which defines the specific conditional judgment logic. The general register address field is used to indicate the address of the general register involved in the conditional judgment, which provides the specific location information of the register involved in the conditional judgment. The immediate field is used to indicate the immediate value used in the conditional judgment, which provides the specific value used directly in the conditional judgment process, so that the conditional judgment can use the data in the register or the immediate value to operate as needed. For example, Figure 9 As shown, the general register condition judgment instruction includes a prefix field, an operator field, a general register operator field, a conditional character, a general register address field, and an immediate field.

[0070] The embodiment of the present invention discloses an operation method based on an artificial intelligence model. Compared with the previous embodiment, this embodiment further explains and optimizes the technical solution. Specifically:

[0071] See also Figure 10 , according to an exemplary embodiment, a flowchart of another operation method based on an artificial intelligence model is shown, such as Figure 10 As shown, including:

[0072] S201: Acquire source data of a target operation; wherein the source data includes a first matrix and a second matrix.

[0073] S202: Determine whether the target operation is block calculation; if so, proceed to S203.

[0074] S203: Determine multiple candidate block partitioning methods; each candidate block partitioning method includes a first candidate block partitioning scale of a first matrix and a second candidate block partitioning scale of a second matrix; the first candidate block partitioning scale and the second candidate block partitioning scale are both smaller than the cache size.

[0075] In a specific implementation, to determine the first block size for the first matrix and the second block size for the second matrix, multiple candidate block sizes must first be identified. Each candidate block size includes a first candidate block size for the first matrix and a second candidate block size for the second matrix. These candidate block sizes are set based on cache size constraints, ensuring that each block size is smaller than the cache capacity. This design optimizes data read efficiency. When the block size fits the cache, the number of main memory accesses is reduced, thereby increasing data processing speed.

[0076] S204: Select a target blocking method from a plurality of candidate blocking methods based on the hardware accelerator resources used by the artificial intelligence model, the number of times the block matrix is ​​read, and the amount of computation required for the block calculation, and determine a first blocking scale for the first matrix and a second blocking scale for the second matrix based on the target blocking method.

[0077] In specific implementations, after identifying multiple candidate partitioning schemes, the next step is to select the most appropriate target partitioning scheme from these candidate partitioning schemes based on the hardware accelerator resources used by the AI ​​model, the number of block matrix reads, and the computational effort required for the block calculations. Hardware accelerators can be designed based on systolic arrays. Hardware accelerator resource considerations involve hardware performance and limitations, such as cache size and the number of processing units. The number of block matrix reads is a key factor, as fewer reads generally mean higher efficiency. The computational effort required for block calculations refers to the number of operations required for each block during the computation; smaller computations generally mean faster processing. Taking these factors into consideration, the target partitioning scheme selected determines the first block size for the first matrix and the second block size for the second matrix. This process involves a trade-off and optimization process, aiming to find a partitioning strategy that strikes the optimal balance between hardware resource utilization, data read efficiency, and computational efficiency. Determining the block size in this way ensures that hardware resources are fully utilized when executing matrix operations, and that both data reads and computations are as efficient as possible, thereby improving the performance of the entire system when processing AI models.

[0078] As a feasible implementation method, a target blocking method is selected from multiple candidate blocking methods based on the hardware accelerator resources used by the artificial intelligence model, the number of times the block matrix is ​​read, and the amount of computational complexity of the block calculation, including: when the hardware accelerator used by the artificial intelligence model supports reading data and computing in parallel, the candidate blocking method with the smallest amount of computational complexity of the block calculation is selected as the target blocking method; when the hardware accelerator used by the artificial intelligence model does not support reading data and computing in parallel, the target blocking method is selected from multiple candidate blocking methods based on the number of times the block matrix is ​​read and the amount of computational complexity of the block calculation.

[0079] In specific implementations, in the process of selecting the target block partitioning method, different strategies will be adopted according to the different characteristics of the hardware accelerator used by the artificial intelligence model. When the hardware accelerator supports parallel reading of data and calculation, that is, data reading and calculation operations can be performed at the same time, this parallel processing capability can effectively reduce the total processing time. In this case, the candidate block partitioning method with the smallest amount of computation for block calculation is selected as the target block partitioning method, that is, the candidate block partitioning method with the smallest overall computational amount of multiplication and accumulation is selected as the target block partitioning method. This is because when data reading and calculation can be performed in parallel, the amount of computation becomes the main factor affecting the processing speed, and the calculation time can cover the reading time. A smaller amount of computation means that the computing task can be completed faster, thereby improving the efficiency of the entire system.

[0080] On the contrary, when the hardware accelerator does not support parallel reading of data and calculation, that is, data reading and calculation operations cannot be performed at the same time, it is necessary to comprehensively consider the number of block matrix reads and the amount of computation required for block calculation to select the target block method. In this case, the number of reads becomes particularly important because each read operation takes a certain amount of time, and too many reads will significantly increase the total processing time. Therefore, it is necessary to find a balance between reducing the number of reads and controlling the amount of computation. By conducting a detailed analysis and comparison of different candidate block partitioning methods, we can determine which method can minimize the number of block matrix reads while ensuring a reasonable amount of computation. In this way, the block partitioning strategy can be optimized to achieve the best performance when hardware resources are limited.

[0081] As a feasible implementation method, a target block mode is selected from a plurality of candidate block modes according to the number of times the block matrix is ​​read and the amount of computation of the block calculation, including: selecting a first candidate block mode and a second candidate block mode from a plurality of candidate block modes; judging whether the ratio between the amount of computation of the block calculation corresponding to the first candidate block mode and the amount of computation of the block calculation corresponding to the second candidate block mode is less than or equal to a preset value; if so, deleting the candidate block mode with the largest number of times the block matrix is ​​read from the first candidate block mode and the second candidate block mode; if not, deleting the candidate block mode with the largest amount of computation from the first candidate block mode and the second candidate block mode; and re-executing the steps of selecting the first candidate block mode and the second candidate block mode from a plurality of candidate block modes until only one candidate block mode remains, and taking the remaining candidate block mode as the target block mode.

[0082] In a specific implementation, when selecting a target block partitioning method, two specific candidate block partitioning methods are first selected from multiple candidate block partitioning methods: a first candidate block partitioning method and a second candidate block partitioning method. This selection process is based on a preliminary screening of different block partitioning methods, with the goal of further comparing and evaluating their performance metrics. Next, it is necessary to determine whether the ratio between the computational effort of the block partitioning corresponding to the first candidate block partitioning method and the computational effort of the block partitioning corresponding to the second candidate block partitioning method is less than or equal to a preset value. Specifically, it is determined whether the ratio between the total computational effort of the multiplication and accumulation computations corresponding to the first candidate block partitioning method and the total computational effort of the multiplication and accumulation computations corresponding to the second candidate block partitioning method is less than or equal to a preset value. This preset value is a critical threshold used to measure whether the relative difference in computational effort between the two block partitioning methods is within an acceptable range; for example, 10 can be selected. If the ratio is less than or equal to the preset value, it indicates that the two block partitioning methods are relatively close in computational effort, and the number of reads becomes the primary decision factor. Therefore, the candidate block partitioning method with the largest number of block matrix reads between the first and second candidate block partitioning methods is eliminated. This is because, when the computational effort is similar, the partitioning method with fewer reads is generally more efficient. If the ratio is greater than a preset value, it indicates a significant difference in computational effort between the two partitioning methods, making computational effort the primary factor in the decision. Therefore, the candidate partitioning method with the highest computational effort between the first and second candidate partitioning methods is removed. This means that the candidate partitioning method with the highest combined multiplication and accumulation effort is removed. This is because, when the number of reads is similar, the partitioning method with the lowest computational effort generally completes the computation faster. After removing a candidate partitioning method, the steps of selecting the first and second candidate partitioning methods are repeated. This process is repeated, comparing and removing one candidate partitioning method at a time, until only one candidate partitioning method remains. Ultimately, the remaining candidate partitioning method is selected as the target partitioning method. This step-by-step screening and comparison method ensures that the optimal partitioning method is selected, taking into account both computational effort and number of reads, thereby achieving efficient matrix operations within limited hardware resources.

[0083] For example, for the calculation of A×B, the size of matrix A is 32×896, the size of matrix B is 896×4864, and the cache size is 256×256. The first candidate block method: Matrix A is divided into block matrices of size 32×128 , the B matrix is ​​divided into 128×256 block matrices , so that the result matrix C is composed of the block matrix Composition, and ,like Figure 11In the above block matrix multiplication calculation, each time the block result matrix is ​​calculated Both need to read the large matrix A once and read the block matrix Once, so the overall calculation result matrix A needs to read matrix A 19 times and read matrix B once. The total amount of multiplication calculation is 32×128×256×7×19=139460608. The matrix with a block size of 32×256 needs to be accumulated 6×19 times, so the total amount of accumulation calculation is 32×256×6×19=933888. The second candidate block method: Matrix A is divided into block matrices of size 32×8 , the B matrix is ​​divided into block matrices of size 8×4864 , so the resulting matrix C is given by Get, such as Figure 12 As shown above. In the above block matrix multiplication calculation, the overall calculation result matrix C requires reading matrix A once and matrix B once, resulting in a total multiplication computation of 32×8×4864×112=139460608. Each intermediate result matrix of size 32×4864 requires 111 accumulations, resulting in a total cumulative computation of 32×4864×111=17276928. If the hardware accelerator used by the AI ​​model supports parallel data reading and computation, the first candidate block partitioning method with the lowest computational effort is selected. If the hardware accelerator used by the AI ​​model does not support parallel data reading and computation, the ratio of the computational effort of the second candidate block partitioning method (17276928 + 139460608) to the computational effort of the first candidate block partitioning method (933888 + 139460608) is approximately 1.1164, which is less than the preset value of 10. In this case, the second candidate block partitioning method with the lowest number of reads is selected.

[0084] S205: Divide the first matrix into blocks and store them into a plurality of first block matrices according to the first block size, and divide the second matrix into blocks and store them into a plurality of second block matrices according to the second block size.

[0085] S206: During the process of executing the target operation, the plurality of first block matrices and the plurality of second block matrices are sequentially read to implement the target operation and obtain the target operation result.

[0086] It can be seen that this embodiment flexibly determines the blocking method of the first matrix and the second matrix based on the hardware accelerator resources used by the artificial intelligence model, the number of times the block matrix is ​​read, and the amount of computational complexity of the block calculation. On the premise of meeting the hardware resources that the hardware accelerator can provide, it effectively reduces the number of data reads and the amount of computation, thereby improving computing efficiency.

[0087] The following introduces an operating device based on an artificial intelligence model provided in an embodiment of the present invention. The operating device based on an artificial intelligence model described below and the operating method based on an artificial intelligence model described above can refer to each other.

[0088] See also Figure 13 , according to an exemplary embodiment, a structural diagram of an operating device based on an artificial intelligence model is shown, such as Figure 13 Shown, including:

[0089] An acquisition module 100 is configured to acquire source data for a target operation; wherein the source data includes a first matrix and a second matrix;

[0090] A determination module 200 is configured to determine whether the target operation is a block calculation; if so, determine a first block size of the first matrix and a second block size of the second matrix;

[0091] A block storage module 300 is configured to store the first matrix into a plurality of first block matrices according to a first block size, and to store the second matrix into a plurality of second block matrices according to a second block size;

[0092] The operation module 400 is used to read the plurality of first block matrices and the plurality of second block matrices in sequence during the process of executing the target operation, so as to implement the target operation and obtain the target operation result.

[0093] The AI ​​model-based operating device provided by an embodiment of the present invention stores the first and second matrices in blocks before executing the target operation, splitting the large matrix into multiple smaller matrices. This block storage strategy makes data reading more efficient during the calculation process, reduces address jump calculations, shortens data acquisition latency, and thus improves computational efficiency and conserves hardware resources.

[0094] Based on the above embodiment, as a preferred implementation, it further includes:

[0095] A construction module is used to construct an instruction architecture to use the instructions in the instruction architecture to perform matrix multiplication and / or nonlinear function operations during the operation of the artificial intelligence model; wherein the instruction architecture includes direct memory access instructions and control instructions, and the control instructions include any one or a combination of any several of the following: operation instructions, configuration register instructions, loop instructions, general register calculation instructions, and general register condition judgment instructions.

[0096] Based on the above embodiment, as a preferred implementation, the direct memory access instruction includes a direct memory read instruction and a direct memory write instruction;

[0097] The direct memory read instruction includes any one or a combination of any several of an operator field, an address field, a data length field, a row length field, a jump length field, a data type field, and a maximum burst transfer length field. The operator field is used to indicate the direct memory access module used to read data. The address field is used to indicate the starting address of the read data. The data length field is used to indicate the data length of the read data. The row length field is used to indicate the length of each row of data when reading data in blocks. The jump length field is used to indicate the length of data that needs to be skipped at the end of each row when reading data in blocks. The data type field is used to indicate the data type of the read data. The maximum burst transfer length field is used to indicate the maximum number of specified data scale units transmitted in a single burst operation during the data reading process.

[0098] The direct memory write instruction includes any one or a combination of any several of the following: an operator field, an address field, a data length field, a row length field, a jump length field, a direct memory access module management field, and a maximum burst transfer length field. The operator field is used to indicate the direct memory access module used to write back the data, the address field is used to indicate the starting address of the write-back data, the data length field is used to indicate the data length of the write-back data, the row length field is used to indicate the length of each row of data when writing back the data in blocks, the jump length field is used to indicate the length of data that needs to be skipped at the end of each row when writing back the data in blocks, the direct memory access module management field is used to indicate the status of the direct memory access module used to write back the data, and the maximum burst transfer length field is used to indicate the maximum number of specified data scale units transmitted in a single burst operation during the write-back data process.

[0099] Based on the above embodiment, as a preferred implementation, the acquisition module 100 is specifically configured to: read the source data of the target operation using a direct memory read instruction;

[0100] Accordingly, the device further comprises:

[0101] The write module is used to write the target operation result into the memory using the direct memory write instruction.

[0102] Based on the above embodiments, as a preferred implementation, the operation instruction includes any one or a combination of several of the operator field, the operation type field, and the identifier field. The operator field is used to indicate the operation, the operation type field is used to indicate the operation type, and the identifier field is used to indicate the operation attributes.

[0103] Based on the above embodiment, as a preferred implementation, the operation module 400 is specifically configured to: execute a target operation using an operation instruction.

[0104] Based on the above embodiments, as a preferred implementation method, the configuration register instruction includes any one or a combination of several of the operator field, register address field, and register data field. The operator field is used to indicate the configuration register operation, the register address field is used to indicate the address of the register to be configured, and the register data field is used to indicate the data to be written to the register.

[0105] Based on the above embodiment, as a preferred implementation, the loop instruction includes a loop start instruction and a loop end instruction;

[0106] The loop start instruction includes an operator field, and the operator field is used to indicate the start of the loop;

[0107] The loop end instruction includes any one or a combination of an operator field, a register address field, and a loop count field. The operator field is used to indicate the end of the loop, the register address field is used to indicate the address of the loop control register, and the loop count field is used to indicate the number of iterations of the loop.

[0108] Based on the above embodiments, as a preferred implementation, the general register calculation instruction includes any one or a combination of any several of an operator field, a general register operator field, a general register address field, and an immediate field. The operator field is used to indicate a general register calculation operation, the general register operator field is used to indicate the type of calculation operation, the general register address field is used to indicate the address of the general register involved in the calculation, and the immediate field is used to indicate the immediate value used in the calculation.

[0109] Based on the above embodiments, as a preferred implementation mode, the general register conditional judgment instruction includes any one or a combination of any several of the operator field, the general register operator field, the conditional symbol, the general register address field, and the immediate field. The operator field is used to indicate a general register conditional judgment operation, the general register operator field is used to indicate the operation type of the conditional judgment, the conditional symbol is used to indicate the conditional judgment type, the general register address field is used to indicate the address of the general register involved in the conditional judgment, and the immediate field is used to indicate the immediate value used in the conditional judgment.

[0110] Based on the above embodiment, as a preferred implementation, the determination module 200 includes:

[0111] a determination unit, configured to determine a plurality of candidate block partitioning modes; wherein each candidate block partitioning mode includes a first candidate block partitioning scale of a first matrix and a second candidate block partitioning scale of a second matrix; wherein both the first candidate block partitioning scale and the second candidate block partitioning scale are smaller than a cache size;

[0112] A selection unit is used to select a target blocking method from multiple candidate blocking methods based on the hardware accelerator resources used by the artificial intelligence model, the number of times the block matrix is ​​read, and the amount of computational effort of the block calculation, and to determine a first blocking scale of the first matrix and a second blocking scale of the second matrix based on the target blocking method.

[0113] Based on the above embodiments, as a preferred implementation mode, the selection unit is specifically used to: when the hardware accelerator used by the artificial intelligence model supports reading data and calculating in parallel, select the candidate block calculation method with the least computational complexity as the target block calculation method.

[0114] Based on the above embodiments, as a preferred implementation mode, the selection unit is specifically used to: when the hardware accelerator used by the artificial intelligence model supports reading data and calculating in parallel, select the candidate block mode with the smallest overall computational amount of multiplication and accumulation as the target block mode.

[0115] Based on the above embodiments, as a preferred implementation mode, the selection unit is specifically used for: when the hardware accelerator used by the artificial intelligence model does not support parallel reading of data and calculation, selecting the target blocking method from multiple candidate blocking methods according to the number of times the block matrix is ​​read and the amount of computation of the block calculation.

[0116] On the basis of the above embodiments, as a preferred implementation mode, the selection unit is specifically used to: when the hardware accelerator used by the artificial intelligence model does not support parallel reading of data and calculation, select the first candidate blocking method and the second candidate blocking method from multiple candidate blocking methods; determine whether the ratio between the computational amount of the block calculation corresponding to the first candidate blocking method and the computational amount of the block calculation corresponding to the second candidate blocking method is less than or equal to a preset value; if so, delete the candidate blocking method with the largest number of block matrix reads in the first candidate blocking method and the second candidate blocking method; if not, delete the candidate blocking method with the largest computational amount in the first candidate blocking method and the second candidate blocking method; re-execute the steps of selecting the first candidate blocking method and the second candidate blocking method from multiple candidate blocking methods until one candidate blocking method remains, and use the remaining candidate blocking method as the target blocking method.

[0117] On the basis of the above embodiments, as a preferred implementation mode, the selection unit is specifically used to: when the hardware accelerator used by the artificial intelligence model does not support parallel reading of data and calculation, select the first candidate blocking method and the second candidate blocking method from multiple candidate blocking methods; determine whether the ratio between the total amount of multiplication and accumulation calculations corresponding to the first candidate blocking method and the total amount of multiplication and accumulation calculations corresponding to the second candidate blocking method is less than or equal to a preset value; if so, delete the candidate blocking method with the largest number of block matrix readings in the first candidate blocking method and the second candidate blocking method; if not, delete the candidate blocking method with the largest total amount of multiplication and accumulation calculations in the first candidate blocking method and the second candidate blocking method; re-execute the step of selecting the first candidate blocking method and the second candidate blocking method from multiple candidate blocking methods until one candidate blocking method remains, and use the remaining candidate blocking method as the target blocking method.

[0118] Regarding the apparatus in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0119] An embodiment of the present invention further provides an electronic device, Figure 14 FIG. 1 is a structural diagram of an electronic device according to an exemplary embodiment. Figure 14 As shown, the electronic equipment includes:

[0120] Communication interface 1, capable of exchanging information with other devices such as network devices;

[0121] The processor 2 is connected to the communication interface 1 to implement information exchange with other devices and is used to execute the operation method based on the artificial intelligence model provided by one or more of the above technical solutions when running the computer program. The computer program is stored in the memory 3.

[0122] Of course, in actual application, the various components in the electronic device are coupled together through the bus system 4. It can be understood that the bus system 4 is used to realize the connection and communication between these components. In addition to the data bus, the bus system 4 also includes a power bus, a control bus, and a status signal bus. However, for the sake of clarity, Figure 14 Various buses are labeled as bus system 4.

[0123] The memory 3 in the embodiment of the present invention is used to store various types of data to support the operation of the electronic device. Examples of such data include: any computer program used to operate on the electronic device.

[0124] It is understood that the memory 3 can be a volatile memory or a non-volatile memory, or can include both volatile and non-volatile memories. Among them, the non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a magnetic random access memory (FRAM), a flash memory, a magnetic surface memory, an optical disc, or a compact disc read-only memory (CD-ROM); the magnetic surface memory can be a magnetic disk or a magnetic tape. The volatile memory can be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static random access memory (SRAM), synchronous static random access memory (SSRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct rambus random access memory (DRRAM).The memory 3 described in the embodiments of the present invention is intended to include but is not limited to these and any other suitable types of memories.

[0125] The method disclosed in the above embodiment of the present invention can be applied to processor 2 or implemented by processor 2. Processor 2 may be an integrated circuit chip with signal processing capabilities. During implementation, each step of the above method can be completed by the hardware integrated logic circuit in processor 2 or by instructions in software form. The above processor 2 can be a general-purpose processor, a DSP, or other programmable logic device, discrete gate or transistor logic device, discrete hardware component, etc. Processor 2 can implement or execute the various methods, steps and logic block diagrams disclosed in the embodiment of the present invention. A general-purpose processor can be a microprocessor or any conventional processor, etc. The steps of the method disclosed in the embodiment of the present invention can be directly embodied as being executed by a hardware decoding processor, or can be executed by a combination of hardware and software modules in the decoding processor. The software module can be located in a storage medium, which is located in memory 3. Processor 2 reads the program in memory 3 and completes the steps of the above method in combination with its hardware.

[0126] When the processor 2 executes the program, the corresponding processes in the various methods of the embodiments of the present invention are implemented, which will not be described here for the sake of brevity.

[0127] An embodiment of the present invention also provides a computer-readable storage medium, which stores a computer program, wherein the computer program is configured to execute the steps of any of the above-mentioned embodiments of the operating method based on the artificial intelligence model when running.

[0128] In an exemplary embodiment, the computer-readable storage medium may include, but is not limited to, various media that can store computer programs, such as a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk, or an optical disk.

[0129] An embodiment of the present invention further provides a computer program product, which includes a computer program. When the computer program is executed by the processor 2, it implements the steps in any of the above-mentioned embodiments of the operation method based on the artificial intelligence model.

[0130] An embodiment of the present invention also provides another computer program product, including a non-volatile computer-readable storage medium, which stores a computer program. When the computer program is executed by the processor 2, it implements the steps of any of the above-mentioned embodiments of the operating method based on the artificial intelligence model.

[0131] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the above description has generally described the components and steps of each example according to their functions. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present invention.

[0132] The above is a detailed introduction to an operating system, method, device and equipment, medium and product based on an artificial intelligence model provided by the present invention. Specific examples are used herein to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea. It should be pointed out that for ordinary technicians in this technical field, without departing from the principles of the present invention, the present invention can also be improved and modified in several ways, and these improvements and modifications also fall within the scope of protection of the present invention.

Claims

1. An operation method based on an artificial intelligence model, characterized in that: include: Constructing an instruction architecture to utilize instructions in the instruction architecture to perform matrix multiplication and / or nonlinear function operations during the operation of the artificial intelligence model; wherein the instruction architecture includes direct memory access instructions, the direct memory access instructions include direct memory read instructions and direct memory write instructions, the direct memory read instructions include a jump length field for indicating the length of data to be skipped at the end of each row when reading data in blocks, and the direct memory write instructions include a jump length field for indicating the length of data to be skipped at the end of each row when writing data back in blocks; Acquire source data of a target operation; wherein the source data includes a first matrix and a second matrix; Determine whether the target operation is a block calculation; if so, determine a first block size of the first matrix and a second block size of the second matrix; Divide the first matrix into blocks and store them into a plurality of first block matrices according to the first block size, and divide the second matrix into blocks and store them into a plurality of second block matrices according to the second block size; During the process of executing the target operation, sequentially reading a plurality of the first block matrices and a plurality of the second block matrices to implement the target operation and obtain a target operation result; Determining a first block size of the first matrix and a second block size of the second matrix includes: Selecting a target block partitioning method from a plurality of candidate block partitioning methods according to hardware accelerator resources used by the artificial intelligence model, the number of times the block matrix is ​​read, and the amount of computation required for block calculation, and determining a first block size of the first matrix and a second block size of the second matrix according to the target block partitioning method; Among them, the target block partitioning method is selected from multiple candidate block partitioning methods based on the hardware accelerator resources used by the artificial intelligence model, the number of times the block matrix is ​​read, and the amount of computation required for block partitioning, including: When the hardware accelerator used by the artificial intelligence model supports parallel reading of data and calculation, the candidate partitioning method with the smallest total computational load of multiplication and accumulation is selected as the target partitioning method; When the hardware accelerator used by the artificial intelligence model does not support parallel reading of data and calculation, selecting a first candidate block mode and a second candidate block mode from the multiple candidate block modes; Determine whether a ratio between a total amount of multiplication and accumulation calculations corresponding to the first candidate block partitioning method and a total amount of multiplication and accumulation calculations corresponding to the second candidate block partitioning method is less than or equal to a preset value; If so, the candidate block partitioning method with the largest number of block matrix reads among the first candidate block partitioning method and the second candidate block partitioning method is deleted; if not, the candidate block partitioning method with the largest total amount of multiplication and accumulation calculations among the first candidate block partitioning method and the second candidate block partitioning method is deleted; The step of selecting the first candidate block partitioning method and the second candidate block partitioning method from the plurality of candidate block partitioning methods is re-executed until one candidate block partitioning method remains, and the remaining candidate block partitioning method is used as the target block partitioning method.

2. The operation method based on the artificial intelligence model according to claim 1, characterized in that: The instruction architecture further includes control instructions, which include any one or a combination of several of operation instructions, register configuration instructions, loop instructions, general register calculation instructions, and general register condition judgment instructions.

3. The operation method based on the artificial intelligence model according to claim 2, characterized in that: The direct memory read instruction further includes any one or a combination of any several of an operator field, an address field, a data length field, a row length field, a data type field, and a maximum burst transfer length field, wherein the operator field is used to indicate a direct memory access module used to read data, the address field is used to indicate a starting address of the read data, the data length field is used to indicate the data length of the read data, the row length field is used to indicate the length of each row of data when reading data in blocks, the data type field is used to indicate the data type of the read data, and the maximum burst transfer length field is used to indicate the maximum number of specified data scale units transmitted in a single burst operation during the data reading process; The direct memory write instruction also includes any one or a combination of any two of an operator field, an address field, a data length field, a row length field, a direct memory access module management field, and a maximum burst transfer length field. The operator field is used to indicate the direct memory access module used to write back the data, the address field is used to indicate the starting address of the write-back data, the data length field is used to indicate the data length of the write-back data, the row length field is used to indicate the length of each row of data when writing back the data in blocks, the direct memory access module management field is used to indicate the status of the direct memory access module used to write back the data, and the maximum burst transfer length field is used to indicate the maximum number of specified data scale units transmitted in a single burst operation during the write-back data process.

4. The operation method based on the artificial intelligence model according to claim 3, characterized in that: Get the source data for the target operation, including: Using the direct memory read instruction to read source data of the target operation; After obtaining the target operation result, the method further includes: The target operation result is written into the memory using the direct memory write instruction.

5. The operation method based on the artificial intelligence model according to claim 2, characterized in that: The operation instruction includes any one or a combination of any two of an operator field, an operation type field, and an identifier field. The operator field is used to indicate an operation operation, the operation type field is used to indicate an operation type, and the identifier field is used to indicate an operation attribute.

6. The operation method based on the artificial intelligence model according to claim 2, characterized in that: Executing the target operation includes: The target operation is performed using the operation instruction.

7. The operation method based on the artificial intelligence model according to claim 2, characterized in that: The configuration register instruction includes any one or a combination of any two of an operator field, a register address field, and a register data field. The operator field is used to indicate a configuration register operation, the register address field is used to indicate the address of the register to be configured, and the register data field is used to indicate the data to be written to the register.

8. The operation method based on the artificial intelligence model according to claim 2, characterized in that: The loop instructions include a loop start instruction and a loop end instruction; The loop start instruction includes an operator field, and the operator field is used to indicate the start of a loop; The loop end instruction includes any one or a combination of an operator field, a register address field, and a loop count field. The operator field is used to indicate the end of the loop, the register address field is used to indicate the address of the loop control register, and the loop count field is used to indicate the number of iterations of the loop.

9. The operation method based on the artificial intelligence model according to claim 2, characterized in that: The general register calculation instruction includes any one or a combination of any two of an operator field, a general register operator field, a general register address field, and an immediate field. The operator field is used to indicate a general register calculation operation, the general register operator field is used to indicate the type of calculation operation, the general register address field is used to indicate the address of a general register involved in the calculation, and the immediate field is used to indicate an immediate value used in the calculation.

10. The operation method based on the artificial intelligence model according to claim 2, characterized in that: The general register conditional judgment instruction includes any one or a combination of any two of an operator field, a general register operator field, a conditional symbol, a general register address field, and an immediate field. The operator field is used to indicate a general register conditional judgment operation, the general register operator field is used to indicate the operation type of the conditional judgment, the conditional symbol is used to indicate the conditional judgment type, the general register address field is used to indicate the address of the general register involved in the conditional judgment, and the immediate field is used to indicate the immediate value used in the conditional judgment.

11. The operation method based on the artificial intelligence model according to claim 1, characterized in that: Before selecting a target block partitioning method from multiple candidate block partitioning methods based on the hardware accelerator resources used by the artificial intelligence model, the number of block matrix reads, and the amount of block calculation, the following is also included: Determine multiple candidate block partitioning methods; wherein each candidate block partitioning method includes a first candidate block size of the first matrix and a second candidate block size of the second matrix, and the first candidate block size and the second candidate block size are both smaller than the cache size.

12. An operating device based on an artificial intelligence model, characterized in that: include: A construction module for constructing an instruction architecture to utilize instructions in the instruction architecture to perform matrix multiplication and / or nonlinear function operations during the operation of the artificial intelligence model; wherein the instruction architecture includes direct memory access instructions, the direct memory access instructions include direct memory read instructions and direct memory write instructions, the direct memory read instructions include a jump length field for indicating the length of data to be skipped at the end of each row when reading data in blocks, and the direct memory write instructions include a jump length field for indicating the length of data to be skipped at the end of each row when writing data back in blocks; An acquisition module, configured to acquire source data for a target operation; wherein the source data includes a first matrix and a second matrix; a determination module, configured to determine whether the target operation is a block calculation; and if so, determine a first block size of the first matrix and a second block size of the second matrix; A block storage module, configured to store the first matrix into multiple first block matrices according to the first block scale, and store the second matrix into multiple second block matrices according to the second block scale; an operation module, configured to sequentially read a plurality of the first block matrices and a plurality of the second block matrices during the process of performing the target operation, so as to implement the target operation and obtain a target operation result; Among them, the determination module is specifically used to: when the hardware accelerator used by the artificial intelligence model supports reading data and calculating in parallel, select the candidate block method with the smallest total computational amount of multiplication and accumulation as the target block method; when the hardware accelerator used by the artificial intelligence model does not support reading data and calculating in parallel, select the first candidate block method and the second candidate block method from the multiple candidate block methods; determine whether the ratio between the total computational amount of multiplication and accumulation corresponding to the first candidate block method and the total computational amount of multiplication and accumulation corresponding to the second candidate block method is less than or equal to a preset value; if so, delete the candidate block method with the largest number of block matrix reads in the first candidate block method and the second candidate block method; if not, delete the candidate block method with the largest total computational amount of multiplication and accumulation in the first candidate block method and the second candidate block method; re-execute the step of selecting the first candidate block method and the second candidate block method from the multiple candidate block methods until one candidate block method remains, and use the remaining candidate block method as the target block method.

13. An electronic device, characterized in that: include: memory for storing computer programs; A processor, configured to implement the steps of the artificial intelligence model-based operating method as described in any one of claims 1 to 11 when executing the computer program.

14. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, which, when executed, implements the steps of the operation method based on the artificial intelligence model as described in any one of claims 1 to 11.

15. A computer program product, characterized in that It includes a computer program, which, when executed, implements the steps of the operation method based on the artificial intelligence model as described in any one of claims 1 to 11.

Images