Method and apparatus for updating model parameter table, and computing system

Abstract

A method and apparatus for updating a model parameter table, and a computing system, relating to the technical field of neural networks. A first chip acquires a base address table of a target model in a model execution stage of the target model, and updates a model parameter table of the target model on the basis of the base address table, an address index table of the target model, and an address offset table of the target model. The base address table is generated on the basis of a memory for this instance of execution of the target model, the address index table comprises at least one address index entry, the address offset table comprises at least one address offset entry, the at least one address index entry has one-to-one correspondence to the at least one address offset entry, each address index entry is used for indicating one base address entry in the base address table, and each address index entry corresponds to at least one memory address entry in the model parameter table. The present application can improve the efficiency of updating a model parameter table.

Description

Method and device for updating model parameter table, and computing system

[0001] The present application claims priority to the Chinese patent application No. 202410877634.9, filed on June 28, 2024, and entitled "Method and device for updating model parameter table, and computing system", the content of which is incorporated herein by reference in its entirety. TECHNICAL FIELD

[0002] The present application relates to the technical field of neural network, and particularly relates to a method and device for updating model parameter table, and a computing system. BACKGROUND

[0003] A neural network model (hereinafter referred to as "model") can be deployed in a computing device including a central processing unit (CPU) chip and a model execution chip. The CPU chip is used to allocate memory for the model in the memory of the model execution chip (for example, the memory allocated for the model is referred to as the memory of the model), and schedule the model to the model execution chip for execution. The model execution chip is used to determine the memory of the model based on a model argument table (model args table) of the model, and execute the model based on the memory of the model. The model argument table is located in the memory of the model execution chip, and is used to record the address of the memory of the model. The memory of the model can include model input memory, model output memory and feature map (FM) memory. The FM memory includes the memory of an operator in the model, and the memory of the operator is the memory allocated for the operator in the memory of the model execution chip, including operator input memory, operator output memory, operator workspace memory, etc.

[0004] Generally, the model can be scheduled by using a sink scheduling mode. The process of scheduling the model by using the sink scheduling mode includes a model loading stage (also referred to as a model sinking stage) and a model execution stage. In the model loading stage, the CPU chip sends the operators in the model to the model stream corresponding to the model in the model execution chip in sequence. In the model execution stage, the CPU chip sends the model execution task of the model to the execution stream in the model execution chip, and when the model execution task in the execution stream is scheduled by the model execution chip, the model execution chip executes the model stream corresponding to the model (i.e., executes the operators in the model stream in the order of the operators in the model stream). If the model needs to be executed multiple times, the CPU chip sends the model execution task of the model to the execution stream in the model execution chip multiple times, and the model execution chip executes the model multiple times according to the model execution tasks sent by the CPU chip multiple times.

[0005] In the sinking scheduling mode, in order to avoid that the model occupies the memory of the model execution chip in the whole life cycle (for example, the period from when the model is loaded to the model execution chip to when the model is unloaded from the model execution chip), the memory of the model is usually recycled after the model execution chip executes the model. Before the model execution chip executes the model next time, the memory of the model is allocated in the memory of the model execution chip again, and the model parameter table of the model is updated (that is, the address in the model parameter table is updated) according to the memory allocated to the model again.

[0006] However, the current scheme of updating the model parameter table needs to copy the model parameter table between the CPU chip and the model execution chip, which leads to low efficiency of updating the model parameter table. The copy between the CPU chip and the model execution chip is also called host memory to device memory (H2D) copy. SUMMARY

[0007] The present application provides a model parameter table updating method and device and a computing system. The technical scheme of the present application is as follows.

[0008] In a first aspect, a model parameter table updating method is provided, applied to a first chip, and the method comprises: in a model execution stage of a target model, obtaining a base address table of the target model, the base address table being generated according to a current execution memory of the target model, the current execution memory being a memory allocated in a memory of the first chip for current execution of the target model; and updating a model parameter table of the target model according to the base address table, an address index table of the target model and an address offset table of the target model, the address index table, the address offset table and the model parameter table being in the memory of the first chip, the address index table comprising at least one address index table entry, the address offset table comprising at least one address offset table entry, the model parameter table comprising at least one memory address table entry, each address index table entry corresponding to one address offset table entry, each address index table entry being used to indicate one base address table entry in the base address table, and each address index table entry corresponding to at least one memory address table entry in the model parameter table.

[0009] The address index table, the address offset table and the model parameter table are all in the memory of the first chip in the model execution stage of the target model. The address index table item is used for recording an address index, and the address index table item is used for indicating a basic address table item in the basic address table. Each basic address table item in the basic address table is used for recording a basic address, each address offset table item in the at least one address offset table is used for recording an address offset, and each memory address table item in the model parameter table is used for recording a memory address of the target model. The memory address recorded by each memory address table item is obtained by offsetting the basic address recorded in the basic address table item indicated by the corresponding address index table item by the address offset recorded in the corresponding address offset table item.

[0010] The technical scheme provided in the present application is suitable for updating a model parameter table in a sinking scheduling mode. In the technical scheme provided in the present application, in the model execution stage of a target model, the memory of a first chip includes an address index table of the target model, an address offset table of the target model and a model parameter table of the target model. After the first chip obtains a basic address table of the target model in the model execution stage, the model parameter table can be updated according to the basic address table, the address index table and the address offset table. In this way, in the process of updating the model parameter table of the target model in the model execution stage of the target model, the model parameter table does not need to be copied from H2D, the time consumption of updating the model parameter table is short, and the updating efficiency is high.

[0011] Optionally, the model parameter table is updated according to the basic address table, the address index table and the address offset table, including: the model parameter table is updated by using an address update operator according to the basic address table, the address index table and the address offset table.

[0012] Optionally, before the model parameter table is updated by using the address update operator according to the basic address table, the address index table and the address offset table, the method further includes: obtaining the address update operator. For example, the first chip generates the address update operator, or the first chip receives the address update operator sent by the second chip. The first chip can obtain the address update operator in the model execution stage of the target model.

[0013] Optionally, the address update operator is obtained, including: receiving an address update task sent by the second chip, the address update task including the address update operator.

[0014] Optionally, the base address table is generated by the second chip according to the memory, and the address updating task further comprises the base address table, and the first chip acquires the base address table by the following steps: the first chip acquires the base address table from the address updating task.

[0015] The technical scheme provided in the application can save the overhead of interaction between the first chip and the second chip.

[0016] Optionally, the base address table comprises at least one base address table item, each base address table item is used for recording a base address, each address offset table item is used for recording an address offset, and each memory address table item is used for recording a memory address of the target model, the memory address table item in the model parameter table comprises a first memory address table item, and the model parameter table is updated according to the base address table, the address index table and the address offset table, comprising the following steps: determining a first address index table item in the address index table according to the first memory address table item, the first address index table item corresponds to the first memory address table item; determining a first base address table item in the base address table according to the first address index table item, the first base address table item is the base address table item indicated by the first address index table item; determining a first address offset table item in the address offset table according to the first address index table item, the first address offset table item corresponds to the first address index table item; and updating the memory address recorded in the first memory address table item according to the base address recorded in the first base address table item and the address offset recorded in the first address offset table item.

[0017] Optionally, the base address table comprises at least one base address table item, each base address table item is used for recording a base address, each address offset table item is used for recording an address offset, and each memory address table item is used for recording a memory address of the target model, the memory address table item in the model parameter table comprises a first memory address table item, and the model parameter table is updated according to the base address table, the address index table and the address offset table, comprising the following steps: determining a first address index table item in the address index table according to the first memory address table item, the first address index table item corresponds to the first memory address table item; determining a first base address table item in the base address table according to the first address index table item, the first base address table item is the base address table item indicated by the first address index table item; determining a first address offset table item in the address offset table according to the first address index table item, the first address offset table item corresponds to the first address index table item; and updating the memory address recorded in the first memory address table item according to the base address recorded in the first base address table item and the address offset recorded in the first address offset table item.

[0018] Optionally, the method further comprises the following steps: in a model loading stage of the target model, acquiring the address index table, the address offset table and the model parameter table; and storing the address index table, the address offset table and the model parameter table in the memory of the first chip.

[0019] The technical scheme provided in the application, in a model loading stage of a target model, the first chip stores the address index table, the address offset table and the model parameter table in the memory of the first chip, so that in a model execution stage of the target model, the address index table, the address offset table and the model parameter table are in the memory of the first chip, and thus, after the first chip obtains the base address table of the target model in the model execution stage of the target model, the model parameter table can be updated according to the base address table, the address index table and the address offset table.

[0020] Optionally, the address index table, the address offset table and the model parameter table are generated by the second chip in the model loading stage, and the address index table, the address offset table and the model parameter table are obtained by receiving the address index table, the address offset table and the model parameter table sent by the second chip. The first chip can also generate the address index table, the address offset table and the model parameter table.

[0021] Optionally, the content of the address index table and the content of the address offset table are both unchanged.

[0022] Optionally, the base address table includes at least one base address table entry, each base address table entry is used for recording a base address, the current execution memory includes at least one memory segment, and the base address in the base address table includes a start address of each memory segment in the at least one memory segment.

[0023] Optionally, the at least one address index table entry includes an absolute address index table entry, a memory address table entry corresponding to the absolute address index table entry in the model parameter table is used for recording an unchanged memory address of the target model, the at least one base address table entry includes an absolute base address table entry, the absolute address index table entry is used for indicating the absolute base address table entry, and the base address recorded in the absolute base address table entry is an absolute base address.

[0024] Optionally, the model parameter table includes at least one fixed memory address table entry and at least one non-fixed memory address table entry, the at least one fixed memory address table entry is used for recording a memory address of the target model that does not need to be updated, and the at least one non-fixed memory address table entry is used for recording a memory address of the target model that needs to be updated. That is, the memory address recorded in the memory address table entry in the model parameter table can include the memory address of the target model that needs to be updated and the memory address of the target model that does not need to be updated.

[0025] Optionally, the target model includes at least one operator, the current execution memory includes memory of each operator in the at least one operator, and the memory of each operator includes at least one of operator input memory, operator output memory or operator workspace memory.

[0026] The memory addresses of the target model include memory addresses of each of the at least one operator.

[0027] Optionally, the target model is a neural network model. For example, the target model is a static shape model.

[0028] Optionally, the first chip includes any one of the following: a neural network processing unit (NPU) chip, a graphics processing unit (GPU) chip, a tensor processing unit (TPU) chip, an intelligence processing unit (IPU) chip, and a vector processing unit (VPU) chip.

[0029] These chips have high computing performance, and updating the model parameter table using these chips can improve the updating efficiency of the model parameter table.

[0030] Optionally, the first chip includes at least one computing unit, and the method provided in the first aspect and the optional implementation manners is executed by the computing unit in the first chip. For example, the first chip includes at least one of the following computing units: a cube computing unit, a vector computing unit, a CPU computing unit, a system direct memory access (SDMA) computing unit, and a domain specific accelerator (DSA) in the field of artificial intelligence (AI).

[0031] These computing units have high computing performance, and therefore can improve the updating efficiency of the model parameter table.

[0032] In addition, the process of updating the memory addresses recorded in the different memory address table entries in the model parameter table can be executed in parallel by different computing units in the first chip, which can also improve the updating efficiency of the model parameter table.

[0033] Optionally, the second chip includes a CPU chip.

[0034] In a second aspect, an updating method of a model parameter table is provided and applied to a second chip. The method comprises: in a model execution stage of a target model, generating a base address table of the target model according to a current execution memory of the target model, the current execution memory being a memory allocated in a memory of a first chip for a current execution of the target model; and sending the base address table to the first chip, so that the first chip updates a model parameter table of the target model according to the base address table, an address index table of the target model, and an address offset table of the target model, the address index table, the address offset table, and the model parameter table being in the memory of the first chip, the address index table comprising at least one address index table entry, the address offset table comprising at least one address offset table entry, the model parameter table comprising at least one memory address table entry, each address index table entry corresponding to one address offset table entry, and each address index table entry corresponding to at least one memory address table entry in the model parameter table.

[0035] In the method, the address index table, the address offset table, and the model parameter table are in the memory of the first chip in the model execution stage of the target model.

[0036] In the method, the address index table, the address offset table, and the model parameter table are in the memory of the first chip in the model execution stage of the target model.

[0037] Optionally, the method further comprises: in the model execution stage, allocating the current execution memory in the memory of the first chip. Specifically, in the model execution stage, the second chip allocates the current execution memory for the target model in the memory of the first chip according to a memory allocation strategy. In the present application, the first chip can execute the target model multiple times. In the model execution stage of each execution of the target model by the first chip, the second chip allocates the current execution memory for the target model in the memory of the first chip according to the memory allocation strategy. The memory allocation strategy used by the second chip to allocate the current execution memory for the target model each time is the same.

[0038] Optionally, the method further comprises: in the model execution stage, sending an address updating task to the first chip, the address updating task comprising an address updating operator, so that the first chip updates the model parameter table according to the base address table, the address index table and the address offset table by using the address updating operator. Optionally, the address updating task further comprises the base address table. That is, the second chip sends the address updating operator and the base address table to the first chip in the same address updating task, which can save the overhead of interaction between the first chip and the second chip.

[0039] Optionally, the method further comprises: in the model loading stage of the target model: generating the address index table, the address offset table and the model parameter table; and sending the address index table, the address offset table and the model parameter table to the first chip.

[0040] The technical scheme provided in the present application comprises that the second chip sends the address index table of the target model, the address offset table of the target model and the model parameter table of the target model to the first chip in the model loading stage of the target model, which facilitates the first chip to store the address index table, the address offset table and the model parameter table to the memory of the first chip in the model loading stage, so that in the model execution stage of the target model, the address index table, the address offset table and the model parameter table are in the memory of the first chip, and thus the first chip can update the model parameter table according to the base address table, the address index table and the address offset table after obtaining the base address table of the target model in the model execution stage of the target model.

[0041] Optionally, the content of the address index table and the content of the address offset table are both unchanged.

[0042] Optionally, the base address table comprises at least one base address table entry, each base address table entry is used for recording a base address, the execution memory comprises at least one memory segment, and the base address in the base address table comprises a start address of each memory segment in the at least one memory segment.

[0043] Optionally, the at least one address index table entry comprises an absolute address index table entry, the memory address table entry corresponding to the absolute address index table entry in the model parameter table is used for recording an unchanged memory address of the target model, the at least one base address table entry comprises an absolute base address table entry, the absolute address index table entry is used for indicating the absolute base address table entry, and the base address recorded in the absolute base address table entry is an absolute base address.

[0044] Optionally, the model parameter table includes at least one fixed memory address table item and at least one non-fixed memory address table item, the at least one fixed memory address table item is used to record a memory address of the target model that does not need to be updated, and the at least one non-fixed memory address table item is used to record a memory address of the target model that needs to be updated. That is, the memory addresses recorded in the memory address table items in the model parameter table can include the memory addresses of the target model that need to be updated and the memory addresses of the target model that do not need to be updated.

[0045] Optionally, the target model includes at least one operator, and the current execution memory includes the memory of each operator in the at least one operator, and the memory of each operator includes at least one of an operator input memory, an operator output memory, or an operator workspace memory.

[0046] The memory address of the target model includes the address of the memory of each operator in the at least one operator.

[0047] Optionally, the target model is a neural network model. For example, the target model is a static shape model.

[0048] Optionally, the first chip includes any one of the following: an NPU chip, a GPU chip, a TPU chip, an IPU chip, and a VPU chip. These chips have high computing performance, and updating the model parameter table using these chips can improve the updating efficiency of the model parameter table.

[0049] The second chip includes a CPU chip.

[0050] In a third aspect, an updating device of a model parameter table is provided, and the updating device is applied to a first chip. The updating device includes at least one function module, and the at least one function module is configured to execute the method provided in the first aspect or any optional manner of the first aspect.

[0051] In a fourth aspect, an updating device of a model parameter table is provided, and the updating device is applied to a second chip. The updating device includes at least one function module, and the at least one function module is configured to execute the method provided in the second aspect or any optional manner of the second aspect.

[0052] The function modules described in the third aspect and the fourth aspect can be implemented based on software, hardware, or a combination of software and hardware, and the function modules described in the third aspect and the fourth aspect can be combined or divided in any manner based on specific implementation.

[0053] In a fifth aspect, a computing system is provided, including a first chip and a second chip. The first chip includes the updating device provided in the third aspect, and the second chip includes the updating device provided in the fourth aspect.

[0054] Optionally, the computing system is a system of a computing device.

[0055] In a sixth aspect, a computer readable storage medium is provided, and the computer readable storage medium stores a computer program. The computer program, when executed, implements the method provided in the first aspect or any possible implementation of the first aspect, or implements the method provided in the second aspect or any possible implementation of the second aspect.

[0056] In a seventh aspect, a computer program product is provided, and the computer program product includes a program or code. The program or code, when executed, implements the method provided in the first aspect or any possible implementation of the first aspect, or implements the method provided in the second aspect or any possible implementation of the second aspect.

[0057] The technical effects of the third aspect to the seventh aspect can refer to the technical effects of the first aspect and the second aspect, which will not be repeated here. BRIEF DESCRIPTION OF DRAWINGS

[0058] FIG. 1 is a schematic diagram of a task parameter table;

[0059] FIG. 2 is a schematic diagram of a model parameter table;

[0060] FIG. 3 is a schematic diagram of another model parameter table;

[0061] FIG. 4 is a schematic diagram of a model;

[0062] FIG. 5 is a schematic diagram of scheduling a model in a host scheduling mode;

[0063] FIG. 6 is a timing diagram of scheduling a model in a host scheduling mode;

[0064] FIG. 7 is a schematic diagram of scheduling a model in a sink scheduling mode;

[0065] FIG. 8 is a timing diagram of scheduling a model in a sink scheduling mode and a host scheduling mode;

[0066] FIG. 9 is a flowchart of a method for updating a model parameter table according to an embodiment of the present application;

[0067] FIG. 10 is a schematic diagram of a model parameter table according to an embodiment of the present application;

[0068] FIG. 11 is a schematic diagram of an address index table according to an embodiment of the present application;

[0069] FIG. 12 is a schematic diagram of another address index table according to an embodiment of the present application;

[0070] FIG. 13 is a schematic diagram of an address offset table according to an embodiment of the present application;

[0071] FIG. 14 is a schematic diagram of another address offset table according to an embodiment of the present application;

[0072] FIG. 15 is a schematic diagram of a base address table according to an embodiment of the present application;

[0073] FIG. 16 is a schematic diagram of another base address table according to an embodiment of the present application;

[0074] FIG. 17 is a schematic diagram of an updating model parameter table according to an embodiment of the present application;

[0075] FIG. 18 is a schematic diagram of another updating model parameter table according to an embodiment of the present application;

[0076] FIG. 19 is a flowchart of another method for updating a model parameter table according to an embodiment of the present application;

[0077] FIG. 20 is a schematic diagram of an updating device for a model parameter table according to an embodiment of the present application;

[0078] FIG. 21 is a schematic diagram of another updating device for a model parameter table according to an embodiment of the present application;

[0079] FIG. 22 is a schematic diagram of a computing system according to an embodiment of the present application. DETAILED DESCRIPTION

[0080] The embodiments of the present application will be further described in detail below with reference to the accompanying drawings.

[0081] For the convenience of understanding, the terms involved in the embodiments of the present application will be introduced first.

[0082] 1. Task argument table (task args table).

[0083] A table used to record the memory address of an operator. Each operator in a model has a task argument table, and the task argument table of each operator is used to record the memory address of the operator. The model can be deployed in a computing device including a central processing unit (CPU) chip and a model execution chip, and the model can be executed by the model execution chip. Operators usually exist in the form of tasks on the model execution chip, and therefore the table used to record the memory address of an operator is called a task argument table. The task argument table is usually stored in the memory of the model execution chip, and the memory used to store the task argument table is usually continuous. That is, the task argument table is stored in the continuous memory in the model execution chip.

[0084] The memory address of an operator is the address of the memory of the operator. The memory of an operator is the memory allocated for the operator in the memory of the model execution chip for the operator to execute. The memory of an operator includes operator input memory, operator output memory, operator workspace memory, etc., and the memory address of an operator includes the address of the operator input memory of the operator, the address of the operator output memory of the operator, the address of the operator workspace memory of the operator, etc. The operator input memory is the memory of the input of the operator, i.e., the memory allocated for the input of the operator, and is used to store the input data required for the execution of the operator. The operator output memory is the memory of the output of the operator, i.e., the memory allocated for the output of the operator, and is used to store the output data generated by the execution of the operator. The operator workspace memory is used to provide a place for information exchange for the execution of the operator. Generally, an operator can have at least one input and at least one output, and memory needs to be allocated for each input and each output of the operator, and thus an operator generally has at least one operator input memory and at least one operator output memory. The number of memories of an operator and the arrangement rule of the memory addresses of the operator in the task parameter table of the operator are determined by the operator.

[0085] For example, an operator has two inputs 1-2 and two outputs 1-2, the memory of the input 1 is the operator input memory 1, the memory of the input 2 is the operator input memory 2, the memory of the output 1 is the operator output memory 1, the memory of the output 2 is the operator output memory 2, and the memory of the operator further includes two operator workspace memories 1-2 and other memories. The address of the operator input memory 1 is the input address 1, the address of the operator input memory 2 is the input address 2, the address of the operator output memory 1 is the output address 1, the address of the operator output memory 2 is the output address 2, the address of the operator workspace memory 1 is the workspace address 1, and the address of the operator workspace memory 2 is the workspace address 2. Assuming that the arrangement order of the memory addresses of the operator in the task parameter table of the operator is input address, output address, workspace address, and other address in sequence, the task parameter table of the operator is shown in FIG. 1, and the task parameter table is in continuous memory.

[0086] 2. Model args table.

[0087] A model includes at least one operator, and the task parameter tables of the operators in the model are collectively referred to as a model parameter table. It can be understood that the model parameter table includes at least one task parameter table, the content of the model parameter table is the memory address of the operator in the model, and the model parameter table is stored in the memory of the model execution chip. The memory for storing different task parameter tables can be continuous or discontinuous. That is, the different task parameter tables included in the model parameter table can be stored in continuous memory of the model execution chip or in discontinuous memory of the model execution chip.

[0088] For example, a model has N operators, N is a positive integer. Assuming that the task parameter tables of the operators 1 to N are task parameter tables 1 to N, the model parameter table of the model is shown in FIG. 2 or FIG. 3. As shown in FIG. 2, the task parameter tables 1 to N are in continuous memory. As shown in FIG. 3, the memory where any two of the task parameter tables 1 to N are located is discontinuous, and the task parameter tables 1 to N are in discontinuous memory. The structure of each of the task parameter tables 1 to N can refer to FIG. 1.

[0089] 3. Model I / O (model I / O) memory.

[0090] The input / output (input / output) memory of a model is referred to as model I / O memory. The model I / O memory includes model input memory and model output memory. The model input memory is the memory of the input of the model (i.e., the memory allocated for the input of the model), and is used to store the input data required for the execution of the model. The model output memory is the memory of the output of the model (i.e., the memory allocated for the output of the model), and is used to store the output data generated by the execution of the model. Generally, a model can have at least one input and at least one output, and memory needs to be allocated for each input and each output of the model. Therefore, a model generally has at least one model input memory and at least one model output memory. If a model has multiple inputs and multiple outputs, it is referred to as a model with multiple model I / Os. If the model I / O memory changes compared with the last execution of the model, it is referred to as a model I / O with changes.

[0091] The operators in a model can include input operators, output operators, and intermediate operators, and the intermediate operators are connected between the input operators and the output operators. The operator input memory of an input operator can be the model input memory. The operator output memory of an output operator can be the model output memory. For example, FIG. 4 is a schematic diagram of a model, which includes operators A to D, the output 1 of operator A is connected to the input 1 of operator B, the output 2 of operator A is connected to the input 1 of operator C, the output 1 of operator B is connected to the input 1 of operator D, and the output 1 of operator C is connected to the input 2 of operator D. Operator A is the input operator of the model, operator D is the output operator of the model, and operators B and C are the intermediate operators of the model. The operator input memory of operator A can be the model input memory of the model, and the operator output memory of operator D can be the model output memory of the model. It should be noted that FIG. 4 is only an example of a model, and a model can include one or more input operators and one or more output operators. The model I / O memory is generally allocated by a front-end framework, which can be software running on a CPU chip, and the embodiments of the present application do not limit this.

[0092] 4. Feature map (FM) memory.

[0093] The memory of the operators in the model (including operator input memory, operator output memory, operator workspace memory, etc.) except for the operator input memory of the input operator (i.e., model input memory) and the operator output memory of the output operator (i.e., model output memory) is collectively referred to as FM memory. That is, the operator output memory of the input operator, the operator workspace memory of the input operator, the operator input memory of the output operator, the operator workspace memory of the output operator, the operator input memory of the intermediate operator, the operator output memory of the intermediate operator, and the operator workspace memory of the intermediate operator in the model are collectively referred to as FM memory. For example, as shown in FIG. 4, the operator output memory of operator A (including the memory of output 1 of operator A and the memory of output 2 of operator A), the operator workspace memory of operator A, the operator input memory of operator D (including the memory of input 1 of operator D and the memory of input 2 of operator D), the operator workspace memory of operator D, the operator input memory of operator B (e.g., the memory of input 1 of operator B), the operator output memory of operator B (e.g., the memory of output 1 of operator B), the operator workspace memory of operator B, the operator input memory of operator C (e.g., the memory of input 1 of operator C), the operator output memory of operator C (e.g., the memory of output 1 of operator C), and the operator workspace memory of operator C are collectively referred to as FM memory. The FM memory can be a continuous memory segment (i.e., a continuous memory segment), or can include multiple non-continuous memory segments. That is, the memory of the operators in the model, except for the operator input memory of the input operator and the operator output memory of the output operator, can be continuous or discontinuous. Different memories of the same operator can be continuous or discontinuous. The FM memory can be planned as continuous memory or non-continuous memory according to a certain memory allocation strategy. In addition, the memories of different operators in the model can overlap (or be referred to as multiplexing), for example, as shown in FIG. 4, the memory of output 1 of operator A and the memory of input 1 of operator B can be the same memory, the memory of output 2 of operator A and the memory of input 1 of operator C can be the same memory, the memory of output 1 of operator B and the memory of input 1 of operator D can be the same memory, and the memory of output 1 of operator C and the memory of input 2 of operator D can be the same memory. The multiplexing of the memories of different operators in the model can also be implemented based on other multiplexing strategies.

[0094] The address of the memory of an operator can be obtained by offsetting the starting address of the memory segment where the memory is located by the address offset corresponding to the memory. The starting address of a memory segment is also referred to as a base address, and the starting address of an FM memory segment is also referred to as an FM memory base address or an FM base address. For example, for any intermediate operator in a model, assuming that the operator input memory 1 of the operator, the operator output memory 1 of the operator, and the operator workspace memory 1 of the operator are in the same FM memory segment, the starting address of the FM memory segment is FM base address B1, the address offset corresponding to the operator input memory 1 is "address offset 1", the address offset corresponding to the operator output memory 1 is "address offset 2", and the address offset corresponding to the operator workspace memory 1 is "address offset 3", the address of the operator input memory 1, the address of the operator output memory 1, and the address of the operator workspace memory 1 can be expressed as follows.

[0095] The address of the operator input memory 1 = FM base address B1 + address offset 1.

[0096] The address of the operator output memory 1 = FM base address B1 + address offset 2.

[0097] The address of the operator workspace memory 1 = FM base address B1 + address offset 3.

[0098] The symbol "+" in the above expressions represents offsetting the corresponding address offset from the FM base address B1. For example, "The address of the operator input memory 1 = FM base address B1 + address offset 1" means that the address of the operator input memory 1 is offset from the FM base address B1 by "address offset 1", and the address of the operator input memory 1 is obtained by offsetting the FM base address B1 by "address offset 1".

[0099] 5. Zero-copy.

[0100] If the input of a certain operator in a model (referred to as op input1) is the same input as a certain input of the model (referred to as model input1), and the memory (referred to as model input memory) allocated by the front-end framework for model input1 is the memory of the model execution chip, the operator input memory of op input1 and the model input memory of model input1 are the same memory, and the address of the operator input memory of op input1 can be directly taken as the address of the model input memory of model input1, and the address of the operator input memory of op input1 can be directly taken as the address of the model input memory of model input1.

[0101] If the output of an operator in a model (e.g., referred to as op output1) is the same output as an output of the model (e.g., referred to as model output1), and the memory allocated by the front-end framework for model output1 (i.e., model output memory) is the memory of the model execution chip, then the operator output memory of op output1 is the same memory as the model output memory of model output1, and the address of the operator output memory of op output1 can be directly taken as the address of the model output memory of model output1, and it is only necessary to update the address of the model output memory of model output1 as the address of the operator output memory of op output1 in the task parameter table of the operator.

[0102] It can be seen that in the scenario where the input of an operator is the input of a model, the output of the operator is the output of the model, and the memory of the input of the model and the memory of the output of the model are the memory of the model execution chip, the address of the operator input memory can be directly used as the address of the model input memory, and the address of the operator output memory can be directly used as the address of the model output memory by updating the task parameter table of the operator, without the need for copying the address between the CPU chip and the model execution chip, which is referred to as zero-copy. The copying between the CPU chip and the model execution chip is also referred to as host memory to device memory (H2D) copying.

[0103] 6. Address updating (or referred to as address refreshing).

[0104] The action of updating the address in the task parameter table is referred to as address updating. Zero-copy is a kind of address updating. Updating the address in the task parameter table by the start address and the address offset of the memory segment is also a kind of address updating. The model parameter table is composed of the task parameter tables of the operators in the model, and obviously, updating the address in the model parameter table is also address updating.

[0105] 7. Model execution chip.

[0106] The chip for executing the model can also be referred to as an artificial intelligence (AI) chip. The model execution chip includes a memory and a computing unit, which is also referred to as a computing resource, an accelerator, or a processing core, etc. The computing unit in the model execution chip can include at least one of a cube computing unit, a vector computing unit, a CPU computing unit, a system direct memory access (SDMA) computing unit, and a domain specific accelator (DSA) in the field of AI. The model execution chip can also include other computing units, which are not limited by embodiments of the present application. In the process of executing the model by the model execution chip, different operators in the model can be executed by different computing units in the model execution chip, and different computing units can execute the operators in the model in parallel.

[0107] The model can be deployed in a computing device including a CPU chip and a model execution chip. The CPU chip is used to allocate memory for the model in the memory of the model execution chip (for example, the memory allocated for the model is referred to as the memory of the model), and schedule the model to the model execution chip for execution. The model execution chip is used to determine the memory of the model based on the model parameter table of the model, and execute the model based on the memory of the model. It is not difficult to understand that the model parameter table is located in the memory of the model execution chip, and the model parameter table is used to record the address of the memory of the model. The memory of the model includes a model input memory, a model output memory, and an FM memory.

[0108] The scheduling mode of the model generally includes a host scheduling mode and a sink scheduling mode.

[0109] In the host scheduling mode, the CPU chip sends the operators in the model to the execution flow in the model execution chip in sequence; the model execution chip executes the operators in the execution flow in sequence according to the order of the operators in the execution flow, so that the model execution chip executes the operators in the model in sequence according to the order of the operators in the execution flow. For example, FIG. 5 is a schematic diagram of scheduling the model shown in FIG. 4 in the host scheduling mode. As shown in FIG. 5, the CPU chip sends the operators A-D in the model to the execution flow in the model execution chip in sequence, and the computing unit in the model execution chip executes the operators in the execution flow in sequence according to the order of the operators in the execution flow, so that the computing unit in the model execution chip executes the operators A-D in sequence according to the order of the operators A-D in the execution flow. It should be noted that in the host scheduling mode, the process of the CPU chip sending the model (specifically, the operators in the model) to the model execution chip is synchronous with the process of the model execution chip executing the model (specifically, the operators in the model). Moreover, each time the model is executed, the CPU chip needs to traverse all the operators in the model and send all the operators in the model to the model execution chip.

[0110] However, in the host scheduling mode, if the speed of the model execution chip executing the operators is faster than the speed of the CPU chip sending the operators to the model execution chip, there may be time gaps between the processes of the model execution chip executing different operators in the model, which will cause the end-to-end (E2E) time of executing the model to be large. For example, FIG. 6 is a timing diagram of scheduling the model shown in FIG. 4 in the host scheduling mode. In FIG. 6, A_l_t represents the time consumed by the CPU chip in sending the operator A to the model execution chip, B_l_t represents the time consumed by the CPU chip in sending the operator B to the model execution chip, C_l_t represents the time consumed by the CPU chip in sending the operator C to the model execution chip, and D_l_t represents the time consumed by the CPU chip in sending the operator D to the model execution chip; A_e_t represents the time consumed by the model execution chip in executing the operator A, B_e_t represents the time consumed by the model execution chip in executing the operator B, C_e_t represents the time consumed by the model execution chip in executing the operator C, and D_e_t represents the time consumed by the model execution chip in executing the operator D. It can be seen that the time of the CPU chip sending the operators A-D to the model execution chip is continuous, but since the speed of the model execution chip executing the operators is faster than the speed of the CPU chip sending the operators to the model execution chip, there is a time gap between the time period of the model execution chip executing the operator A and the time period of the model execution chip executing the operator B, there is a time gap between the time period of the model execution chip executing the operator B and the time period of the model execution chip executing the operator C, and there is a time gap between the time period of the model execution chip executing the operator C and the time period of the model execution chip executing the operator D, which causes the E2E time of executing the model to be large.

[0111] In the sinking scheduling mode, the process of the scheduling model includes a model loading stage (also referred to as a model sinking stage) and a model execution stage. In the model loading stage, the CPU chip sends the operators in the model to the model stream corresponding to the model in the model execution chip in sequence. In the model execution stage, the CPU chip sends the model execution task of the model to the execution stream in the model execution chip, and the model execution chip executes the model stream corresponding to the model (i.e., executes the operators in the model stream in sequence according to the order of the operators in the model stream) when the model execution task in the execution stream is scheduled. If the model needs to be executed multiple times, the CPU chip sends the model execution task of the model to the execution stream in the model execution chip multiple times, and the model execution chip executes the model multiple times according to the model execution tasks sent multiple times by the CPU chip. For example, FIG. 7 is a schematic diagram of scheduling the model shown in FIG. 4 in the sinking scheduling mode. As shown in FIG. 7, in the model loading stage, the CPU chip sends the operators A-D in the model to the model stream in the model execution chip in sequence. In the model execution stage, the CPU chip sends the model execution task E of the model to the execution stream in the model execution chip, the computing unit in the model execution chip schedules the model execution task E according to the order of the model execution task E in the execution stream, and the computing unit in the model execution chip executes the operators A-D in the model stream in sequence according to the order of the operators A-D in the model stream when the model execution task E is scheduled. It should be noted that in the sinking scheduling mode, the model execution chip does not execute the model immediately after the CPU chip sends the model (specifically, the operators in the model) to the model execution chip, but executes the model under the trigger of the model execution task. Moreover, each time the model is executed, the CPU chip only needs to send the model execution task of the model to the model execution chip, without traversing all the operators in the model and sending all the operators in the model to the model execution chip, which makes the E2E time of executing the model in the sinking scheduling mode shorter than the E2E time of executing the model in the host scheduling mode. For example, FIG. 8 is a timing comparison diagram of scheduling the model shown in FIG. 4 in the sinking scheduling mode and scheduling the model shown in FIG. 4 in the host scheduling mode. In FIG. 8, m_l_t represents the time consumed by the CPU chip for sending the model execution task of the model to the model execution chip, and the meanings of A_l_t, B_l_t, C_l_t, D_l_t, A_e_t, B_e_t, C_e_t, and D_e_t are described with reference to FIG. 6 and related descriptions. As shown in FIG. 8, compared with the host scheduling mode, the sinking scheduling mode has a head overhead m_l_t. In the sinking scheduling mode, the CPU chip does not participate in scheduling after sending the model execution task to the model execution chip in the model execution stage. Compared with the host scheduling mode, the E2E time of executing the model in the sinking scheduling mode is shorter, and the sinking scheduling mode has certain benefits in terms of the time consumed for executing the model.

[0112] In the sinking scheduling mode, in order to avoid the model occupying the memory of the model execution chip in the whole life cycle (for example, the period from when the model is loaded to the model execution chip to when the model is unloaded from the model execution chip), the memory of the model is usually reclaimed after the model execution chip finishes executing the model. Before the model execution chip executes the model next time (specifically, in the model execution stage of executing the model next time), the memory of the model is allocated in the memory of the model execution chip again, and the model parameter table of the model is updated according to the memory allocated to the model again. In this case, the m_l_t described above includes the time consumption of the CPU chip sending the model execution task to the model execution chip, and also includes the time consumption of updating the model parameter table. The shorter the time consumption of updating the model parameter table, the smaller the m_l_t, and the smaller the E2E time consumption of executing the model.

[0113] At present, after the model parameter table is usually updated by the CPU chip, the updated model parameter table is copied from the memory of the CPU chip to the memory of the model execution chip. Specifically, the model parameter table includes the task parameter table of the operator in the model, and the CPU chip traverses the operators in the model in sequence. For each operator traversed: the CPU chip creates the task parameter table of the operator in the memory of the CPU chip, the CPU chip updates the task parameter table in the memory of the CPU chip according to the memory allocated to the operator in the memory of the model execution chip again, the CPU chip copies the updated task parameter table in the memory of the CPU chip to the memory of the model execution chip, so that the updated task parameter table covers the task parameter table of the operator in the memory of the model execution chip, and the update of the task parameter table in the memory of the model execution chip is realized. However, the current scheme of updating the model parameter table needs to copy the model parameter table between the CPU chip and the model execution chip (that is, needs to perform H2D copy of the model parameter table), and needs to traverse the operators of the model, which leads to long time consumption and low efficiency of updating the model parameter table.

[0114] The embodiment of the present application provides a technical solution, which is suitable for updating a model parameter table in a sinking scheduling mode. In the technical solution provided by the embodiment of the present application, in a model execution stage of a target model, a memory of a first chip includes an address index table of the target model, an address offset table of the target model and a model parameter table of the target model. In the model execution stage, after the first chip obtains a basic address table of the target model, the model parameter table can be updated according to the basic address table, the address index table and the address offset table. In this way, in the process of updating the model parameter table of the target model in the model execution stage of the target model, the model parameter table does not need to be copied from H2D, and the target model does not need to be traversed by an operator. Therefore, the time consumption of updating the model parameter table is shorter, and the updating efficiency is higher. The basic address table is generated according to a current execution memory of the target model, and the current execution memory is a memory allocated in the memory of the first chip for current execution of the target model. The basic address table includes at least one basic address table item, and each basic address table item is used for recording a basic address. The basic address is an address of the memory of the first chip. The address index table includes at least one address index table item, and each address index table item is used for recording an address index. Each address index table item (specifically, the address index recorded by the address index table item) is used for indicating a basic address table item in the basic address table. The address offset table includes at least one address offset table item, and each address offset table item is used for recording an address offset. The at least one address index table item and the at least one address offset table item correspond to each other. The model parameter table includes at least one memory address table item, and each memory address table item is used for recording a memory address of the target model. Each address index table item in the at least one address index table item corresponds to at least one memory address table item in the model parameter table. In each memory address table item, the memory address is obtained by offsetting the address offset amount in the corresponding address offset table item from the basic address in the basic address table item indicated by the corresponding address index table item.

[0115] The first chip can be a model execution chip, and the first chip includes a calculation unit. The process of updating the model parameter table can be performed by the calculation unit in the first chip. For example, the first chip includes at least one of the following calculation units: a cube calculation unit, a vector calculation unit, a CPU calculation unit, an SDMA calculation unit, and an AIDS A. The first chip includes any one of the following: a neural network processing unit (NPU) chip, a graphics processing unit (GPU) chip, a tensor processing unit (TPU) chip, an intelligence processing unit (IPU) chip, and a vector processing unit (VPU) chip. The first chip can also include other calculation units, and the first chip can also be any other possible chip. The embodiments of the present application do not limit this. It should be noted that the operator usually exists in the form of one or more tasks in the model execution chip. In order to facilitate the description, the operator is still described as an operator when the related description of the operator in the model execution chip is involved.

[0116] The method embodiment of the present application will be described below.

[0117] Please refer to FIG. 9, which shows a flowchart of a method for updating a model parameter table according to an embodiment of the present application. The updating method is performed by a first chip, specifically by a calculation unit in the first chip. The updating method includes the following steps S901 to S904. The first chip performs S901 to S902 in the model loading stage of a target model. The first chip performs S903 to S904 in the model execution stage of the target model.

[0118] S901. The first chip obtains a model parameter table T0 of a target model, an address index table T1 of the target model, and an address offset table T2 of the target model. The model parameter table T0 includes at least one memory address table item. The address index table T1 includes at least one address index table item. The address offset table T2 includes at least one address offset table item. Each address index table item in the at least one address index table item corresponds to at least one memory address table item in the model parameter table T0.

[0119] The model parameter table T0 includes at least one memory address table item, and each memory address table item is used to record a memory address of the target model. The memory address of the target model is an address of a memory of the target model, and the memory of the target model is a memory allocated for the target model in the memory of the first chip for the target model to execute. The memory of the target model can include a model input memory, a model output memory and an FM memory. The target model includes at least one operator, the memory of the target model includes a memory of each operator in the at least one operator, and the memory of the operator includes an operator input memory, an operator output memory, an operator workspace memory and the like. The at least one operator can include an input operator and an output operator, and the FM memory includes a memory of the at least one operator, except for the operator input memory of the input operator (the operator input memory of the input operator is the model input memory) and the operator output memory of the output operator (the operator output memory of the output operator is the model output memory). In an optional embodiment, the target model is a static shape model. The size of the operator input memory, the operator output memory, the operator workspace memory and the like required by each operator in the static shape model is fixed and unchangeable. In an optional embodiment, the memory of the target model includes at least one memory segment (or referred to as a memory block), and the memory of the target model can be continuous or discontinuous. For example, the memory of the target model is a continuous memory segment, or the memory of the target model includes discontinuous multiple memory segments. The number of memory segments included in the memory of the target model and the continuity of the memory of the target model depend on a memory allocation strategy adopted for allocating the memory of the target model. In the embodiments of the present application, the first chip can execute the target model multiple times, and the memory of the target model for the target model to execute each time can be allocated in the memory of the first chip according to the memory allocation strategy in a model execution phase of the first chip executing the target model each time, and the memory allocation strategy adopted each time for allocating the memory of the target model is the same.

[0120] It is understandable that the model parameter table T0 is used to record the addresses of the model input memory, the addresses of the model output memory, the addresses of the operator input memory, the addresses of the operator output memory, the addresses of the operator workspace memory, and the like of the target model. The bit width of the memory address recorded in each memory address table entry in the model parameter table T0 can be 32 bits or 64 bits. As an example, FIG. 10 is a schematic diagram of a model parameter table T0 provided by an embodiment of the present application. As shown in FIG. 10, the model parameter table T0 includes k memory address table entries 1 to k, where k is a positive integer. The memory address table entries 1 to k are used to record memory addresses A1 to Ak, which can include the addresses of the model input memory, the addresses of the model output memory, the addresses of the operator input memory, the addresses of the operator output memory, the addresses of the operator workspace memory, and the like of the target model. For example, the target model is as shown in FIG. 4, and without considering the operator workspace memory, the memory address table entry 1 is used to record the address of the model input memory (e.g., the memory of the input of the operator A) of the target model, for example, the memory address A1 is the address of the model input memory of the target model; the memory address table entry 2 is used to record the address of the memory of the output 1 of the operator A, for example, the memory address A2 is the address of the memory of the output 1 of the operator A; the memory address table entry 3 is used to record the address of the memory of the output 2 of the operator A, for example, the memory address A3 is the address of the memory of the output 2 of the operator A; the memory address table entry 4 is used to record the address of the memory of the input 1 of the operator B, for example, the memory address A4 is the address of the memory of the input 1 of the operator B; the memory address table entry 5 is used to record the address of the memory of the output 1 of the operator B, for example, the memory address A5 is the address of the memory of the output 1 of the operator B; the memory address table entry 6 is used to record the address of the memory of the input 1 of the operator C, for example, the memory address A6 is the address of the memory of the input 1 of the operator C; the memory address table entry 7 is used to record the address of the memory of the output 1 of the operator C, for example, the memory address A7 is the address of the memory of the output 1 of the operator C; and so on. In an optional embodiment, the memory of the output 1 of the operator A and the memory of the input 1 of the operator B are the same memory, the memory of the output 2 of the operator A and the memory of the input 1 of the operator C are the same memory, the memory of the output 1 of the operator B and the memory of the input 1 of the operator D are the same memory, and the memory of the output 1 of the operator C and the memory of the input 2 of the operator D are the same memory.

[0121] In an optional embodiment, the memory address of the target model recorded in the model parameter table T0 includes a fixed memory address and a non-fixed memory address. The fixed memory address is a memory address that does not change, and the non-fixed memory address is a memory address that can change. For example, the fixed memory address is an address that does not change during the lifetime of the target model on the first chip, and the non-fixed memory address is an address that can change (e.g., due to re-allocation of memory for the target model) during the lifetime of the target model on the first chip. The lifetime of the target model on the first chip can be the period of time from when the target model is loaded onto the first chip to when the target model is unloaded from the first chip. The fixed memory address of the target model can indicate a location in the memory of the first chip for storing the model parameter table T0, or can indicate a separate block of memory of the first chip. The memory address entry in the model parameter table T0 for recording the fixed memory address of the target model can be referred to as a fixed memory address entry, and the memory address entry in the model parameter table T0 for recording the non-fixed memory address of the target model can be referred to as a non-fixed memory address entry. In an optional embodiment, the model parameter table T0 includes at least one fixed memory address entry and at least one non-fixed memory address entry. For example, the memory address entries 1-k shown in FIG. 10 include at least one fixed memory address entry and at least one non-fixed memory address entry. The at least one fixed memory address entry can be distributed continuously in the model parameter table T0, and the at least one non-fixed memory address entry can be distributed continuously in the model parameter table T0. For example, in the memory address entries 1-k, the memory address entries 1-x are non-fixed memory address entries, and the memory address entries x+1-k are fixed memory address entries, where x is a positive integer less than k. Alternatively, the at least one fixed memory address entry and the at least one non-fixed memory address entry can be distributed mixedly (or alternatively, distributed intermittently) in the model parameter table T0. For example, the memory address entries 1, 3, 5, and 8 are fixed memory address entries, and the memory address entries 2, 4, 6, 7, and 9 are non-fixed memory address entries. The present embodiments do not limit the distribution of the fixed memory address entries and the non-fixed memory address entries in the model parameter table T0.

[0122] In the embodiments of the present application, the model parameter table T0 can be updated in full or in part. Updating the model parameter table T0 in full means that all memory addresses recorded in the model parameter table T0 (including fixed memory addresses and non-fixed memory addresses) are updated. Updating the model parameter table T0 in part means that only part of the memory addresses recorded in the model parameter table T0 are updated, for example, the non-fixed memory addresses recorded in the model parameter table T0 are updated, and the fixed memory addresses recorded in the model parameter table T0 are not updated. In the embodiment of updating the model parameter table T0 in full, the fixed memory addresses recorded in the model parameter table T0 remain unchanged before and after the update, and the non-fixed memory addresses recorded in the model parameter table T0 can be different before and after the update. In the embodiment of updating the model parameter table T0 in part, the non-fixed memory addresses recorded in the model parameter table T0 can be different before and after the update, which is not limited in the embodiments of the present application.

[0123] The address index table T1 includes at least one address index table item, each address index table item being used to record an address index. Each address index table item in the at least one address index table item corresponds to at least one memory address table item in the model parameter table T0, different address index table items in the at least one address index table item correspond to different memory address table items in the model parameter table T0, and the address index recorded in each address index table item is used to update the memory address recorded in the memory address table item corresponding to the address index table item. Specifically, each address index table item is used to indicate one base address table item in the base address table of the target model (specifically, the address index recorded in the address index table item is used to indicate one base address table item in the base address table of the target model, and the address index recorded in the address index table item can be the index of the base address table item indicated by the address index), the base address table item is used to record a base address, and the base address recorded in the base address table item indicated by each address index table item is used to update the memory address recorded in the memory address table item corresponding to the address index table item. The base address recorded in the base address table of the target model can include the starting address of the memory segment of the target model, or can include an absolute base address. The absolute base address can be "0", or can be another absolute base address. The base address table item used to record the absolute base address in the base address table can be referred to as an absolute base address table item. The base address table item used to record a non-absolute base address in the base address table can be referred to as a non-absolute base address table item. In the embodiments of the present application, the address indexes recorded in different address index table items in the at least one address index table item can be the same or different, and the number of different address indexes recorded in the at least one address index table item is determined according to the number of memory segments included in the memory of the target model. For example, the number of different address indexes recorded in the at least one address index table item is equal to the number of memory segments included in the memory of the target model, and each address index recorded in the at least one address index table item can be the index of one memory segment included in the memory of the target model.

[0124] In an embodiment of the present application, the address index table entries in the address index table T1 correspond to the memory address table entries in the model parameter table T0 according to a first correspondence. In one embodiment, at least one address index table entry in the address index table T1 corresponds to at least one memory address table entry in the model parameter table T0 according to the first correspondence. As an example, the first correspondence is that the i-th address index table entry in the address index table T1 corresponds to the i-th memory address table entry in the model parameter table T0, where i is a positive integer, i is not greater than the total number of address index table entries in the address index table T1, and i is not greater than the total number of memory address table entries in the model parameter table T0. That is, the 1st address index table entry in the address index table T1 corresponds to the 1st memory address table entry in the model parameter table T0, the 2nd address index table entry in the address index table T1 corresponds to the 2nd memory address table entry in the model parameter table T0, the 3rd address index table entry in the address index table T1 corresponds to the 3rd memory address table entry in the model parameter table T0, and so on. As another example, the first correspondence is that the 1st address index table entry in the address index table T1 corresponds to the 3rd memory address table entry in the model parameter table T0, the 2nd address index table entry in the address index table T1 corresponds to the 5th memory address table entry in the model parameter table T0, the 3rd address index table entry in the address index table T1 corresponds to the 6th memory address table entry in the model parameter table T0, and so on. In another embodiment, the address index table entries in the address index table T1 correspond to the memory address table entries in the model parameter table T0 according to the first correspondence, and each address index table entry in the address index table T1 corresponds to at least one memory address table entry in the model parameter table T0. As an example, the first correspondence is that the 1st address index table entry in the address index table T1 corresponds to the 2nd and 3rd memory address table entries in the model parameter table T0, the 2nd address index table entry in the address index table T1 corresponds to the 1st memory address table entry in the model parameter table T0, the 3rd address index table entry in the address index table T1 corresponds to the 4th, 6th and 7th memory address table entries in the model parameter table T0, and so on. The first correspondence can also be other correspondence, and the embodiments of the present application do not limit the content and form of the first correspondence.

[0125] In an optional embodiment, the model parameter table T0 is updated in full in the embodiments of the present application. Taking an example of the memory address entries in the model parameter table T0 including at least one fixed memory address entry and at least one non-fixed memory address entry, in the embodiment of updating the model parameter table T0 in full, the at least one address index entry in the address index table T1 includes at least one absolute address index entry and at least one non-absolute address index entry. Each of the at least one absolute address index entry corresponds to at least one fixed memory address entry in the model parameter table T0, and each of the at least one non-absolute address index entry corresponds to at least one non-fixed memory address entry in the model parameter table T0. The address index recorded in the absolute address index entry is referred to as an absolute address index, and the address index recorded in the non-absolute address index entry is referred to as a non-absolute address index. In one embodiment of updating the model parameter table T0 in full, the at least one address index entry in the address index table T1 corresponds to the at least one memory address entry in the model parameter table T0 one by one, the at least one address index entry is all the address index entries in the address index table T1, the at least one memory address entry is all the memory address entries in the model parameter table T0, the size of the address index table T1 is equal to the size of the model parameter table T0, that is, the number of address index entries included in the address index table T1 is equal to the number of memory address entries included in the model parameter table T0. In another embodiment of updating the model parameter table T0 in full, the at least one address index entry in the address index table T1 corresponds to all the memory address entries in the model parameter table T0, and there are some address index entries in the at least one address index entry corresponding to multiple memory address entries in the model parameter table T0 (for example, a certain address index entry in the at least one address index entry corresponds to multiple memory address entries in the model parameter table T0), the at least one address index entry is all the address index entries in the address index table T1, the size of the address index table T1 is less than the size of the model parameter table T0, that is, the number of address index entries included in the address index table T1 is less than the number of memory address entries included in the model parameter table T0.

[0126] As an example, FIG. 11 is a schematic diagram of an address index table T1 according to an embodiment of the present application. The address index table T1 shown in FIG. 11 is used for full quantity updating of the model parameter table T0. As shown in FIG. 11, the address index table T1 includes q address index entries 1 to q, q being a positive integer less than or equal to k. Each of the address index entries 1 to q corresponds to at least one of the memory address entries in the model parameter table T0 shown in FIG. 10, and each of the address index entries 1 to q records an address index, which is used to update the memory address recorded in the memory address entry corresponding to the address index entry. In one example, q = k, and the address index entries 1 to q correspond to the memory address entries 1 to k in the model parameter table T0 shown in FIG. 10. For example, the address index entry 1 corresponds to the memory address entry 1, and the address index "1" recorded in the address index entry 1 is used to update the memory address recorded in the memory address entry 1. The address index entry 2 corresponds to the memory address entry 2, and the address index "2" recorded in the address index entry 2 is used to update the memory address recorded in the memory address entry 2. The address index entry 9 corresponds to the memory address entry 9, and the address index "G" recorded in the address index entry 9 is used to update the memory address recorded in the memory address entry 9. Similarly, the address index recorded in each of the address index entries is used to indicate a base address entry in the base address table of the target model, and the base address recorded in the base address entry indicated by the address index recorded in each of the address index entries is used to update the memory address recorded in the memory address entry corresponding to the address index entry. For example, the address index "1" recorded in the address index entry 1 is used to indicate the first base address entry in the base address table of the target model, and the base address recorded in the first base address entry is used to update the memory address recorded in the memory address entry 1. The address index "2" recorded in the address index entry 2 is used to indicate the second base address entry in the base address table of the target model, and the base address recorded in the second base address entry is used to update the memory address recorded in the memory address entry 2. The address index "G" recorded in the address index entry 9 is used to indicate the absolute base address entry (e.g., the last entry in the base address table of the target model) in the base address table of the target model, and the absolute base address recorded in the absolute base address entry is used to update the memory address recorded in the memory address entry 9. Similarly, the address index recorded in each of the address index entries is used to indicate a base address entry in the base address table of the target model, and the base address recorded in the base address entry indicated by the address index recorded in each of the address index entries is used to update the memory address recorded in the memory address entry corresponding to the address index entry. In another example, q is less than k, and some of the address index entries 1 to q correspond to multiple memory address entries in the model parameter table T0. For example, the address index entry 1 corresponds to the memory address entry 2 and the memory address entry 3, and the address index "1" recorded in the address index entry 1 is used to update the memory address recorded in the memory address entry 2 and the memory address recorded in the memory address entry 3.The address index table item 2 corresponds to the memory address table item 1, and the address index "2" recorded in the address index table item 2 is used to update the memory address recorded in the memory address table item 1. The address index table item 3 corresponds to the memory address table item 4, the memory address table item 6 and the memory address table item 7, and the address index "1" recorded in the address index table item 3 is used to update the memory address recorded in the memory address table item 4, the memory address recorded in the memory address table item 6 and the memory address recorded in the memory address table item 7. Similarly, the address index table item 4 corresponds to the memory address table item 5, the memory address table item 8 and the memory address table item 9, and the address index "2" recorded in the address index table item 4 is used to update the memory address recorded in the memory address table item 5, the memory address recorded in the memory address table item 8 and the memory address recorded in the memory address table item 9. It should be noted that in the address index table T1 shown in FIG. 11, the address index table items 1-8 and k are non-absolute address index table items, the address indexes "1", "2", "3", "4", "5", "6" and the like are non-absolute address indexes, the address index table item 9 is an absolute address index table item, and the address index "G" is an absolute address index.

[0127] In an optional embodiment, the model parameter table T0 is partially updated in the embodiments of the present application. Taking an example in which the memory address table items in the model parameter table T0 include at least one fixed memory address table item and at least one non-fixed memory address table item, the at least one fixed memory address table item is used to record the memory address (i.e., the fixed memory address) of the target model that does not need to be updated, the at least one non-fixed memory address table item is used to record the memory address (i.e., the non-fixed memory address) of the target model that needs to be updated, and the at least one address index table item in the address index table T1 is a non-absolute address index table item. Each address index table item in the at least one address index table item corresponds to at least one non-fixed memory address table item in the model parameter table T0. The size of the address index table T1 is smaller than the size of the model parameter table T0, i.e., the number of address index table items included in the address index table T1 is less than the number of memory address table items included in the model parameter table T0. In one embodiment of partially updating the model parameter table T0, the at least one address index table item in the address index table T1 corresponds to the at least one non-fixed memory address table item in the model parameter table T0 one by one, and the at least one non-fixed memory address table item is all the non-fixed memory address table items in the model parameter table T0. In another embodiment of partially updating the model parameter table T0, the at least one address index table item in the address index table T1 corresponds to all the non-fixed memory address table items in the model parameter table T0, and there are some address index table items in the at least one address index table item that correspond to multiple non-fixed memory address table items in the model parameter table T0, for example, a certain address index table item in the at least one address index table item corresponds to multiple non-fixed memory address table items in the model parameter table T0.

[0128] As an example, FIG. 12 is a schematic diagram of another address index table T1 provided by embodiments of the present application. The address index table T1 shown in FIG. 12 is used for partial update of the model parameter table T0. As shown in FIG. 12, the address index table T1 includes w address index table entries 1-w, where w is a positive integer smaller than k. Each of the address index table entries 1-w corresponds to at least one non-fixed memory address table entry in the model parameter table T0 shown in FIG. 10, and each of the address index table entries 1-w records an address index, which is used to update the memory address recorded in the non-fixed memory address table entry corresponding to the address index table entry. In one example, in the model parameter table T0 shown in FIG. 10, the memory address table entries 1-w are non-fixed memory address table entries, the memory address table entries w+1-k are fixed memory address table entries, and the address index table entries 1-w correspond to the memory address table entries 1-w one by one. For example, the address index table entry 1 corresponds to the memory address table entry 1, and the address index "1" recorded in the address index table entry 1 is used to update the memory address recorded in the memory address table entry 1; the address index table entry 2 corresponds to the memory address table entry 2, and the address index "2" recorded in the address index table entry 2 is used to update the memory address recorded in the memory address table entry 2; the address index table entry 9 corresponds to the memory address table entry 9, and the address index "G" recorded in the address index table entry 9 is used to update the memory address recorded in the memory address table entry 9; and so on. In another example, some of the address index table entries 1-w correspond to multiple non-fixed memory address table entries in the model parameter table T0. For example, the memory address table entries 1, 2, 3, 4, 6, and 7 are non-fixed memory address table entries; the address index table entry 1 corresponds to the memory address table entries 2 and 3, and the address index "1" recorded in the address index table entry 1 is used to update the memory address recorded in the memory address table entry 2 and the memory address recorded in the memory address table entry 3; the address index table entry 2 corresponds to the memory address table entry 1, and the address index "2" recorded in the address index table entry 2 is used to update the memory address recorded in the memory address table entry 1; the address index table entry 3 corresponds to the memory address table entries 4, 6, and 7, and the address index "1" recorded in the address index table entry 3 is used to update the memory address recorded in the memory address table entry 4, the memory address recorded in the memory address table entry 6, and the memory address recorded in the memory address table entry 7. It should be noted that the difference between FIG. 12 and FIG. 11 is that the address index table T1 shown in FIG. 11 includes absolute address index table entries, while the address index table T1 shown in FIG. 12 does not include absolute address index table entries. The number of address index table entries included in the address index table T1 shown in FIG. 12 can be smaller than the number of address index table entries included in the address index table T1 shown in FIG. 11.

[0129] The address offset table T2 includes at least one address offset entry, each of which records an address offset. The at least one address offset entry corresponds to the at least one address index entry in the address index table T1. Since each of the at least one address index entry corresponds to at least one memory address entry in the model parameter table T0, each of the at least one address offset entry also corresponds to at least one memory address entry in the model parameter table T0. For any corresponding address index entry and address offset entry, the memory address entry corresponding to the address offset entry is the same as the memory address entry corresponding to the address index entry. That is, the memory address entry corresponding to the address offset entry is the same as the memory address entry corresponding to the address index entry. The address offset recorded in each address offset entry is used to update the memory address recorded in the memory address entry corresponding to the address offset entry. Specifically, the address offset recorded in each address offset entry and the address index recorded in the address index entry corresponding to the address offset entry are used to update the memory address recorded in the memory address entry corresponding to the address index entry. The address offset recorded in each address offset entry is an offset relative to the base address recorded in the base address entry indicated by the address index entry corresponding to the address offset entry. The address offset recorded in each address offset entry and the base address recorded in the base address entry indicated by the address index entry corresponding to the address offset entry are used to update the memory address recorded in the memory address entry corresponding to the address index entry.

[0130] In the embodiments of the present application, the address offset table entries in the address offset table T2 and the address index table entries in the address index table T1 are in one-to-one correspondence according to a second correspondence. As an example, the second correspondence is that the i th address offset table entry in the address offset table T2 corresponds to the i th address index table entry in the address index table T1, i is a positive integer, i is not greater than the total number of the address offset table entries in the address offset table T2, and i is not greater than the total number of the address index table entries in the address index table T1. That is, the 1 st address offset table entry in the address offset table T2 corresponds to the 1 st address index table entry in the address index table T1, the 2 nd address offset table entry in the address offset table T2 corresponds to the 2 nd address index table entry in the address index table T1, the 3 rd address offset table entry in the address offset table T2 corresponds to the 3 rd address index table entry in the address index table T1, and so on. As another example, the second correspondence is that the 1 st address offset table entry in the address offset table T2 corresponds to the 3 rd address index table entry in the address index table T1, the 2 nd address offset table entry in the address offset table T2 corresponds to the 5 th address index table entry in the address index table T1, the 3 rd address offset table entry in the address offset table T2 corresponds to the 6 th address index table entry in the address index table T1, and so on. The second correspondence can also be other correspondence, and the embodiments of the present application do not make any limitation in this aspect.

[0131] As an example, FIG. 13 is a schematic diagram of an address offset table T2 provided by an embodiment of the present application. The address offset table T2 shown in FIG. 13 is used for full quantity updating of the model parameter table T0. As shown in FIG. 13, the address offset table T2 includes q address offset table entries 1-q. The address offset table entries 1-q correspond one-to-one to the address index table entries 1-q in the address index table T1 shown in FIG. 11, and each of the address offset table entries 1-q records an address offset. The address offset recorded in each of the address offset table entries 1-q is used to update the memory address recorded in the memory address table entry corresponding to the address offset table entry, i.e., the memory address table entry corresponding to the address index table entry corresponding to the address offset table entry. Specifically, the address offset recorded in each of the address offset table entries 1-q and the address index recorded in the address index table entry corresponding to the address offset table entry are used to update the memory address recorded in the memory address table entry corresponding to the address index table entry. In one example, q=k, the address offset table entries 1-q correspond one-to-one to the address index table entries 1-q, and the address index table entries 1-q correspond one-to-one to the memory address table entries 1-k in the model parameter table T0 shown in FIG. 10 (i.e., the address offset table entries 1-q, the address index table entries 1-q, and the memory address table entries 1-k correspond one-to-one). For example, the address offset table entry 1 corresponds to the address index table entry 1, and the address index table entry 1 corresponds to the memory address table entry 1. The address offset S1 recorded in the address offset table entry 1 and the address index "1" recorded in the address index table entry 1 are used to update the memory address recorded in the address index table entry 1. The address offset table entry 2 corresponds to the address index table entry 2, and the address index table entry 2 corresponds to the memory address table entry 2. The address offset S2 recorded in the address offset table entry 2 and the address index "2" recorded in the address index table entry 2 are used to update the memory address recorded in the address index table entry 2. The address offset table entry 3 corresponds to the address index table entry 3, and the address index table entry 3 corresponds to the memory address table entry 3. The address offset S3 recorded in the address offset table entry 3 and the address index "1" recorded in the address index table entry 3 are used to update the memory address recorded in the address index table entry 3. And so on. In another example, q is less than k, the address offset table entries 1-q correspond one-to-one to the address index table entries 1-q, and some of the address index table entries 1-q correspond to multiple memory address table entries in the model parameter table T0. For example, the address offset table entry 1 corresponds to the address index table entry 1, and the address index table entry 1 corresponds to the memory address table entry 2 and the memory address table entry 3. The address offset S1 recorded in the address offset table entry 1 and the address index "1" recorded in the address index table entry 1 are used to update the memory address recorded in the memory address table entry 2 and the memory address recorded in the memory address table entry 3.The address offset table item 2 corresponds to the address index table item 2, and the address index table item 2 corresponds to the memory address table item 1. The address offset S2 recorded in the address offset table item 2 and the address index "2" recorded in the address index table item 2 are used to update the memory address recorded in the memory address table item 1. The address offset table item 3 corresponds to the address index table item 3, and the address index table item 3 corresponds to the memory address table item 4, the memory address table item 6 and the memory address table item 7. The address offset S3 recorded in the address offset table item 3 and the address index "1" recorded in the address index table item 3 are used to update the memory address recorded in the memory address table item 4, the memory address recorded in the memory address table item 6 and the memory address recorded in the memory address table item 7. The same applies to other address offset table items.

[0132] As an example, FIG. 14 is a schematic diagram of another address offset table T2 provided by an embodiment of the present application. The address offset table T2 shown in FIG. 14 is used to perform partial update on the model parameter table T0. As shown in FIG. 14, the address offset table T2 includes w address offset table items 1-w. The address offset table items 1-w one-to-one correspond to the address index table items 1-w in the address index table T1 shown in FIG. 12. Each of the address offset table items 1-w records an address offset, and the address offset recorded in each of the address offset table items is used to update the memory address recorded in the memory address table item corresponding to the address offset table item. The memory address table item corresponding to the address offset table item is the memory address table item corresponding to the address index table item corresponding to the address offset table item. Specifically, the address offset recorded in each of the address offset table items and the address index recorded in the address index table item corresponding to the address offset table item are used to update the memory address recorded in the memory address table item corresponding to the address index table item.

[0133] Based on the above description, at least one address index table entry in the address index table T1 corresponds to at least one address offset table entry in the address offset table T2. In the embodiment of full quantity updating of the model parameter table T0, the at least one address index table entry includes an absolute address index table entry, the absolute address index table entry is used to indicate an absolute base address table entry in the base address table, and the absolute base address table entry is used to record an absolute base address. In the case that the absolute base address is "0", the address offset recorded in the address offset table entry corresponding to the absolute address index table entry can be the same as the memory address recorded in the memory address table entry corresponding to the absolute address index table entry. Each address index table entry in the at least one address index table entry is used to record an address index, each address offset table entry in the at least one address offset table entry is used to record an address offset, each memory address table entry in the model parameter table T0 is used to record a memory address of the target model, the address index recorded in each address index table entry is used to indicate a base address table entry in the base address table of the target model, each base address table entry in the base address table is used to record a base address, and the address offset recorded in each address offset table entry is an offset compared to the base address in the base address table entry indicated by the address index recorded in the address index table entry corresponding to the address offset table entry.

[0134] Based on the above description, whether the model parameter table T0 is full quantity updated or partially updated, the size of the address index table T1 is equal to the size of the address offset table T2. That is, the number of address index table entries included in the address index table T1 is equal to the number of address offset table entries included in the address offset table T2. The address index table T1 and the address offset table T2 depend on the target model and the memory allocation strategy adopted for allocating memory for the target model. In the life cycle of the target model existing on the first chip, the address index table T1 and the address offset table T2 are fixed and unchanged in the case that the memory allocation strategy does not change. That is, the size and content (that is, the address index in the address index table T1) of the address index table T1 are fixed and unchanged, and the size and content (that is, the address offset in the address offset table T2) of the address offset table T2 are fixed and unchanged.

[0135] In the model loading stage of the target model, the first chip acquires the model parameter table T0, the address index table T1 and the address offset table T2. In one embodiment, in the model loading stage of the target model, the first chip generates the model parameter table T0, the address index table T1 and the address offset table T2. In another embodiment, in the model loading stage of the target model, the first chip receives the model parameter table T0, the address index table T1 and the address offset table T2 sent by the second chip. The model parameter table T0, the address index table T1 and the address offset table T2 are generated by the second chip. In an example, in the model loading stage of the target model: after the second chip generates the model parameter table T0, the address index table T1 and the address offset table T2, the second chip sends the model parameter table T0, the address index table T1 and the address offset table T2 to the first chip, and the first chip receives the model parameter table T0, the address index table T1 and the address offset table T2 sent by the second chip. The second chip can be a CPU chip, and the first chip and the second chip are in the same computing system. The computing system can be a system of a computing device.

[0136] The implementation process of the first chip generating the model parameter table T0, the address index table T1 and the address offset table T2 is similar to that of the second chip generating the model parameter table T0, the address index table T1 and the address offset table T2. This embodiment takes the first chip generating the model parameter table T0, the address index table T1 and the address offset table T2 as an example for description. In a specific embodiment, the first chip generates the model parameter table T0, the address index table T1 and the address offset table T2 according to the target model and the memory allocation strategy of the target model. The memory allocation strategy includes the number of memory segments to be allocated for the target model, the size of each memory segment to be allocated for the target model, the memory segment where each model input memory, model output memory, operator input memory, operator output memory and operator workspace memory of the target model is located, and the position (or distribution) of each model input memory, model output memory, operator input memory, operator output memory and operator workspace memory of the target model in the memory segment. The memory allocation strategy can also include the reuse of the model input memory, model output memory, operator input memory, operator output memory and operator workspace memory of the target model, which is not limited in this embodiment.

[0137] In a specific embodiment, the first chip determines the size of the model parameter table T0 (i.e., the number of memory address table entries included in the model parameter table T0) and the meaning of each memory address table entry in the model parameter table T0 according to the target model; and the first chip generates the model parameter table T0 according to the size of the model parameter table T0 and the meaning of each memory address table entry in the model parameter table T0. The meaning of the memory address table entry is also the function of the memory address table entry. The meaning of each memory address table entry in the model parameter table T0 can be the address of which memory (e.g., which model input memory, which model output memory, which operator input memory, which operator output memory, which operator workspace memory) of the target model that the memory address table entry is used to record. For example, the target model is the model shown in FIG. 4. The meaning of the memory address table entry 1 in the model parameter table T0 can be that the memory address table entry 1 is used to record the address of the operator input memory of the operator A. The meaning of the memory address table entry 2 in the model parameter table T0 can be that the memory address table entry 2 is used to record the address of the operator output memory 1 of the operator A. The meaning of the memory address table entry 3 in the model parameter table T0 can be that the memory address table entry 3 is used to record the address of the operator output memory 2 of the operator A. The meaning of the memory address table entry 4 in the model parameter table T0 can be that the memory address table entry 4 is used to record the address of the operator input memory 1 of the operator B. The meaning of the memory address table entry 5 in the model parameter table T0 can be that the memory address table entry 5 is used to record the address of the operator output memory 1 of the operator B. The meaning of the memory address table entry 6 in the model parameter table T0 can be that the memory address table entry 6 is used to record the address of the operator input memory 1 of the operator C. The meaning of the memory address table entry 7 in the model parameter table T0 can be that the memory address table entry 7 is used to record the address of the operator output memory 1 of the operator C. And so on. The operator input memory of the operator A is also the input memory of the operator A, which can be the model input memory of the target model. The operator output memory 1 of the operator A is the memory of the output 1 of the operator A. The operator output memory 2 of the operator A is the memory of the output 2 of the operator A. The operator input memory 1 of the operator B is the memory of the input 1 of the operator B. The operator output memory 1 of the operator B is the memory of the output 1 of the operator B. The operator input memory 1 of the operator C is the memory of the input 1 of the operator C. The operator output memory 1 of the operator C is the memory of the output 1 of the operator C. It should be noted that in the model loading stage of the target model, the content in each memory address table entry in the model parameter table T0 generated by the first chip can be empty (e.g., the value in the memory address table entry is a null value, or no content is recorded in the memory address table entry), or the content in each memory address table entry in the model parameter table T0 generated by the first chip is an initial value. Regardless of which case, the meaning of each memory address table entry in the model parameter table T0 has been determined.

[0138] After the first chip generates the model parameter table T0, the first chip generates an address index table T1 and an address offset table T2 according to the model parameter table T0 and the memory allocation strategy of the target model. In specific embodiments, the first chip creates an initial address index table and an initial address offset table according to the model parameter table T0, the size of the initial address index table is equal to the size of the initial address offset table, and the size of the initial address index table is less than or equal to the size of the model parameter table T0. The initial address index table includes at least one address index table item, the initial address offset table includes at least one address offset table item, the at least one address index table item corresponds to the at least one address offset table item one by one, each address index table item in the at least one address index table item corresponds to at least one memory address table item in the model parameter table T0, and the same address index table item corresponds to a memory address table item recording the same memory address. The first chip assigns a value to the address index table item in the initial address index table corresponding to each memory address table item in the model parameter table T0 according to the meaning of the memory address table item and the memory allocation strategy of the target model, and assigns a value to the address offset table item in the initial address offset table corresponding to the memory address table item, to obtain the address index table T1 and the address offset table T2. In specific embodiments, the memory allocation strategy includes the positions of each model input memory, model output memory, operator input memory, operator output memory, operator workspace memory, etc. of the target model in the memory segment; the value assigned by the first chip to each address index table item is used to indicate the memory segment where the memory address recorded by the memory address table item corresponding to the address index table item is located, and the value assigned by the first chip to each address offset table item is used to indicate the position of the memory address recorded by the memory address table item corresponding to the address offset table item in the memory segment where it is located. For example, the target model is the model shown in FIG. 4, the meaning of the memory address table item 1 in the model parameter table T0 can be: the memory address table item 1 is used to record the address of the operator input memory of the operator A; the address index table item 1 in the address index table T1 corresponds to the memory address table item 1, and the address index table item 1 corresponds to the address offset table item 1 in the address offset table T2; assuming that the operator input memory of the operator A is in the memory segment 1, the first chip records the index of the memory segment 1 in the address index table item 1, and the first chip records the offset of the operator input memory of the operator A compared to the starting address of the memory segment 1 in the address offset table item 1. For another example, the meaning of the memory address table item 2 in the model parameter table T0 can be: the memory address table item 2 is used to record the address of the operator output memory 1 of the operator A; the address index table item 2 in the address index table T1 corresponds to the memory address table item 2, and the address index table item 2 corresponds to the address offset table item 2 in the address offset table T2; assuming that the operator output memory 1 of the operator A is in the memory segment 1, the first chip records the index of the memory segment 1 in the address index table item 2, and the first chip records the offset of the operator output memory 1 of the operator A compared to the starting address of the memory segment 1 in the address offset table item 2. And so on.

[0139] S902. The first chip stores the model parameter table T0, the address index table T1 and the address offset table T2 in the memory of the first chip.

[0140] In an optional embodiment, the first chip stores the model parameter table T0 in a first memory segment of the first chip, the first chip stores the address index table T1 in a second memory segment of the first chip, and the first chip stores the address offset table T2 in a third memory segment of the first chip. The first memory segment can be a continuous memory segment, the second memory segment can be a continuous memory segment, and the third memory segment can be a continuous memory segment. The first memory segment, the second memory segment and the third memory segment can be continuous or discontinuous, and the present application does not limit the same.

[0141] S903. The first chip obtains a base address table T3 of the target model, and the base address table T3 is generated according to a current execution memory of the target model. The current execution memory is a memory allocated in the memory of the first chip for the current execution of the target model, and each address index table entry in the address index table T1 is used to indicate a base address table entry in the base address table T3.

[0142] The current execution memory is a memory allocated in the memory of the first chip for the current execution of the target model in the model execution phase of the target model by using a memory allocation strategy. The current execution memory of the target model has the same meaning as the “memory of the target model” described in S901. It can be understood that the current execution memory can include a model input memory, a model output memory and an FM memory. The current execution memory can include at least one memory segment. For example, the current execution memory is a continuous memory segment, or the current execution memory includes non-continuous multiple memory segments. The number of memory segments included in the current execution memory and the continuity of the memory segments depend on the memory allocation strategy used for allocating the memory for the target model. In the embodiments of the present application, the first chip can execute the target model multiple times, and the first chip can allocate the current execution memory for the target model in the model execution phase of each execution of the target model in the memory of the first chip according to the memory allocation strategy. The memory allocation strategy used for allocating the current execution memory for the target model each time is the same, so that in the model execution phase of each execution of the target model, the base address table T3 can be generated according to the current execution memory allocated for the target model, and the model parameter table T0 of the target model can be updated according to the current generated base address table T3 and the unchanged address index table T1 and the unchanged address offset table T2.

[0143] The base address table T3 includes at least one base address table entry, and each base address table entry is used to record a base address. The base addresses recorded in the base address table T3 include the starting address of the memory segment included in the memory of the current execution of the target model, and can also include an absolute base address. The absolute base address is used to update a fixed memory address in the model parameter table of the target model. The base addresses in the base address table T3 except the absolute base address can be referred to as non-absolute base addresses, and the non-absolute base addresses are used to update a non-fixed memory address in the model parameter table of the target model. For example, the starting address of the memory segment is a non-absolute base address. In the base address table T3, the base address table entry used to record the absolute base address is referred to as an absolute base address table entry, and the base address table entry used to record the non-absolute base address can be referred to as a non-absolute base address table entry. The absolute base address table entry can be located at any position in the base address table T3. For example, the absolute base address table entry is the last table entry in the base address table T3. In the embodiment of the present application, the memory of the current execution is allocated by using a memory allocation strategy, the base address table T3 is generated according to the memory of the current execution, and the address index table T1 is generated according to the memory allocation strategy, so the base address table T3 has a relationship with the address index table T1. Specifically, each address index table entry (specifically, the address index recorded in the address index table entry) in the address index table T1 is used to indicate a base address table entry in the base address table T3. For example, each absolute address index table entry in the address index table T1 is used to indicate an absolute base address table entry in the base address table T3, and each non-absolute address index table entry in the address index table T1 is used to indicate a non-absolute base address table entry in the base address table T3. The base address recorded in the non-absolute base address table entry can be the starting address of the memory segment indicated by the address index recorded in the non-absolute address index table entry. In the embodiment of the full update of the model parameter table T0, the address index table T1 includes the absolute address index table entry and the non-absolute address index table entry, the base address table T3 includes the absolute base address table entry and the non-absolute base address table entry, the absolute address index table entry in the address index table T1 is used to indicate the absolute base address table entry in the base address table T3, and the non-absolute address index table entry in the address index table T1 is used to indicate the non-absolute base address table entry in the base address table T3. In the embodiment of the partial update of the model parameter table T0, the address index table T1 does not include the absolute address index table entry, the base address table T3 does not include the absolute base address table entry, the address index table entries in the address index table T1 are all non-absolute address index table entries, and the base address table entries in the base address table T3 are all non-absolute base address table entries.

[0144] As an example, in the embodiment of full quantity updating of the model parameter table T0, the address index table T1 is shown in Fig. 11, and the base address table T3 is shown in Fig. 15. Referring to Fig. 15, the base address table T3 includes m base address entries 1 to m, and the base address entries 1 to m-1 record base addresses B1 to B(m-1), and the base address entry m records a base address "0", m is a positive integer. The base address entry m is an absolute base address entry, the base address "0" is an absolute base address, and the base address entries 1 to m-1 are non-absolute base address entries, and the base addresses B1 to B(m-1) are non-absolute base addresses. Each of the base addresses B1 to B(m-1) can be a starting address of a memory segment included in the current execution memory. For example, the current execution memory includes m-1 memory segments 1 to m-1, the base address B1 is a starting address of the memory segment 1, the base address B2 is a starting address of the memory segment 2, the base address B3 is a starting address of the memory segment 3, and so on, and the base address B(m-1) is a starting address of the memory segment m-1. In combination of Fig. 11 and Fig. 15: the address index table entry 1 and the address index table entry 3 both record an address index "1", the address index "1" is used for indicating the base address entry 1, and thus the address index table entry 1 and the address index table entry 3 are used for indicating the base address entry 1; the address index table entry 2 and the address index table entry q both record an address index "2", the address index "2" is used for indicating the base address entry 2, and thus the address index table entry 2 and the address index table entry q are used for indicating the base address entry 2; the address index table entry 5 and the address index table entry 8 both record an address index "3", the address index "3" is used for indicating the base address entry 3, and thus the address index table entry 5 and the address index table entry 8 are used for indicating the base address entry 3; and the address index table entry 9 records an absolute address index "G", the absolute address index "G" is used for indicating the base address entry m (absolute base address entry), and thus the address index table entry 9 is used for indicating the base address entry m.

[0145] As an example, in the embodiment of partially updating the model parameter table T0, the address index table T1 is shown in Fig. 12, and the base address table T3 is shown in Fig. 16. Referring to Fig. 16, the base address table T3 includes p base address entries 1-p, and the base address entries 1-p record the base addresses B1-Bp, where p is a positive integer. The base address entries 1-p are all non-absolute base address entries, and the base addresses B1-Bp are all non-absolute base addresses. Each of the base addresses B1-Bp is the starting address of a memory segment included in the current execution memory. For example, the current execution memory includes p memory segments 1-p, the base address B1 is the starting address of the memory segment 1, the base address B2 is the starting address of the memory segment 2, the base address B3 is the starting address of the memory segment 3, and so on, and the base address Bp is the starting address of the memory segment p. In combination of Fig. 12 and Fig. 16: the address index "1" is recorded in the address index entry 1 and the address index entry 3, and the address index "1" is used to indicate the base address entry 1, so the address index entry 1 and the address index entry 3 are both used to indicate the base address entry 1; the address index "2" is recorded in the address index entry 2 and the address index entry w, and the address index "2" is used to indicate the base address entry 2, so the address index entry 2 and the address index entry w are both used to indicate the base address entry 2; the address index "5" is recorded in the address index entry 4, and the address index "5" is used to indicate the base address entry 5. And so on.

[0146] In an optional embodiment, the base addresses B1-B(m-1) in the base address table T3 shown in Fig. 15 are all starting addresses of memory segments included in the current execution memory, and the base addresses B1-Bp in the base address table T3 shown in Fig. 16 are all starting addresses of memory segments included in the current execution memory, where p is equal to m-1. The difference between the base address table T3 shown in Fig. 15 and the base address table T3 shown in Fig. 16 is that the base address table T3 shown in Fig. 15 includes absolute base address entries, and the base address table T3 shown in Fig. 16 does not include absolute base address entries.

[0147] In the embodiments of the present application, the current execution memory can be allocated by the first chip or the second chip. In one embodiment, during the model execution stage of the target model, the first chip allocates the current execution memory in the memory of the first chip according to a memory allocation strategy. In another embodiment, during the model execution stage of the target model, the second chip allocates the current execution memory in the memory of the first chip according to the memory allocation strategy. In the another embodiment, the second chip can obtain the information of the available memory of the first chip, the second chip determines the available memory of the first chip according to the information of the available memory of the first chip, and then the second chip allocates the current execution memory in the available memory of the first chip according to the memory allocation strategy. The implementation process of the first chip allocating the current execution memory in the memory of the first chip according to the memory allocation strategy is similar to the implementation process of the second chip allocating the current execution memory in the memory of the first chip according to the memory allocation strategy, and the embodiment takes the first chip allocating the current execution memory in the memory of the first chip according to the memory allocation strategy as an example for description. In specific embodiments, the memory allocation strategy includes the number of memory segments (i.e., the number of memory segments included in the current execution memory) that need to be allocated for the target model, the size of each memory segment (i.e., the size of each memory segment included in the current execution memory) that need to be allocated for the target model, the memory segments where each model input memory, model output memory, operator input memory, operator output memory and operator workspace memory of the target model are located, and the positions of each model input memory, model output memory, operator input memory, operator output memory and operator workspace memory of the target model in the memory segments. The first chip determines the number of memory segments included in the current execution memory, the size of each memory segment included in the current execution memory, the memory segments where each model input memory, model output memory, operator input memory, operator output memory and operator workspace memory of the target model are located, and the positions of each model input memory, model output memory, operator input memory, operator output memory and operator workspace memory of the target model in the memory segments according to the memory allocation strategy, and then allocates the local execution memory for the target model based on the information. Since the address index table T1 and the address offset table T2 are also generated according to the memory allocation strategy, each memory segment included in the current execution memory satisfies the relationship between the address index table T1 and the address offset table T2.For example, the address index table entries in the address index table T1 correspond to the address offset table entries in the address offset table T2 in a one-to-one manner. The address index recorded in each non-absolute address index table entry in the address index table T1 is the index of the memory segment in which the memory address recorded in the memory address table entry corresponding to the non-absolute address index table entry is located. The address offset recorded in each address offset table entry in the address offset table T2 is the offset of the memory address recorded in the memory address table entry corresponding to the address offset table entry from the start address of the memory segment indicated by the address index recorded in the address index table entry corresponding to the address offset table entry. The address offset recorded in the address offset table entry corresponding to the absolute address index table entry in the address offset table T2 can be equal to the memory address recorded in the memory address table entry corresponding to the absolute address index table entry. It should be noted that the allocation of the current execution memory in the memory of the first chip can refer to the allocation of the FM memory. The current execution memory can also include model input memory and model output memory, which are allocated by the front-end framework, and the embodiments of the present application do not limit this.

[0148] In the embodiments of the present application, the first chip obtaining the base address table T3 can include two implementation manners. In one implementation manner, the first chip generates the base address table T3 according to the current execution memory of the target model. In another implementation manner, the first chip receives the base address table T3 sent by the second chip. In the another implementation manner, the base address table T3 is generated by the second chip according to the current execution memory of the target model, and the second chip sends the base address table T3 to the first chip after generating the base address table T3. In specific embodiments, the second chip sends an address update task of the target model to the first chip, and the address update task includes the base address table T3. The first chip receives the address update task, and the first chip obtains the base address table T3 from the address update task. In optional embodiments, the address update task further includes an address update operator, and the address update operator is used for the first chip to update the model parameter table T0 of the target model.

[0149] The implementation process of the first chip for generating the base address table T3 according to the current execution memory is similar to that of the second chip for generating the base address table T3 according to the current execution memory. This embodiment takes the first chip for generating the base address table T3 according to the current execution memory as an example for illustration. It should be noted that since the base address recorded in the non-absolute base address table item in the base address table T3 is the starting address of the memory segment included in the memory of the target model, the number of the memory segments included in the memory of the target model depends on the memory allocation strategy, and the address index table T1 and the address offset table T2 are also generated according to the memory allocation strategy, therefore, the structure of the base address table T3 (including the number of the base address table items in the base address table T3, the position of the base address table item indicated by the address index recorded in each address index table item in the base address table T3, etc.) can be determined in the process of generating the address index table T1 and the address offset table T2. That is, the structure of the base address table T3 can be determined in the model loading stage of the target model. Generating the base address table T3 in the model execution stage of the target model can be assigning values to the base address table T3 according to the current execution memory, or in other words, updating the base addresses recorded in the base address table T3 according to the current execution memory. In specific embodiments, the first chip determines the structure of the base address table T3 according to the memory allocation strategy, the first chip creates an initial base address table according to the structure of the base address table T3, and the first chip assigns values to the initial base address table according to the current execution memory to obtain the base address table T3. For example, the first chip records the starting address of each memory segment included in the current execution memory as a base address, and the first chip records a base address in the base address table item indicated by each address index recorded in the address index table T1 in the initial base address table. In the embodiment of full-quantity updating the model parameter table T0, the first chip also records absolute base addresses in the initial base address table. For example, the first chip records the absolute base addresses in the last table item of the initial base address table.

[0150] S904. The first chip updates the model parameter table T0 according to the base address table T3, the address index table T1 and the address offset table T2.

[0151] In the model execution stage of the target model, the model parameter table T0, the address index table T1 and the address offset table T2 are all in the memory of the first chip. For example, before the model execution stage of the target model starts, the model parameter table T0, the address index table T1 and the address offset table T2 are stored in the memory of the first chip. Embodiments of the present application store the model parameter table T0, the address index table T1 and the address offset table T2 in the memory of the first chip in the model loading stage of the target model, so that in the model execution stage, the model parameter table T0, the address index table T1 and the address offset table T2 are all in the memory of the first chip. In addition, when the first chip executes S904, the base address table T3 is in the memory of the first chip. The first chip updates the model parameter table T0 in the memory of the first chip according to the base address table T3 in the memory of the first chip, the address index table T1 in the memory of the first chip and the address offset table T2 in the memory of the first chip. The first chip updating the model parameter table T0 specifically means updating the memory address recorded in the memory address table item in the model parameter table T0. In optional embodiments, the memory address table item in the model parameter table T0 includes a first memory address table item, the address index table item corresponding to the first memory address table item in the address index table T1 is a first address index table item, the first address index table item is used to indicate a first base address table item in the base address table T3, and the first address index table item corresponds to a first address offset table item in the address offset table T2. The first chip updating the model parameter table T0 according to the base address table T3, the address index table T1 and the address offset table T2 includes: the first chip determining the first address index table item in the address index table T1 according to the first memory address table item; the first chip determining the first base address table item in the base address table T3 according to the first address index table item; the first chip determining the first address offset table item in the address offset table T2 according to the first address index table item; and the first chip updating the memory address recorded in the first memory address table item according to the base address recorded in the first base address table item and the address offset recorded in the first address offset table item. In specific embodiments, the first chip determines the first base address table item in the base address table T3 according to the address index recorded in the first address index table item; the first chip determines a first updated address according to the base address recorded in the first base address table item and the address offset recorded in the first address offset table item; and the first chip updates the memory address recorded in the first memory address table item with the first updated address. For example, the first chip deletes the original memory address recorded in the first memory address table item, and the first chip records the first updated address in the first memory address table item. The first memory address table item is any memory address table item in the model parameter table T0 that needs to be updated. In embodiments of full update of the model parameter table T0, the first memory address table item is any memory address table item in the model parameter table T0. In embodiments of partial update of the model parameter table T0, the first memory address table item is any non-fixed memory address table item in the model parameter table T0.

[0152] In an optional embodiment, the first chip updates the model parameter table T0 according to the base address table T3, the address index table T1 and the address offset table T2 by using an address updating operator. In a specific embodiment, the first chip inputs the base address table T3, the address index table T1 and the address offset table T2 into the address updating operator, which updates the model parameter table T0 according to the base address table T3, the address index table T1 and the address offset table T2 and outputs the updated model parameter table T0. In a specific embodiment, the address updating operator determines a first address index table item in the address index table T1 according to a first memory address table item in the model parameter table T0; the address updating operator determines a first base address table item in the base address table T3 according to the first address index table item; the address updating operator determines a first address offset table item in the address offset table T2 according to the first address index table item; the address updating operator determines a first updated address according to a base address recorded in the first base address table item and an address offset recorded in the first address offset table item; and the address updating operator updates the memory address recorded in the first memory address table item by using the first updated address.

[0153] As an example, FIG. 17 is a schematic diagram of updating the model parameter table T0 according to an embodiment of the present application. FIG. 17 illustrates an example of updating the model parameter table T0 according to the address index table T1 shown in FIG. 11, the address offset table T2 shown in FIG. 13, and the base address table T3 shown in FIG. 15, and an example of q=k. As shown in FIG. 17, the first chip inputs the address index table T1, the address offset table T2, and the base address table T3 to the address updating operator, and the address updating operator updates the model parameter table T0 according to the address index table T1, the address offset table T2, and the base address table T3. In an optional embodiment, the address index table item 1 corresponds to the address offset table item 1, and the address index table item 1 corresponds to the memory address table item 1; the address index table item 2 corresponds to the address offset table item 2, and the address index table item 2 corresponds to the memory address table item 2; the address index table item 3 corresponds to the address offset table item 3, and the address index table item 3 corresponds to the memory address table item 3; and so on, the address index table item q corresponds to the address offset table item q, and the address index table item q corresponds to the memory address table item k. Moreover, the address index "1" is used to indicate the base address table item 1, the address index "2" is used to indicate the base address table item 2, the address index "3" is used to indicate the base address table item 3, and so on. Then: the address updating operator determines the base address table item 1 according to the address index "1" recorded in the address index table item 1, and updates the memory address recorded in the memory address table item 1 according to the base address B1 recorded in the base address table item 1 and the address offset S1 recorded in the address offset table item 1; the address updating operator determines the base address table item 2 according to the address index "2" recorded in the address index table item 2, and updates the memory address recorded in the memory address table item 2 according to the base address B2 recorded in the base address table item 2 and the address offset S2 recorded in the address offset table item 2; and so on, the address updating operator determines the base address table item 2 according to the address index "2" recorded in the address index table item q, and updates the memory address recorded in the memory address table item k according to the base address B2 recorded in the base address table item 2 and the address offset Sq recorded in the address offset table item q. The logic of the address updating operator updating the model parameter table T0 according to the address index table T1, the address offset table T2, and the base address table T3 can be expressed as follows. In the following expressions, the symbol "+" represents the address offset corresponding to the base address offset.

[0154] Memory address A1 = base address B1 + address offset S1;

[0155] Memory address A2 = base address B2 + address offset S2;

[0156] Memory address A3 = base address B1 + address offset S3;

[0157] Memory address A4 = base address B5 + address offset S4;

[0158] Memory address A5 = base address B3 + address offset S5;

[0159] Memory address A6 = base address B4 + address offset S6;

[0160] Memory address A7 = base address B6 + address offset S7;

[0161] Memory address A8 = base address B3 + address offset S8;

[0162] Memory address A9 = 0 + address offset S9;

[0163] ...

[0164] Memory address Ak = base address B2 + address offset Sq.

[0165] As another example, FIG. 18 is a schematic diagram of another example of updating the model parameter table T0 according to an embodiment of the present application. FIG. 18 illustrates an example of updating the model parameter table T0 shown in FIG. 10 in part according to the address index table T1 shown in FIG. 12, the address offset table T2 shown in FIG. 14, and the base address table T3 shown in FIG. 16. As shown in FIG. 18, the first chip inputs the address index table T1, the address offset table T2, and the base address table T3 to the address updating operator, which updates the memory addresses recorded in the non-fixed memory address table entries in the model parameter table T0 according to the address index table T1, the address offset table T2, and the base address table T3, and does not update the memory addresses recorded in the fixed memory address table entries in the model parameter table T0. In an optional embodiment, the memory address table entries 1-w are non-fixed memory address table entries, and the memory address table entries w+1-k are fixed memory address table entries. The address updating operator updates the memory addresses recorded in the memory address table entries 1-w, and does not update the memory addresses recorded in the memory address table entries w+1-k. For example, the address index table entry 1 corresponds to the address offset table entry 1, and the address index table entry 1 corresponds to the memory address table entry 1; the address index table entry 2 corresponds to the address offset table entry 2, and the address index table entry 2 corresponds to the memory address table entry 2; the address index table entry 3 corresponds to the address offset table entry 3, and the address index table entry 3 corresponds to the memory address table entry 3; and so on, the address index table entry w corresponds to the address offset table entry w, and the address index table entry w corresponds to the memory address table entry w. Furthermore, the address index “1” is used to indicate the base address table entry 1, the address index “2” is used to indicate the base address table entry 2, the address index “3” is used to indicate the base address table entry 3, and so on. The logic of the address updating operator updating the memory addresses recorded in the memory address table entries 1-w can be expressed as follows. The symbol “+” represents the address offset corresponding to the base address offset.

[0166] Memory address A1 = Base address B1 + Address offset S1;

[0167] Memory address A2 = Base address B2 + Address offset S2;

[0168] Memory address A3 = Base address B1 + Address offset S3;

[0169] Memory address A4 = Base address B5 + Address offset S4;

[0170] Memory address A5 = Base address B3 + Address offset S5;

[0171] Memory address A6 = Base address B4 + Address offset S6;

[0172] Memory address A7 = Base address B6 + Address offset S7;

[0173] ...

[0174] The memory address Aw is the base address B2 plus the address offset Sw.

[0175] It should be noted that FIG. 17 and FIG. 18 are only examples, and in actual applications, the first chip also inputs the model parameter table T0 that needs to be updated into the address updating operator. That is, the first chip inputs the address index table T1, the address offset table T2, the base address table T3 and the model parameter table T0 that needs to be updated into the address updating operator, and the address updating operator updates the model parameter table T0 that needs to be updated according to the address index table T1, the address offset table T2 and the base address table T3 to obtain the updated model parameter table T0, and outputs the updated model parameter table T0.

[0176] In an optional embodiment, before the first chip updates the model parameter table T0 by using the address updating operator, the first chip acquires the address updating operator. For example, the first chip generates the address updating operator. Alternatively, the first chip receives the address updating operator sent by the second chip. In a specific embodiment, the first chip receives an address updating task sent by the second chip, and the address updating task includes the address updating operator. Optionally, the base address table T3 is generated by the second chip, and the address updating task further includes the base address table T3. The second chip carries the address updating operator and the base address table T3 in the same address updating task to send to the first chip, which can save the overhead of interaction between the first chip and the second chip.

[0177] To sum up, in the model execution stage of the target model, the memory of the first chip includes the address index table of the target model, the address offset table of the target model and the model parameter table of the target model. After the first chip acquires the base address table of the target model, the first chip updates the model parameter table according to the base address table, the address index table and the address offset table. In this way, in the process of updating the model parameter table of the target model in the model execution stage of the target model, there is no need to perform H2D copying on the model parameter table, and there is no need to traverse the operators of the target model, so that the time consumption of updating the model parameter table of the target model is shorter, and the updating efficiency is higher.

[0178] In the technical scheme provided in the embodiments of the present application, the operations performed by the first chip can be performed by a computing unit in the first chip. The computing unit can be a cube computing unit, a vector computing unit, a CPU computing unit, an SDMA computing unit, an AIDSA, etc. These computing units have high computing performance, so that the updating efficiency of the model parameter table can be improved. Optionally, the process of updating the memory addresses recorded in different memory address table items in S904 is performed in parallel by different computing units in the first chip, which can also improve the updating efficiency of the model parameter table.

[0179] It should be noted that S901 to S902 are optional steps of the embodiments of the present application. The first chip executes S901 to S902 in the model loading stage of the target model, and the first chip only needs to execute S903 to S904 to update the model parameter table of the target model each time the target model is updated. After the target model is loaded into the first chip, the first chip can execute the target model multiple times, and the first chip executes S903 to S904 in each execution stage of the target model (i.e., the model execution stage) to update the model parameter table of the target model. The model execution stage shown in FIG. 9 can be any execution stage of the target model by the first chip after the target model is loaded into the first chip. In the life cycle of the target model existing on the first chip, the address index table of the target model and the address offset table of the target model are fixed and unchanged, and the content of the base address table of the target model changes with the memory of the current execution of the target model. In addition, the address index table and the address offset table are usually large, and the base address table is usually small (the number of memory segments included in the current execution memory allocated each time the target model is executed is usually small). The embodiments of the present application organize the information required to update the model parameter table into the unchanged address index table, the unchanged address offset table, and the variable base address table. In each execution stage of the target model, the first chip only needs to obtain the smaller and variable base address table, and assemble the obtained base address table with the address index table and the address offset table obtained in the model loading stage to update the model parameter table of the target model, which has high update efficiency.

[0180] The technical scheme provided by the embodiments of the present application can shorten the time consumption of updating the model parameter table in the sink scheduling mode, thereby reducing the head overhead m_l_t (referring to FIG. 8) of the sink scheduling mode, shortening the time consumption of scheduling the target model in the sink scheduling mode, and improving the efficiency of executing the target model.

[0181] Please refer to FIG. 19, which shows a flowchart of another method for updating a model parameter table provided by the embodiments of the present application. The updating method is applied to a computing system including a first chip and a second chip. The first chip can be a model execution chip, and the second chip can be a CPU chip. Referring to FIG. 19, the updating method includes the following steps S1901 to S1908. Among them, the first chip and the second chip execute S1901 to S1904 in the model loading stage of the target model. The first chip and the second chip execute S1905 to S1908 in the model execution stage of the target model.

[0182] S1901. The second chip generates a model parameter table T0 of the target model, an address index table T1 of the target model, and an address offset table T2 of the target model, the model parameter table T0 includes at least one memory address table item, the address index table T1 includes at least one address index table item, the address offset table T2 includes at least one address offset table item, each of the at least one address index table item corresponds to at least one memory address table item in the model parameter table T0, and the at least one address index table item corresponds to the at least one address offset table item in a one-to-one manner.

[0183] S1902. The second chip sends the model parameter table T0, the address index table T1, and the address offset table T2 to the first chip.

[0184] S1903. The first chip receives the model parameter table T0, the address index table T1, and the address offset table T2 sent by the second chip.

[0185] S1904. The first chip stores the model parameter table T0, the address index table T1, and the address offset table T2 in the memory of the first chip.

[0186] S1905. The second chip generates a base address table T3 of the target model according to a current execution memory of the target model, the current execution memory is a memory allocated by the first chip for the current execution of the target model, and the base address table T3 includes at least one base address table item, wherein each address index table item in the address index table T1 is used to indicate one base address table item in the base address table T3.

[0187] In an optional embodiment, the current execution memory is a memory allocated by the first chip for the current execution of the target model in a model execution stage of the target model. For example, in the model execution stage of the target model, the second chip allocates the current execution memory in the memory of the first chip according to a memory allocation strategy of the target model, and the second chip generates the base address table T3 according to the current execution memory. For details, refer to the related description in the embodiment shown in FIG. 9, which will not be repeated here.

[0188] S1906. The second chip sends the base address table T3 to the first chip.

[0189] In an optional embodiment, the second chip sends an address update task to the first chip, and the address update task includes the base address table T3. The address update task can also include an address update operator, so that the first chip can update the model parameter table T0 by using the address update operator.

[0190] S1907. The first chip receives the base address table T3 sent by the second chip.

[0191] After receiving the base address table T3, the first chip can store the base address table T3 in the memory of the first chip.

[0192] S1908. The first chip updates the model parameter table T0 according to the base address table T3, the address index table T1 and the address offset table T2.

[0193] The implementation process of S1901 to S1908 can refer to the implementation process of S901 to S904 described above, which will not be repeated here. It should be noted that S1901 to S1904 are optional steps of the embodiments of the present application. The first chip and the second chip perform S1901 to S1904 in the model loading stage of the target model, and then each time the model parameter table of the target model is updated, the first chip and the second chip perform S1905 to S1908.

[0194] The above is an introduction to the method embodiments of the present application. Next, the device embodiments of the present application will be introduced. The device of the present application is used to execute the method of the present application. For details not disclosed in the device embodiments of the present application, please refer to the method embodiments.

[0195] Please refer to FIG. 20, which shows a schematic diagram of a model parameter table updating device 200 according to an embodiment of the present application. The updating device 200 is applied to a first chip. For example, the updating device 200 is the first chip or part of the components in the first chip. As shown in FIG. 20, the updating device 200 includes a first obtaining module 201 and an updating module 202.

[0196] The first obtaining module 201 is configured to, in a model execution stage of a target model, obtain a base address table of the target model, the base address table being generated according to a current execution memory of the target model, the current execution memory being a memory allocated in a memory of the first chip for a current execution of the target model. The function implementation of the first obtaining module 201 can refer to the related description in S903.

[0197] The updating module 202 is configured to update a model parameter table of the target model according to the base address table, an address index table of the target model and an address offset table of the target model, the address index table, the address offset table and the model parameter table being in the memory of the first chip, the address index table including at least one address index table item, the address offset table including at least one address offset table item, the model parameter table including at least one memory address table item, each address index table item corresponding to one address offset table item, each address index table item being used to indicate one base address table item in the base address table, and each address index table item corresponding to at least one memory address table item in the model parameter table. The function implementation of the updating module 202 can refer to the related description in S904.

[0198] Optionally, the updating module 202 is configured to update the model parameter table by using an address updating operator according to the base address table, the address index table and the address offset table.

[0199] Optionally, please continue to refer to FIG. 20, the updating device 200 further includes a second obtaining module 203 configured to obtain an address updating operator.

[0200] Optionally, the second obtaining module 203 is configured to receive an address updating task sent by the second chip, and the address updating task includes the address updating operator.

[0201] Optionally, the base address table is generated by the second chip according to the memory, and the address updating task further includes the base address table, and the first obtaining module 201 is configured to obtain the base address table from the address updating task.

[0202] Optionally, the base address table includes at least one base address table item, each base address table item is configured to record a base address, each address offset table item is configured to record an address offset, and each memory address table item is configured to record a memory address of a target model, the memory address table items in the model parameter table include a first memory address table item, and the updating module 202 is configured to: determine a first address index table item in the address index table according to the first memory address table item, the first address index table item corresponds to the first memory address table item; determine a first base address table item in the base address table according to the first address index table item, the first base address table item is a base address table item indicated by the first address index table item; determine a first address offset table item in the address offset table according to the first address index table item, the first address offset table item corresponds to the first address index table item; and update a memory address recorded in the first memory address table item according to a base address recorded in the first base address table item and an address offset recorded in the first address offset table item.

[0203] Optionally, please continue to refer to FIG. 20, the updating device 20 further includes a third obtaining module 204 and a storage module 205.

[0204] The third obtaining module 204 is configured to obtain the address index table, the address offset table and the model parameter table in a model loading stage of the target model. The function implementation of the third obtaining module 204 can refer to the related description in S901.

[0205] The storage module 205 is configured to store the address index table, the address offset table and the model parameter table in the memory of the first chip in the model loading stage. The function implementation of the storage module 205 can refer to the related description in S902.

[0206] Optionally, the address index table, the address offset table and the model parameter table are generated by the second chip in a model loading stage, and the third obtaining module 204 is configured to receive the address index table, the address offset table and the model parameter table sent by the second chip in the model loading stage.

[0207] Optionally, the content of the address index table and the content of the address offset table are both unchanged.

[0208] Optionally, the base address table includes at least one base address table entry, each base address table entry is configured to record a base address, the current execution memory includes at least one memory segment, and the base address in the base address table includes a start address of each memory segment in the at least one memory segment.

[0209] Optionally, the at least one address index table entry includes an absolute address index table entry, the memory address table entry corresponding to the absolute address index table entry in the model parameter table is configured to record an unchanged memory address of the target model, the at least one base address table entry includes an absolute base address table entry, the absolute address index table entry is configured to indicate the absolute base address table entry, and the base address recorded in the absolute base address table entry is an absolute base address.

[0210] Optionally, the model parameter table includes at least one fixed memory address table entry and at least one non-fixed memory address table entry, the at least one fixed memory address table entry is configured to record a memory address of the target model that does not need to be updated, and the at least one non-fixed memory address table entry is configured to record a memory address of the target model that needs to be updated. That is, the memory address recorded in the memory address table entry in the model parameter table includes the memory address of the target model that needs to be updated and the memory address of the target model that does not need to be updated.

[0211] Optionally, the target model includes at least one operator, and the current execution memory includes memory of each operator in the at least one operator, and the memory of each operator includes at least one of operator input memory, operator output memory or operator workspace memory.

[0212] The memory address of the target model includes an address of the memory of each operator in the at least one operator.

[0213] Optionally, the target model is a neural network model. For example, the target model is a static shape model.

[0214] Optionally, the first chip includes any one of the following: an NPU chip, a GPU chip, a TPU chip, an IPU chip and a VPU chip.

[0215] Optionally, the second chip includes a CPU chip.

[0216] In summary, the technical scheme provided by the embodiments of the present application includes the address index table of the target model, the address offset table of the target model and the model parameter table of the target model in the memory of the first chip in the model execution stage of the target model. In the model execution stage, the first chip updates the model parameter table according to the base address table, the address index table and the address offset table of the target model after obtaining the base address table of the target model. In this way, in the process of updating the model parameter table of the target model in the model execution stage of the target model, the H2D copy of the model parameter table is not required, and the operator traversal of the target model is not required, and therefore, the time consumption of updating the model parameter table of the target model is short, and the updating efficiency is high.

[0217] Please refer to FIG. 21, which shows a schematic diagram of another model parameter table updating device 210 provided by the embodiments of the present application. The updating device 210 is applied to the second chip. For example, the updating device 210 is the second chip or part of the components in the second chip. As shown in FIG. 21, the updating device 210 includes a generating module 211 and a sending module 212.

[0218] The generating module 211 is configured to generate a base address table of a target model according to a current execution memory of the target model in a model execution stage of the target model, the current execution memory being a memory allocated in a memory of a first chip for a current execution of the target model.

[0219] The sending module 212 is configured to send the base address table to the first chip, so that the first chip updates a model parameter table of the target model according to the base address table, an address index table of the target model and an address offset table of the target model, the address index table, the address offset table and the model parameter table being in the memory of the first chip, the address index table including at least one address index table item, the address offset table including at least one address offset table item, the model parameter table including at least one memory address table item, each address index table item corresponding to one address offset table item, and each address index table item corresponding to at least one memory address table item in the model parameter table.

[0220] Optionally, please continue to refer to FIG. 11, the updating device 210 further includes an allocating module 213 configured to allocate the current execution memory in the memory of the first chip in the model execution stage.

[0221] Optionally, the sending module 212 is configured to send an address updating task to the first chip in the model execution stage, the address updating task including an address updating operator, so that the first chip updates the model parameter table by using the address updating operator according to the base address table, the address index table and the address offset table.

[0222] Optionally, the generating module 211 is further configured to generate the address index table, the address offset table and the model parameter table in a model loading stage of the target model; and the sending module 212 is further configured to send the address index table, the address offset table and the model parameter table to the first chip in the model loading stage.

[0223] Optionally, the content of the address index table and the content of the address offset table are both unchanged.

[0224] Optionally, the base address table includes at least one base address table entry, each base address table entry is configured to record a base address, the current execution memory includes at least one memory segment, and the base address in the base address table includes a start address of each memory segment in the at least one memory segment.

[0225] Optionally, the at least one address index table entry includes an absolute address index table entry, a memory address table entry corresponding to the absolute address index table entry in the model parameter table is configured to record an unchanged memory address of the target model, the at least one base address table entry includes an absolute base address table entry, the absolute address index table entry is configured to indicate the absolute base address table entry, and the base address recorded in the absolute base address table entry is an absolute base address.

[0226] Optionally, the model parameter table includes at least one fixed memory address table entry and at least one non-fixed memory address table entry, the at least one fixed memory address table entry is configured to record a memory address of the target model that does not need to be updated, and the at least one non-fixed memory address table entry is configured to record a memory address of the target model that needs to be updated. That is, the memory address recorded in the memory address table entry in the model parameter table includes the memory address of the target model that needs to be updated and the memory address of the target model that does not need to be updated.

[0227] Optionally, the target model includes at least one operator, and the current execution memory includes memory of each operator in the at least one operator, and the memory of each operator includes at least one of operator input memory, operator output memory or operator workspace memory.

[0228] The memory address of the target model includes an address of the memory of each operator in the at least one operator.

[0229] Optionally, the target model is a neural network model. For example, the target model is a static shape model.

[0230] Optionally, the first chip includes any one of an NPU chip, a GPU chip, a TPU chip, an IPU chip and a VPU chip.

[0231] Optionally, the second chip includes a CPU chip.

[0232] The sending module 212 can be an interface of the second chip.

[0233] To sum up, the technical scheme provided by the embodiments of the present application, in the model execution stage of the target model, the memory of the first chip includes the address index table of the target model, the address offset table of the target model and the model parameter table of the target model. The second chip generates the base address table of the target model in the model execution stage, and sends the base address table to the first chip. The first chip updates the model parameter table according to the base address table, the address index table and the address offset table in the model execution stage. In this way, in the process of updating the model parameter table of the target model in the model execution stage of the target model, there is no need to perform H2D copy on the model parameter table, and there is no need to traverse the operators of the target model. Therefore, the time consumption of updating the model parameter table of the target model is shorter, and the updating efficiency is higher.

[0234] It should be understood that the updating device of the model parameter table provided by the embodiments of the present application can also be implemented by an application-specific integrated circuit (ASIC) or a programmable logic device (PLD). The PLD can be a complex programmable logic device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL) or any combination thereof. The updating method of the model parameter table provided by the above-mentioned method embodiments can also be implemented by software. When the updating method of the model parameter table provided by the above-mentioned method embodiments is implemented by software, each module in the above-mentioned updating device of the model parameter table can also be a software module.

[0235] Based on the same inventive concept, the embodiments of the present application provide a computing system, which includes a first chip and a second chip. The first chip includes the updating device 200 as shown in FIG. 20, and the second chip includes the updating device 210 as shown in claim 21.

[0236] In an optional embodiment, the computing system is a system of a computing device (i.e., an internal system of the computing device), and the computing system further includes a memory, a communication interface, a bus and the like structure.

[0237] As an example, refer to FIG. 22, which shows a schematic diagram of a computing system 220 provided by an embodiment of the present application. FIG. 22 illustrates the system of the computing system 220 as an example of a computing device. The computing system 220 includes a first chip 221, a second chip 222, a memory 223, a communication interface 224, and a bus 225. The first chip 221, the second chip 222, the memory 223, and the communication interface 224 are communicatively connected through the bus 225. The connection mode between the first chip 221, the second chip 222, the memory 223, and the communication interface 224 shown in FIG. 22 is only an example, and the first chip 221, the second chip 222, the memory 223, and the communication interface 224 can also be connected in other ways, and the embodiments of the present application do not limit the connection mode of the first chip 221, the second chip 222, the memory 223, and the communication interface 224.

[0238] The first chip 221 can be a model execution chip. The first chip 221 is used to execute part of the steps of the method embodiments shown in FIG. 9 and the method embodiments shown in FIG. 19. The first chip 221 updates the model parameter table by executing the method embodiments shown in FIG. 9, or the first chip 221 updates the model parameter table by cooperating with the second chip 222 to execute part of the method embodiments shown in FIG. 19. The first chip 221 can include a memory and a computing unit, and the computing unit executes the related steps in the above method embodiments. The computing unit includes a cube computing unit, a vector computing unit, a CPU computing unit, an SDMA computing unit, etc. The computing unit is also called a computing resource, an accelerator, or a processing core, etc. The first chip includes any one of the following: an NPU chip, a GPU chip, a TPU chip, an IPU chip, a VPU chip.

[0239] The second chip 222 can be a CPU chip. The second chip 222 is used to execute part of the steps of the method embodiments shown in FIG. 19. The second chip 222 updates the model parameter table by cooperating with the first chip 221 to execute part of the method embodiments shown in FIG. 19. The second chip 222 can include multiple CPU cores, for example, the second chip 222 is a multi-core CPU chip.

[0240] The first chip 221 and / or the second chip 222 can execute the related steps in the above method embodiments by reading and executing a computer program (such as the computer program 2230) stored in the memory (such as the memory 223). The first chip 221 and / or the second chip 222 can use data stored in the memory (such as the memory 223) in the process of executing the related steps in the above method embodiments.

[0241] The memory 223 is configured to store a computer program 2230, which can include instructions, data, programs, codes, and the like. The memory 223 can include various types of storage media, such as random access memory (RAM), read-only memory (ROM), non-volatile RAM (NVRAM), programmable ROM (PROM), erasable PROM (EPROM), electrically erasable PROM (EEPROM), flash memory, optical storage, and registers, and the like.

[0242] The communication interface 224 can include an input / output (I / O) interface, a physical interface, and a logical interface, and the like, which are used to realize the interconnection of devices inside the computing system 220, and are used to realize the interconnection between the computing system 220 (i.e., the computing device) and other devices (for example, network devices). The physical interface can be an Ethernet interface, which can be used to realize the interconnection between the computing system 220 (i.e., the computing device) and other devices. The logical interface is the interface inside the computing system 220, which can be used to realize the interconnection of devices inside the computing system 220. It is easy to understand that the communication interface 224 can be used for the computing system 220 (i.e., the computing device) to communicate with other devices. For example, the communication interface 224 is used for the transmission and reception of information between the computing system 220 and other devices.

[0243] The bus 225 can be any type, which is used to realize the interconnection of the first chip 221, the second chip 222, the memory 223, and the communication interface 224. For example, the bus 225 can be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus, and the like. The bus 225 can be divided into an address bus, a data bus, a control bus, and the like. For the convenience of representation, only one thick line is used in FIG. 22, but it does not mean that there is only one bus or only one type of bus.

[0244] The computing system 220 shown in FIG. 22 is only exemplary, and in the implementation process, the computing system 220 can also include other components, which are not listed one by one herein.

[0245] Based on the same inventive concept, the embodiment of the present application provides a computer readable storage medium, which stores a computer program, and the computer program is executed to implement all or part of the steps of the method embodiments shown in FIG. 9 and FIG. 19.

[0246] In one embodiment, the computer program is executed by the first chip or the updating device 200 shown in FIG. 20, and the computer program is executed to implement the steps S1903, S1904, S1907 and S1908 in the method embodiment shown in FIG. 9 and the method embodiment shown in FIG. 19.

[0247] In another embodiment, the computer program is executed by the second chip or the updating device 210 shown in FIG. 21, and the computer program is executed to implement the steps S1901, S1902, S1905 and S1906 in the method embodiment shown in FIG. 19.

[0248] Based on the same inventive concept, the embodiment of the present application provides a computer program product, which includes a program or code, and the program or code is executed to implement all or part of the steps of the method embodiments shown in FIG. 9 and FIG. 19.

[0249] In one embodiment, the program or code is executed by the first chip or the updating device 200 shown in FIG. 20, and the program or code is executed to implement the steps S1903, S1904, S1907 and S1908 in the method embodiment shown in FIG. 9 and the method embodiment shown in FIG. 19.

[0250] In another embodiment, the program or code is executed by the second chip or the updating device 210 shown in FIG. 21, and the program or code is executed to implement the steps S1901, S1902, S1905 and S1906 in the method embodiment shown in FIG. 19.

[0251] In the above embodiments, all or part of the embodiments can be implemented by software, hardware, firmware or any combination thereof. When implemented by software, all or part of the embodiments can be implemented in the form of a computer program product including one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of the present application are generated. The computer can be a general-purpose computer, a computer network, or other programmable apparatus. The computer instructions can be stored in a computer readable storage medium of the computer, or transmitted from one computer readable storage medium to another computer readable storage medium, for example, the computer instructions can be transmitted from one website site, computer, server or data center to another website site, computer, server or data center through wired (such as coaxial cable, optical fiber, digital subscriber line) or wireless (such as infrared, wireless, microwave, etc.) mode. The computer readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server, data center, etc. integrated with one or more available media. The available media can be magnetic media (such as floppy disk, hard disk, magnetic tape), optical media, or semiconductor media (such as solid state disk) and the like.

[0252] It should be understood that the term "at least one" in the present application refers to one or more, and the term "multiple" refers to two or more. In the present application, unless otherwise specified, the symbol " / " generally represents the meaning of or, for example, A / B can represent A or B. The term "and / or" in the present application only describes the association relationship of the associated objects, which means that there can be three relationships, for example, A and / or B can represent three cases of A alone, A and B together, and B alone. In addition, in order to facilitate clear description, the present application uses "first", "second", "third" and the like to distinguish the same items or similar items with basically the same function and role. Those skilled in the art can understand that "first", "second", "third" and the like do not limit the quantity and execution order.

[0253] The method embodiments and device embodiments provided by the embodiments of the present application and other different types of embodiments can be mutually referred to, the order of operation of the method embodiments can be adjusted appropriately, and the operation can be increased or decreased in response to the situation. Any person skilled in the art can easily think of changes within the technical scope disclosed in the present application, which should be covered within the protection scope of the present application, so no further description is given.

[0254] In the corresponding embodiments provided by the present application, it should be understood that the disclosed apparatus, etc. can be implemented by other construction manners. For example, the apparatus embodiments described above are merely illustrative, for example, the division of the modules is merely a logical function division, and actual implementation can have another division manner, for example, a plurality of modules or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the displayed or discussed each other can be indirect coupling or communication connection through some interfaces, apparatuses or modules, and can be electrical or other forms. The modules described as separate components can be or can not be physically separated, and the components described as modules can be or can not be physical modules, and can be located in one place or distributed on a plurality of network nodes. Part or all of the modules can be selected to achieve the purpose of the embodiments according to actual needs.

[0255] The above describes only exemplary embodiments of the present application, but the protection scope of the present application is not limited thereto, and any skilled person in the art can easily think of various equivalent modifications or replacements within the technical range disclosed by the present application, which should be covered by the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A method of updating a model parameter table, characterized by, The method applied to the first chip comprises: In a model execution stage of the target model, a base address table of the target model is acquired, the base address table being generated according to a current execution memory of the target model, the current execution memory being memory allocated in a memory of the first chip for current execution of the target model; According to the base address table, an address index table of the target model and an address offset table of the target model, a model parameter table of the target model is updated, the address index table, the address offset table and the model parameter table being in the memory of the first chip, the address index table comprising at least one address index table entry, the address offset table comprising at least one address offset table entry, the model parameter table comprising at least one memory address table entry, the at least one address index table entry corresponding to the at least one address offset table entry one by one, each address index table entry being used for indicating a base address table entry in the base address table, each address index table entry corresponding to at least one memory address table entry in the model parameter table.

2. The method of claim 1, wherein, The updating of the model parameter table of the target model according to the base address table, the address index table of the target model and the address offset table of the target model comprises: According to the base address table, the address index table and the address offset table, an address updating operator is used to update the model parameter table.

3. The method of claim 2, wherein, The method further comprises acquiring the address updating operator.

4. The method of claim 3, wherein, The acquiring of the address updating operator comprises: Receiving an address updating task sent by a second chip, the address updating task comprising the address updating operator.

5. The method of claim 4, wherein, The base address table being generated by the second chip according to the current execution memory, the address updating task further comprising the base address table, The acquiring of the base address table of the target model comprises acquiring the base address table from the address updating task.

6. The method according to any one of claims 1 to 5, wherein The base address table comprises at least one base address table entry, each base address table entry being used for recording a base address, each address offset table entry being used for recording an address offset, and each memory address table entry being used for recording a memory address of the target model, the memory address table entry in the model parameter table comprising a first memory address table entry, and the updating of the model parameter table of the target model according to the base address table, the address index table of the target model and the address offset table of the target model comprises: According to the first memory address table entry, a first address index table entry is determined in the address index table, the first address index table entry corresponding to the first memory address table entry; According to the first address index table entry, a first base address table entry is determined in the base address table, the first base address table entry being the base address table entry indicated by the first address index table entry; According to the first address index table entry, a first address offset table entry is determined in the address offset table, the first address offset table entry corresponding to the first address index table entry; According to the base address recorded in the first base address table item and the address offset recorded in the first address offset table item, the memory address recorded in the first memory address table item is updated.

7. The method according to any one of claims 1 to 6, characterized in that, The method further comprises: In a model loading stage of the target model: obtain the address index table, the address offset table and the model parameter table; store the address index table, the address offset table and the model parameter table in the memory of the first chip.

8. The method of claim 7, wherein, The address index table, the address offset table and the model parameter table are generated by a second chip in the model loading stage, and the obtaining of the address index table, the address offset table and the model parameter table comprises: receiving the address index table, the address offset table and the model parameter table sent by the second chip.

9. A method of updating a model parameter table, characterized by, Applied to a second chip, the method comprises: In a model execution stage of a target model, a base address table of the target model is generated according to a current execution memory of the target model, the current execution memory being memory allocated in a memory of a first chip for current execution of the target model; send the base address table to the first chip, so that the first chip updates a model parameter table of the target model according to the base address table, an address index table of the target model and an address offset table of the target model, the address index table, the address offset table and the model parameter table all being in the memory of the first chip, the address index table comprising at least one address index table item, the address offset table comprising at least one address offset table item, the model parameter table comprising at least one memory address table item, each address index table item corresponding to one base address table item in the base address table, and each address index table item corresponding to at least one memory address table item in the model parameter table.

10. The method of claim 9, wherein, The method further comprises: In the model execution stage, allocate the current execution memory in the memory of the first chip.

11. The method according to claim 9 or 10, characterized in that, The method further comprises: in the model execution stage, send an address updating task to the first chip, the address updating task comprising an address updating operator, so that the first chip updates the model parameter table according to the base address table, the address index table and the address offset table by using the address updating operator.

12. The method according to any one of claims 9 to 11, characterized in that, The method further comprises: In a model loading stage of the target model: generate the address index table, the address offset table and the model parameter table; send the address index table, the address offset table and the model parameter table to the first chip.

13. An apparatus for updating a model parameter table, characterized by: Applied to a first chip, the updating device comprises: a first obtaining module, configured to, in a model execution stage of a target model, obtain a base address table of the target model, the base address table being generated according to a current execution memory of the target model, the current execution memory being memory allocated in a memory of the first chip for current execution of the target model; The updating module is configured to update the model parameter table of the target model according to the base address table, the address index table, and the address offset table.

14. The updating apparatus according to claim 13, characterized by The updating module is configured to update the model parameter table using an address updating operator according to the base address table, the address index table, and the address offset table.

15. The updating apparatus according to claim 14, characterized by, The updating device further includes: The second obtaining module is configured to obtain the address updating operator.

16. The updating apparatus according to claim 15, characterized by The second obtaining module is configured to receive an address updating task sent by the second chip, and the address updating task includes the address updating operator.

17. The updating apparatus according to claim 16, characterized by The base address table is generated by the second chip according to the memory of the current execution, and the address updating task further includes the base address table. The first obtaining module is configured to obtain the base address table from the address updating task.

18. The updating device of any one of claims 13-17, wherein: The base address table includes at least one base address table entry, each of which is configured to record a base address, each of the address offset table entries is configured to record an address offset, and each of the memory address table entries is configured to record a memory address of the target model. The memory address table entry in the model parameter table includes a first memory address table entry, and the updating module is configured to: determine a first address index table entry in the address index table according to the first memory address table entry, the first address index table entry corresponding to the first memory address table entry; determine a first base address table entry in the base address table according to the first address index table entry, the first base address table entry being the base address table entry indicated by the first address index table entry; determine a first address offset table entry in the address offset table according to the first address index table entry, the first address offset table entry corresponding to the first address index table entry; update the memory address recorded in the first memory address table entry according to the base address recorded in the first base address table entry and the address offset recorded in the first address offset table entry.

19. The updating device according to any one of claims 13 to 18, characterized in that, The updating device further includes: The third obtaining module is configured to obtain the address index table, the address offset table, and the model parameter table during a model loading stage of the target model. The storage module is configured to store the address index table, the address offset table, and the model parameter table in the memory of the first chip during the model loading stage.

20. The updating apparatus according to claim 19, wherein The address index table, the address offset table and the model parameter table are generated by a second chip in the model loading stage, and the third obtaining module is configured to receive the address index table, the address offset table and the model parameter table sent by the second chip in the model loading stage.

21. An apparatus for updating a model parameter table, characterized by: The updating device is applied to a second chip, and the updating device comprises: The generating module is configured to generate a basic address table of a target model according to a current execution memory of the target model in a model execution stage of the target model, the current execution memory being memory allocated in a memory of a first chip for current execution of the target model; The sending module is configured to send the basic address table to the first chip, so that the first chip updates a model parameter table of the target model according to the basic address table, an address index table of the target model and an address offset table of the target model, the address index table, the address offset table and the model parameter table being in the memory of the first chip, the address index table comprising at least one address index table item, the address offset table comprising at least one address offset table item, the model parameter table comprising at least one memory address table item, each of the address index table items corresponding to one of the address offset table items, and each of the address index table items being used to indicate one of the basic address table items, and each of the address index table items corresponding to at least one of the memory address table items in the model parameter table.

22. The updating apparatus according to claim 21, wherein The updating device further comprises: The allocating module is configured to allocate the current execution memory in the memory of the first chip in the model execution stage.

23. The updating apparatus according to claim 21 or 22, characterized by, The sending module is configured to send an address updating task to the first chip in the model execution stage, the address updating task comprising an address updating operator, so that the first chip updates the model parameter table by using the address updating operator according to the basic address table, the address index table and the address offset table.

24. The updating device of any one of claims 21 to 23, wherein The generating module is further configured to generate the address index table, the address offset table and the model parameter table in a model loading stage of the target model; The sending module is further configured to send the address index table, the address offset table and the model parameter table to the first chip in the model loading stage.

25. The method according to any one of claims 1 to 12, or the updating apparatus according to any one of claims 13 to 24, characterized in that, The content of the address index table and the content of the address offset table are both unchanged.

26. The method of any one of claims 1 to 12, 25, or the updating apparatus of any one of claims 13 to 25, characterized in that, The basic address table comprises at least one basic address table item, each of the basic address table items being used to record a basic address, and the current execution memory comprises at least one memory segment, and the basic address in the basic address table comprises a start address of each of the at least one memory segment.

27. The method or updating device of claim 26, wherein, The at least one address index table item includes an absolute address index table item, a memory address item corresponding to the absolute address index table item in the model parameter table is used to record an unchanged memory address of the target model, the at least one base address table item includes an absolute base address table item, the absolute address index table item is used to indicate the absolute base address table item, and a base address recorded in the absolute base address table item is an absolute base address. Alternatively, the model parameter table includes at least one fixed memory address table item, and the at least one fixed memory address table item is used to record a memory address of the target model that does not need to be updated.

28. The method of any one of claims 1-12, 25-27, or the updating apparatus of any one of claims 13-27, characterized in that, The first chip includes any one of the following: a neural network processor (NPU) chip, a graphics processing unit (GPU) chip, a tensor processing unit (TPU) chip, an intelligent processing unit (IPU) chip, and a vector processing unit (VPU) chip.

29. A computing system, comprising: The method comprises the following steps: The first chip and the second chip are included. The first chip includes the updating device according to any one of claims 13 to 20 and 25 to 28. The second chip includes the updating device according to any one of claims 21 to 28.

30. A computer-readable storage medium, characterized in that, The computer program stored in the computer readable storage medium is executed to implement the method according to any one of claims 1 to 12 and 25 to 28.

31. A computer program product, characterised in that, The computer program product includes a program or code, and the program or code is executed to implement the method according to any one of claims 1 to 12 and 25 to 28.

Images