Model parameter calling method and device, equipment and readable storage medium

Abstract

The application discloses a model parameter calling method and device, equipment and a readable storage medium. In the scheme, the calculation of each parameter scheduling unit of the model is executed in series. The scheme is applied to a device using a model. When the model executes the calculation of each parameter scheduling unit, the device loads the parameters of all parameter scheduling units of the model stored in the storage memory into the running memory on demand. Through the above scheme, the dynamic reuse of the running memory of the device can be realized, the parameters of the target parameter scheduling unit being executed and the parameters of the next parameter scheduling unit adjacent to the target parameter scheduling unit are ensured to be stored in the running memory, so that the calculation of the target parameter scheduling unit being executed is not affected and the required parameters for the target parameter scheduling unit to be executed are prepared, thereby adapting the neural network model with a large amount of model parameters to the intelligent hardware device with a small running memory configuration.

Description

Technical Field

[0001] This application relates to the field of machine translation technology, and more specifically, to a method, apparatus, device, and readable storage medium for calling model parameters. Background Technology

[0002] With the development of deep learning, more and more smart hardware devices can provide services based on neural network models. For example, some scanning dictionary pens and smart desk lamps that can integrate translation functions can provide translation services based on machine translation models. Before implementing a service based on a neural network model, the model parameters stored in the smart hardware device's memory need to be loaded into the device's runtime memory. Currently, the common approach is to load all the model parameters stored in the smart hardware device's memory into its runtime memory at once.

[0003] For neural network models implementing certain services (such as translation services), increasing the number of model parameters will significantly improve the corresponding service performance. However, in some scenarios, due to cost and other considerations, smart hardware devices generally have limited RAM. Because of this limited RAM, for neural network models with a larger number of parameters than the device's RAM, it's impossible to load all the model parameters stored in the smart hardware's memory into its RAM at once. This results in the smart hardware being unable to adapt to neural network models with a large number of parameters.

[0004] Therefore, how to provide a method for calling model parameters to adapt neural network models with a large number of model parameters to smart hardware devices with limited memory configuration has become a technical problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0005] In view of the above problems, this application proposes a method, apparatus, device, and readable storage medium for calling model parameters. The specific solution is as follows:

[0006] A method for invoking model parameters, wherein the model includes multiple parameter scheduling units, the calculations of each parameter scheduling unit are executed serially, the method is applied to a device employing the model, the device includes storage memory and runtime memory, the storage memory stores the parameters of all parameter scheduling units of the model, and the method includes:

[0007] When performing calculations for each parameter scheduling unit, the parameters of all parameter scheduling units of the model stored in the storage memory are loaded into the running memory as needed, so that the running memory stores the parameters of N parameter scheduling units; where N is an integer greater than or equal to 2, the storage space occupied by the parameters of the N parameter scheduling units is less than or equal to the running memory of the device, and the parameters of the N parameter scheduling units include the parameters of the target parameter scheduling unit being executed, and the parameters of the next parameter scheduling unit adjacent to the target parameter scheduling unit.

[0008] Optionally, before performing the calculations of each parameter scheduling unit, the method further includes:

[0009] Determine the scheduling units for each parameter of the model;

[0010] The model is initialized by storing the parameters of all parameter scheduling units of the model into the device's storage memory, and loading the parameters of the first N parameter scheduling units of the model into the device's running memory, where N is an integer greater than or equal to 2, and the storage space occupied by the parameters of the first N parameter scheduling units is less than or equal to the device's running memory.

[0011] Optionally, the parameter scheduling unit for determining the model includes:

[0012] The multiple sub-networks of the model are treated as a single parameter scheduling unit.

[0013] Optionally, the parameter scheduling unit for determining the model includes:

[0014] One sub-network of the model is used as a parameter scheduling unit.

[0015] Optionally, the process of performing the calculation of the target parameter scheduling unit includes:

[0016] The parameters of the target parameter scheduling unit are obtained from the device's running memory;

[0017] The target parameter scheduling unit performs calculations based on the parameters of the target parameter scheduling unit, and updates the device's running memory when there are no parameters of the next parameter scheduling unit adjacent to the target parameter scheduling unit in the device's running memory.

[0018] Optionally, when the device's running memory stores parameters of two parameter scheduling units, and the device's running memory includes the parameters of the target parameter scheduling unit and the parameters of the previous parameter scheduling unit adjacent to the target parameter scheduling unit;

[0019] Updating the device's running memory includes:

[0020] The parameters of the next parameter scheduling unit adjacent to the target parameter scheduling unit are loaded from the device's storage memory into the device's running memory, overwriting the parameters of the previous parameter scheduling unit adjacent to the target parameter scheduling unit.

[0021] Optionally, when the device's running memory stores parameters of two or more parameter scheduling units, and the device's running memory includes the parameters of the target parameter scheduling unit, as well as the parameters of multiple parameter scheduling units that are calculated before the target parameter scheduling unit;

[0022] Updating the device's running memory includes:

[0023] The parameters of the next parameter scheduling unit adjacent to the target parameter scheduling unit are loaded from the device's storage memory into the device's running memory, overwriting the parameters of the first parameter scheduling unit that was calculated in at least one parameter scheduling unit that was calculated before the target parameter scheduling unit.

[0024] A model parameter calling device, wherein the model includes multiple parameter scheduling units, the calculations of each parameter scheduling unit are executed serially, the device is applied to a device employing the model, the device includes storage memory and running memory, the storage memory stores the parameters of all parameter scheduling units of the model, and the device includes:

[0025] The computation execution module is used to load the parameters of all parameter scheduling units of the model stored in the storage memory into the running memory as needed when performing the computation of each parameter scheduling unit, so that the running memory stores the parameters of N parameter scheduling units; where N is an integer greater than or equal to 2, the storage space occupied by the parameters of the N parameter scheduling units is less than or equal to the running memory of the device, and the parameters of the N parameter scheduling units include the parameters of the target parameter scheduling unit being executed, and the parameters of the next parameter scheduling unit adjacent to the target parameter scheduling unit.

[0026] Optionally, the device further includes:

[0027] The parameter scheduling unit determination module is used to determine each parameter scheduling unit of the model before performing the calculation of each parameter scheduling unit.

[0028] An initialization module is used to initialize the model, store the parameters of all parameter scheduling units of the model in the device's storage memory, and load the parameters of the first N parameter scheduling units of the model into the device's running memory, where N is an integer greater than or equal to 2, and the storage space occupied by the parameters of the first N parameter scheduling units is less than or equal to the device's running memory.

[0029] Optionally, the parameter scheduling unit determination module includes:

[0030] The first determining module is used to treat multiple sub-networks of the model as a parameter scheduling unit.

[0031] Optionally, the parameter scheduling unit determination module includes:

[0032] The second determining module is used to treat a sub-network of the model as a parameter scheduling unit.

[0033] Optionally, the computation execution module includes:

[0034] The calculation and execution module of the target parameter scheduling unit;

[0035] The calculation and execution module of the target parameter scheduling unit includes:

[0036] The acquisition submodule is used to acquire the parameters of the target parameter scheduling unit from the device's running memory;

[0037] The runtime memory update submodule is used to perform the calculation of the target parameter scheduling unit based on the parameters of the target parameter scheduling unit, and update the runtime memory of the device when there are no parameters of the next parameter scheduling unit adjacent to the target parameter scheduling unit in the runtime memory of the device.

[0038] Optionally, the runtime memory update submodule includes:

[0039] The first running memory update submodule is used to, when the running memory of the device stores parameters of two parameter scheduling units, including the parameters of the target parameter scheduling unit and the parameters of the previous parameter scheduling unit adjacent to the target parameter scheduling unit, load the parameters of the next parameter scheduling unit adjacent to the target parameter scheduling unit from the device's storage memory into the device's running memory, overwriting the parameters of the previous parameter scheduling unit adjacent to the target parameter scheduling unit.

[0040] Optionally, the runtime memory update submodule includes:

[0041] The second runtime memory update submodule is used when the device's runtime memory stores parameters of two or more parameter scheduling units, including the parameters of the target parameter scheduling unit and the parameters of multiple parameter scheduling units that performed calculations before the target parameter scheduling unit; the next parameter scheduling unit adjacent to the target parameter scheduling unit is loaded from the device's storage memory into the device's runtime memory, overwriting the parameters of the first parameter scheduling unit that performed calculations before the target parameter scheduling unit.

[0042] A model parameter retrieval device, including a memory and a processor;

[0043] The memory is used to store programs;

[0044] The processor is used to execute the program and implement the various steps of the model parameter calling method as described above.

[0045] A readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the model parameter invocation method as described above.

[0046] Using the above technical solution, this application discloses a model parameter calling method, apparatus, device, and readable storage medium. In this solution, the model includes multiple parameter scheduling units, and the calculations of each parameter scheduling unit are executed serially. The method is applied to a device that uses the model. The device includes storage memory and running memory. The storage memory stores the parameters of all parameter scheduling units of the model. When the model executes the calculations of each parameter scheduling unit, the parameters of all parameter scheduling units of the model stored in the storage memory are loaded into the running memory as needed, so that the running memory stores the parameters of N parameter scheduling units, where N is an integer greater than or equal to 2. The storage space occupied by the parameters of the N parameter scheduling units is less than or equal to the running memory of the device. The parameters of the N parameter scheduling units include the parameters of the currently executing target parameter scheduling unit and the parameters of the next parameter scheduling unit adjacent to the target parameter scheduling unit. The above scheme enables dynamic reuse of the device's running memory, ensuring that the running memory stores the parameters of the currently executing target parameter scheduling unit, as well as the parameters of the next parameter scheduling unit adjacent to the target parameter scheduling unit. This does not affect the calculation of the currently executing target parameter scheduling unit and prepares the necessary parameters for the target parameter scheduling unit to be executed, thus enabling the adaptation of neural network models with a large number of model parameters on smart hardware devices with small running memory configurations. Attached Figure Description

[0047] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit the scope of this application. Furthermore, the same reference numerals denote the same parts throughout the drawings. In the drawings:

[0048] Figure 1 This is a schematic diagram of the structure of a model disclosed in an embodiment of this application;

[0049] Figure 2 This is a flowchart illustrating the method for calculating the target parameter scheduling unit of the model execution disclosed in an embodiment of this application;

[0050] Figure 3 This is a schematic diagram of a transformer structure provided in an embodiment of this application;

[0051] Figure 4 A schematic diagram illustrating parameter transfer between PSRAM and SRAM during the parameter calling process of a transformer model;

[0052] Figure 5 This is a schematic diagram of a model parameter calling device disclosed in an embodiment of this application;

[0053] Figure 6 This is a hardware structure block diagram of a model parameter calling device provided in an embodiment of this application. Detailed Implementation

[0054] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0055] In order to adapt neural network models with a large number of parameters to smart hardware devices with limited RAM, the inventors of this case conducted research. One existing approach is to reduce the number of model parameters of the neural network model by using methods such as parameter sharing, model quantization, and adaptive softmax, so as to adapt the neural network model with a large number of model parameters to smart hardware devices with limited RAM. However, the above approach will affect the performance of the neural network model.

[0056] In view of the problems with the above approach, the inventors of this case conducted in-depth research and finally proposed a method for calling model parameters.

[0057] The following embodiments will be used to describe the model parameter calling method provided in this application.

[0058] It should be noted that the model parameter calling method provided in this application is applicable to models with a specific structure. The model with a specific structure includes multiple parameter scheduling units, and the calculation of each parameter scheduling unit is executed serially. There can be many types of models with this specific structure, such as the transformer, which is currently the most commonly used model structure in machine translation.

[0059] For ease of understanding, please refer to Figure 1 , Figure 1 This is a schematic diagram of the structure of a model disclosed in an embodiment of this application. Figure 1 As shown, the model includes parameter scheduling unit 1, parameter scheduling unit 2, ..., parameter scheduling unit m. The calculations of parameter scheduling unit 1, parameter scheduling unit 2, parameter scheduling unit 3, ..., parameter scheduling unit m are performed sequentially. That is, the input of the model first enters parameter scheduling unit 1, and the calculation of parameter scheduling unit 1 begins. Only after the calculation of parameter scheduling unit 1 is completed will the calculation of parameter scheduling unit 2 be performed, and only after the calculation of parameter scheduling unit 2 is completed will the calculation of parameter scheduling unit 3 be performed, and so on, until the calculation of parameter scheduling unit m is completed, at which point the output of the entire model is obtained.

[0060] It should be further noted that the model parameter scheduling method provided in this application is applied to a device employing the aforementioned specific structure. This device includes storage memory and running memory, which can be implemented using various forms of memory, such as SRAM (Static Random-Access Memory), PSRAM (Pseudo Static Random-Access Memory), FLASH memory, etc. This application does not impose any limitations on these. Considering the cost and bandwidth of various memory types, in this application, as one possible implementation, the device's storage memory can be PSRAM, and the running memory can be SRAM.

[0061] In this application, the model can be loaded into the device first. Specifically, the parameter scheduling units of the model can be determined first, and then the model can be initialized. The parameters of all parameter scheduling units of the model are stored in the device's storage memory, and the parameters of the first N parameter scheduling units of the model are loaded into the device's running memory, where N is an integer greater than or equal to 2. The storage space occupied by the parameters of the first N parameter scheduling units is less than the device's running memory. When N is 2, the running memory usage is minimized.

[0062] It should be noted that a model generally consists of multiple subnetworks. Therefore, in this application, the subnetworks of the model can be used as the partitioning unit to divide the model into various parameter scheduling units. As one possible implementation, multiple subnetworks of the model can be treated as a single parameter scheduling unit. Considering minimizing runtime memory usage, as another possible implementation, each subnetwork of the model can also be treated as a single parameter scheduling unit.

[0063] The device's running memory can be used as shared memory. When the model executes the calculations of each parameter scheduling unit, the parameters of all parameter scheduling units of the model stored in the storage memory are loaded into the running memory as needed, so that the running memory stores the parameters of N parameter scheduling units; where N is an integer greater than or equal to 2, and the storage space occupied by the parameters of the N parameter scheduling units is less than or equal to the running memory of the device. The parameters of the N parameter scheduling units include the parameters of the target parameter scheduling unit being executed, and the parameters of the next parameter scheduling unit adjacent to the target parameter scheduling unit.

[0064] The above scheme enables dynamic reuse of the device's running memory, ensuring that the running memory stores the parameters of the currently executing target parameter scheduling unit, as well as the parameters of the next parameter scheduling unit adjacent to the target parameter scheduling unit. This does not affect the calculation of the currently executing target parameter scheduling unit and prepares the necessary parameters for the target parameter scheduling unit to be executed, thus enabling the adaptation of neural network models with a large number of model parameters on smart hardware devices with small running memory configurations.

[0065] In another embodiment of this application, the calculation process of the execution target parameter scheduling unit is described in detail. It should be noted that the calculation process of each parameter scheduling unit in the model is carried out in the following manner.

[0066] Reference Figure 2 , Figure 2 This is a flowchart illustrating the method for calculating the target parameter scheduling unit of the model execution disclosed in an embodiment of this application. The method may include:

[0067] Step S201: Obtain the parameters of the target parameter scheduling unit from the device's running memory.

[0068] As indicated in the above embodiments, when performing calculations for each parameter scheduling unit, the parameters of all parameter scheduling units of the model stored in the storage memory are loaded into the running memory as needed, so that the running memory stores the parameters of N parameter scheduling units. The parameters of the N parameter scheduling units include the parameters of the target parameter scheduling unit being executed. Therefore, when performing calculations for the target parameter scheduling unit, the parameters of the target parameter scheduling unit can be obtained from the running memory of the device.

[0069] Step S202: Execute the calculation of the target parameter scheduling unit based on the parameters of the target parameter scheduling unit, and update the running memory of the device when there are no parameters of the next parameter scheduling unit adjacent to the target parameter scheduling unit in the running memory of the device.

[0070] After retrieving the parameters of the target parameter scheduling unit from the device's RAM, the calculation of the target parameter scheduling unit can be performed based on these parameters. Simultaneously, if the device's RAM does not contain parameters for the next parameter scheduling unit adjacent to the target parameter scheduling unit, the device's RAM is updated to ensure that the parameters of the N parameter scheduling units include those of the next parameter scheduling unit adjacent to the target parameter scheduling unit. If the device's RAM contains parameters for the next parameter scheduling unit adjacent to the target parameter scheduling unit, then no update operation is required to update the device's RAM.

[0071] As one possible implementation, when the device's running memory stores parameters of two parameter scheduling units, and the device's running memory includes the parameters of the target parameter scheduling unit and the parameters of the previous parameter scheduling unit adjacent to the target parameter scheduling unit;

[0072] Updating the device's running memory includes:

[0073] The parameters of the next parameter scheduling unit adjacent to the target parameter scheduling unit are loaded from the device's storage memory into the device's running memory, overwriting the parameters of the previous parameter scheduling unit adjacent to the target parameter scheduling unit.

[0074] As another possible implementation, when the device's running memory stores parameters of two or more parameter scheduling units, the device's running memory includes the parameters of the target parameter scheduling unit, as well as the parameters of multiple parameter scheduling units that are calculated before the target parameter scheduling unit;

[0075] Updating the device's running memory includes:

[0076] The parameters of the next parameter scheduling unit adjacent to the target parameter scheduling unit are loaded from the device's storage memory into the device's running memory, overwriting the parameters of the first parameter scheduling unit that was calculated in at least one parameter scheduling unit that was calculated before the target parameter scheduling unit.

[0077] To better understand the model parameter calling method provided in this application, this application uses the transformer, the most commonly used model structure in machine translation, as an example to illustrate the solution.

[0078] See Figure 3 , Figure 3 This is a schematic diagram of a transformer structure provided in an embodiment of this application. Figure 3 As shown, the transformer consists of two parts: an encoder and a decoder. The encoder and decoder layers are organized into several network layers with the same structure. Taking the standard transformer basic network structure as an example, it contains 6 encoder layers and 6 decoder layers. Each encoder and decoder layer contains multiple sub-network modules. Specifically, as shown in the figure, each encoder layer contains a multi-head attention network and a feedforward network, and each decoder layer contains a masked multi-head attention network, a multi-head attention network, and a feedforward network.

[0079] In Transformer, the encoder, decoder, and each sub-network module of each layer in the encoder and decoder are executed serially. That is, the next sub-network module can only be executed after the previous sub-network module has been completed. This serial execution characteristic of Transformer makes it compatible with the model parameter calling method provided in this application.

[0080] If a smart hardware device with integrated translation function has SRAM as its running memory and PSRAM as its storage memory, and the hardware device implements translation services based on the transformer model, where each sub-network in the transformer model serves as a parameter scheduling unit.

[0081] First, the transformer model is initialized, with all model parameters stored in PSRAM. The parameters required for the two sub-networks (multi-head attention network and feedforward network) of the first layer of the encoder are loaded into SRAM. When a translation request begins, the multi-head attention network (the first sub-network of the first layer of the encoder) is computed. Once this sub-network is computed, the feedforward network (the second sub-network) is computed. Simultaneously, the parameters corresponding to the multi-head attention network (the first sub-network of the second layer of the encoder) are transferred from PSRAM to SRAM, overwriting the parameters corresponding to the multi-head attention network (the first sub-network of the first layer of the encoder). This process continues until the translation of the sentence is completed.

[0082] For easier understanding, please refer to Figure 4 , Figure 4 This diagram illustrates the parameter transfer between PSRAM and SRAM during the parameter calling process for the transformer model.

[0083] The following describes a model parameter calling device disclosed in an embodiment of this application. The model parameter calling device described below can be referred to in correspondence with the model parameter calling method described above.

[0084] Reference Figure 5 , Figure 5 This is a schematic diagram of a model parameter calling device disclosed in an embodiment of this application. The model includes multiple parameter scheduling units, and the calculations of each parameter scheduling unit are executed serially. The device is applied to a device that uses the model. The device includes storage memory and running memory. The storage memory stores the parameters of all parameter scheduling units of the model, such as... Figure 5 As shown, the model parameter calling device may include:

[0085] The calculation execution module 11 is used to load the parameters of all parameter scheduling units of the model stored in the storage memory into the running memory as needed when performing calculations of each parameter scheduling unit, so that the running memory stores the parameters of N parameter scheduling units; where N is an integer greater than or equal to 2, the storage space occupied by the parameters of the N parameter scheduling units is less than or equal to the running memory of the device, and the parameters of the N parameter scheduling units include the parameters of the target parameter scheduling unit being executed, and the parameters of the next parameter scheduling unit adjacent to the target parameter scheduling unit.

[0086] As one possible implementation, the device further includes:

[0087] The parameter scheduling unit determination module is used to determine each parameter scheduling unit of the model before performing the calculation of each parameter scheduling unit.

[0088] An initialization module is used to initialize the model, store the parameters of all parameter scheduling units of the model in the device's storage memory, and load the parameters of the first N parameter scheduling units of the model into the device's running memory, where N is an integer greater than or equal to 2, and the storage space occupied by the parameters of the first N parameter scheduling units is less than or equal to the device's running memory.

[0089] As one possible implementation, the parameter scheduling unit determination module includes:

[0090] The first determining module is used to treat multiple sub-networks of the model as a parameter scheduling unit.

[0091] As one possible implementation, the parameter scheduling unit determination module includes:

[0092] The second determining module is used to treat a sub-network of the model as a parameter scheduling unit.

[0093] As one possible implementation, the computation execution module includes:

[0094] The calculation and execution module of the target parameter scheduling unit;

[0095] The calculation and execution module of the target parameter scheduling unit includes:

[0096] The acquisition submodule is used to acquire the parameters of the target parameter scheduling unit from the device's running memory;

[0097] The runtime memory update submodule is used to perform the calculation of the target parameter scheduling unit based on the parameters of the target parameter scheduling unit, and update the runtime memory of the device when there are no parameters of the next parameter scheduling unit adjacent to the target parameter scheduling unit in the runtime memory of the device.

[0098] As one possible implementation, the runtime memory update submodule includes:

[0099] The first running memory update submodule is used to, when the running memory of the device stores parameters of two parameter scheduling units, including the parameters of the target parameter scheduling unit and the parameters of the previous parameter scheduling unit adjacent to the target parameter scheduling unit, load the parameters of the next parameter scheduling unit adjacent to the target parameter scheduling unit from the device's storage memory into the device's running memory, overwriting the parameters of the previous parameter scheduling unit adjacent to the target parameter scheduling unit.

[0100] As one possible implementation, the runtime memory update submodule includes:

[0101] The second runtime memory update submodule is used when the device's runtime memory stores parameters of two or more parameter scheduling units, including the parameters of the target parameter scheduling unit and the parameters of multiple parameter scheduling units that performed calculations before the target parameter scheduling unit; the next parameter scheduling unit adjacent to the target parameter scheduling unit is loaded from the device's storage memory into the device's runtime memory, overwriting the parameters of the first parameter scheduling unit that performed calculations before the target parameter scheduling unit.

[0102] Reference Figure 6 , Figure 6 A hardware structure block diagram of a model parameter calling device provided in this application embodiment is shown below. Figure 6 The hardware structure of the model parameter calling device may include: at least one processor 1, at least one communication interface 2, at least one memory 3, and at least one communication bus 4;

[0103] In this embodiment of the application, the number of processor 1, communication interface 2, memory 3, and communication bus 4 is at least one, and processor 1, communication interface 2, and memory 3 communicate with each other through communication bus 4;

[0104] Processor 1 may be a central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement embodiments of the present invention.

[0105] Memory 3 may include high-speed RAM, and may also include non-volatile memory, such as at least one disk storage device;

[0106] The memory stores a program, which the processor can call. The program is used for:

[0107] When performing calculations for each parameter scheduling unit, the parameters of all parameter scheduling units of the model stored in the storage memory are loaded into the running memory as needed, so that the running memory stores the parameters of N parameter scheduling units; where N is an integer greater than or equal to 2, the storage space occupied by the parameters of the N parameter scheduling units is less than or equal to the running memory of the device, and the parameters of the N parameter scheduling units include the parameters of the target parameter scheduling unit being executed, and the parameters of the next parameter scheduling unit adjacent to the target parameter scheduling unit.

[0108] Optionally, the refined and extended functions of the program can be found in the description above.

[0109] This application embodiment also provides a readable storage medium that can store a program suitable for execution by a processor, the program being used for:

[0110] When performing calculations for each parameter scheduling unit, the parameters of all parameter scheduling units of the model stored in the storage memory are loaded into the running memory as needed, so that the running memory stores the parameters of N parameter scheduling units; where N is an integer greater than or equal to 2, the storage space occupied by the parameters of the N parameter scheduling units is less than or equal to the running memory of the device, and the parameters of the N parameter scheduling units include the parameters of the target parameter scheduling unit being executed, and the parameters of the next parameter scheduling unit adjacent to the target parameter scheduling unit.

[0111] Optionally, the refined and extended functions of the program can be found in the description above.

[0112] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0113] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.

[0114] The above description of the disclosed embodiments enables those skilled in the art to make or use this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for calling model parameters, characterized in that, The model includes multiple parameter scheduling units, each of which is obtained by dividing the model into sub-networks. The calculations of each parameter scheduling unit are executed serially. The method is applied to a device employing the model. The device includes storage memory and runtime memory. The storage memory stores the parameters of all parameter scheduling units of the model. The method includes: When performing calculations for each parameter scheduling unit, the parameters of all parameter scheduling units of the model stored in the storage memory are loaded into the running memory as needed, so that the running memory stores the parameters of N parameter scheduling units; where N is an integer greater than or equal to 2, the storage space occupied by the parameters of the N parameter scheduling units is less than or equal to the running memory of the device, and the parameters of the N parameter scheduling units include the parameters of the target parameter scheduling unit being executed, and the parameters of the next parameter scheduling unit adjacent to the target parameter scheduling unit.

2. The method according to claim 1, characterized in that, Before performing the calculations of each parameter scheduling unit, the method further includes: Determine the scheduling units for each parameter of the model; The model is initialized by storing the parameters of all parameter scheduling units of the model into the device's storage memory, and loading the parameters of the first N parameter scheduling units of the model into the device's running memory, where N is an integer greater than or equal to 2, and the storage space occupied by the parameters of the first N parameter scheduling units is less than or equal to the device's running memory.

3. The method according to claim 2, characterized in that, The parameter scheduling units of the determined model include: The multiple sub-networks of the model are treated as a single parameter scheduling unit.

4. The method according to claim 2, characterized in that, The parameter scheduling units of the determined model include: One sub-network of the model is used as a parameter scheduling unit.

5. The method according to any one of claims 1 to 4, characterized in that, The process of calculating the target parameter scheduling unit includes: Obtain the parameters of the target parameter scheduling unit from the device's running memory; The target parameter scheduling unit performs calculations based on the parameters of the target parameter scheduling unit, and updates the device's running memory when there are no parameters of the next parameter scheduling unit adjacent to the target parameter scheduling unit in the device's running memory.

6. The method according to claim 5, characterized in that, When the device's running memory stores parameters of two parameter scheduling units, and the device's running memory includes the parameters of the target parameter scheduling unit and the parameters of the previous parameter scheduling unit adjacent to the target parameter scheduling unit; Updating the device's running memory includes: The parameters of the next parameter scheduling unit adjacent to the target parameter scheduling unit are loaded from the device's storage memory into the device's running memory, overwriting the parameters of the previous parameter scheduling unit adjacent to the target parameter scheduling unit.

7. The method according to claim 5, characterized in that, When the device's running memory stores parameters of two or more parameter scheduling units, and the device's running memory includes the parameters of the target parameter scheduling unit, as well as the parameters of multiple parameter scheduling units that are calculated before the target parameter scheduling unit; Updating the device's running memory includes: The parameters of the next parameter scheduling unit adjacent to the target parameter scheduling unit are loaded from the device's storage memory into the device's running memory, overwriting the parameters of the first parameter scheduling unit that was calculated in at least one parameter scheduling unit that was calculated before the target parameter scheduling unit.

8. A model parameter calling device, characterized in that, The model includes multiple parameter scheduling units, each of which is obtained by dividing the model into sub-networks. The calculations of each parameter scheduling unit are executed serially. The device is applied to a device employing the model. The device includes storage memory and runtime memory. The storage memory stores the parameters of all parameter scheduling units of the model. The device includes: The computation execution module is used to load the parameters of all parameter scheduling units of the model stored in the storage memory into the running memory as needed when performing the computation of each parameter scheduling unit, so that the running memory stores the parameters of N parameter scheduling units; where N is an integer greater than or equal to 2, the storage space occupied by the parameters of the N parameter scheduling units is less than or equal to the running memory of the device, and the parameters of the N parameter scheduling units include the parameters of the target parameter scheduling unit being executed, and the parameters of the next parameter scheduling unit adjacent to the target parameter scheduling unit.

9. A model parameter retrieval device, characterized in that, Including memory and processor; The memory is used to store programs; The processor is configured to execute the program to implement each step of the model parameter invocation method as described in any one of claims 1 to 7.

10. A readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the various steps of the model parameter invocation method as described in any one of claims 1 to 7.

Images