Model parameter determination method, model storage method, model loading method and equipment

Abstract

The invention relates to the technical field of computing equipment, and particularly provides a model parameter determination method, a model storage method and equipment, and a model loading method and equipment. The model parameter determination method comprises the following steps: in response to a model parameter loading request sent by a terminal device, obtaining a locally stored address index relationship and model parameters of multiple target types; based on the address index relationship, obtaining one or more target types of model parameters from the multiple target types of model parameters; and sending the model parameters of the one or more target types to the terminal equipment. The model parameters of the target model are stored in the first computing device according to the target type classification, so that when the terminal device has a model parameter loading request, the first computing device can respond to the model parameter loading request to return locally stored model parameters of one or more target types to the terminal device; according to the method, the terminal equipment only loads part of model parameters of the target model or loads the model parameters as required flexibly.

Description

Technical Field

[0001] The present application relates to the technical field of computing devices, and in particular to a method for determining model parameters, and a method and device for saving and loading models. Background Art

[0002] In the field of deep learning, model saving and loading are crucial links. Deep learning models, especially large language models, have complex structures and a large number of parameters, which makes the model training, reasoning, and fine-tuning process very time-consuming and resource-intensive. Therefore, how to efficiently save and load trained models is of great significance to improving the efficiency and practicality of deep learning applications. Summary of the invention

[0003] In view of the above problems, the present application is proposed. The embodiments of the present application provide a method for determining model parameters, a method and a device for saving and loading a model.

[0004] According to a first aspect of an embodiment of the present application, a method for determining a model parameter is provided, which is applied to a first computing device, and the method includes:

[0005] In response to a model parameter loading request sent by a terminal device, a locally stored address index relationship and model parameters of multiple target types are obtained; the model parameter loading request is used to indicate the model parameters of one or more target types that need to be loaded; the address index relationship is generated based on the storage address of the model parameter of each target type, and the address index relationship is used to indicate the index address corresponding to the storage address of the model parameter of each target type;

[0006] Based on the address index relationship, obtain model parameters of one or more target types from model parameters of multiple target types;

[0007] Send one or more target type model parameters to the terminal device.

[0008] In an embodiment of the present application, when the terminal device has a model parameter loading request, the first computing device can respond to the model parameter loading request and return one or more model parameters of the target type stored locally to the terminal device based on the address index relationship, so that the terminal device can flexibly load only part of the model parameters of the target model or load on demand, thereby improving the loading efficiency of the model parameters.

[0009] According to the method for determining model parameters of the first aspect of the embodiment of the present application, the model parameter loading request carries a selected target application scenario, and the selected target application scenario is a scenario to which one or more target type model parameters will be applied;

[0010] Based on the address index relationship, obtain one or more target type model parameters from multiple target type model parameters, including:

[0011] Based on the selected target application scenario, determining a model parameter label for obtaining model parameters of one or more target types;

[0012] The address index relationship is queried using the model parameter tag to determine the index address corresponding to the storage address of the model parameter of one or more target types;

[0013] Get one or more target type model parameters from multiple target type model parameters based on the index address.

[0014] According to the method for determining model parameters of the first aspect of the embodiment of the present application, the model parameter loading request carries a model parameter tag of the selected target type;

[0015] Based on the address index relationship, obtain one or more target type model parameters from multiple target type model parameters, including:

[0016] The address index relationship is queried using the model parameter tag to determine the index address corresponding to the storage address of the model parameter of one or more target types;

[0017] Get one or more target type model parameters from multiple target type model parameters based on the index address.

[0018] According to the method for determining model parameters of the first aspect of the embodiment of the present application, the address index relationship is ground, and the address index relationship is queried using a model parameter tag to determine the index address corresponding to the storage address of the model parameter of one or more target types, including:

[0019] Determine a target address index table including a model parameter label from a first address index table and a second address index table included in the address index table; the first address index table is generated based on the storage addresses of the first type of parameters, the second type of parameters and the fourth type of parameters, and the second address index table is generated based on the storage address of the third type of parameters, the first type of parameters are parameters that will be iterated during the model training process, the second type of parameters are model training configuration parameters, the third type of parameters are parameters corresponding to specific functional network layers, and the fourth type of parameters are optimizer parameters that update the parameters of the model during the model training process;

[0020] Traversing the target address index table to determine the index entry including the model parameter tag;

[0021] The index address in the index entry is used as the index address corresponding to the storage address of the model parameter of one or more target types.

[0022] A second aspect of an embodiment of the present application provides a model saving method, which is applied to a second computing device, and the method includes:

[0023] Based on the target model, determining model parameters of multiple target types in the target model;

[0024] The model parameters of each target type are stored respectively, and an address index relationship is generated based on the storage address of the model parameters of each target type; wherein the address index relationship is used to indicate the index address corresponding to the storage address of the model parameters of each target type;

[0025] Save index relations.

[0026] According to the model saving method of the second aspect of the embodiment of the present application, the model parameters of multiple target types include first-category parameters and second-category parameters, the first-category parameters are parameters that will be iterated during the model training process, and the second-category parameters are model training configuration parameters.

[0027] According to the model saving method of the second aspect of the embodiment of the present application, the first type of parameters include weights and biases of the target model, and the second type of parameters include hyperparameters of the target model.

[0028] According to the model saving method of the second aspect of the embodiment of the present application, the model parameters of multiple target types include third-category parameters, which are parameters corresponding to specific functional network layers.

[0029] According to the model saving method of the second aspect of the embodiment of the present application, the model parameters of multiple target types include fourth-category parameters, which are optimizer parameters for updating the parameters of the model during the model training process.

[0030] According to the model saving method of the second aspect of the embodiment of the present application, when the model parameters of the multiple target types include first-category parameters and third-category parameters, the storage space corresponding to the storage address of the third-category parameters is part of the storage space corresponding to the storage address of the first-category parameters;

[0031] The first type of parameters are the parameters that will be iterated during the model training process, and the third type of parameters are the parameters corresponding to specific functional network layers.

[0032] According to the model saving method of the second aspect of the embodiment of the present application, when the model parameters of the multiple target types also include second-category parameters and fourth-category parameters, the address index relationship includes a first address index relationship generated based on the storage addresses of the first-category parameters, the second-category parameters, and the fourth-category parameters, and a second address index relationship generated based on the storage address of the third-category parameters;

[0033] The second type of parameters are model training configuration parameters, and the fourth type of parameters are optimizer parameters that update the model parameters during model training.

[0034] According to the model saving method of the second aspect of the embodiment of the present application, the index address in the first address index relationship includes a start address and an offset length;

[0035] The index address in the second address index relationship includes a tensor shape and an offset address. The tensor shape is used to characterize the size of the storage space occupied by the third type of parameters, and the offset address is used to characterize the offset size relative to the starting address of the first type of parameters.

[0036] A third aspect of the embodiments of the present application provides a model loading method, which is applied to a terminal device, and the method includes:

[0037] Sending a model parameter loading request to a first computing device, wherein the first computing device stores model parameters of multiple target types in the target model; the model parameter loading request is used to indicate model parameters of one or more target types that need to be loaded;

[0038] Receive one or more target type model parameters returned by the first computing device.

[0039] In the scheme of this embodiment, the model parameters of the target model are saved in the first computing device according to the target type classification. In this way, when the terminal device has a model parameter loading request, the first computing device can return one or more locally stored model parameters of the target type to the terminal device in response to the model parameter loading request, so that the terminal device can flexibly load only part of the model parameters of the target model or load on demand.

[0040] According to the model loading method of the third aspect of the embodiment of the present application, before sending the model parameter loading request to the first computing device, the method further includes:

[0041] Acquire multiple target application scenarios to be selected sent by the first computing device;

[0042] A model parameter loading request is generated based on a selected target application scenario, where the selected target application scenario is a scenario to which model parameters of one or more target types will be applied.

[0043] According to the model loading method of the third aspect of the embodiment of the present application, before the step of sending a model parameter loading request to the first computing device, the method further includes:

[0044] Obtain model parameter tags of multiple target types to be selected sent by the first computing device;

[0045] Generates a model parameter loading request based on the model parameter tags of the selected target type.

[0046] A fourth aspect of the embodiments of the present application provides a model loading device, which is applied to a terminal device, and the device includes:

[0047] A first sending module is used to send a model parameter loading request to a first computing device, wherein the first computing device stores model parameters of multiple target types in the target model; the model parameter loading request is used to indicate model parameters of one or more target types that need to be loaded;

[0048] The receiving module is used to receive one or more model parameters of the target type returned by the first computing device.

[0049] According to a fifth aspect of the embodiments of the present application, a device for determining a model parameter is provided, which is applied to a first computing device, and the device includes:

[0050] A first acquisition module, configured to acquire a locally stored address index relationship and model parameters of multiple target types in response to a model parameter loading request sent by a terminal device; the model parameter loading request is used to indicate model parameters of one or more target types that need to be loaded; the address index relationship is generated based on the storage address of the model parameter of each target type, and the address index relationship is used to indicate the index address corresponding to the storage address of the model parameter of each target type;

[0051] A second acquisition module, configured to acquire model parameters of one or more target types from model parameters of multiple target types based on the address index relationship;

[0052] The second sending module is used to send one or more target type model parameters to the terminal device.

[0053] According to a sixth aspect of the embodiments of the present application, a model storage device is provided, which is applied to a second computing device, and the device includes:

[0054] A determination module, used for determining model parameters of multiple target types in the target model based on the target model;

[0055] A processing module, used to store the model parameters of each target type respectively, and generate an address index relationship based on the storage address of the model parameters of each target type; wherein the address index relationship is used to indicate the index address corresponding to the storage address of the model parameters of each target type;

[0056] The saving module is used to save index relations.

[0057] A seventh aspect of the embodiments of the present application provides a data processing system, including:

[0058] A terminal device, a first computing device, and a second computing device;

[0059] The terminal device is used to execute the method of the first aspect;

[0060] The first computing device is used to execute the method of the second aspect;

[0061] The second computing device is used to execute the method of the third aspect.

[0062] An eighth aspect of the embodiments of the present application provides an electronic device, including a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of the method of the first aspect, the second aspect, or the third aspect.

[0063] According to a ninth aspect of the embodiments of the present application, a computer-readable storage medium is provided, on which a computer program / instruction is stored. When the computer program / instruction is executed by a processor, the steps of the method of the first aspect, the second aspect or the third aspect are implemented.

[0064] According to the tenth aspect of the embodiments of the present application, a computer program product is provided, comprising a computer program / instruction, which, when executed by a processor, implements the steps of the method of the first aspect, the second aspect or the third aspect.

[0065] It is to be understood that both the foregoing general description and the following detailed description are exemplary, and are intended to provide further explanation of the technology as claimed. BRIEF DESCRIPTION OF THE DRAWINGS

[0066] By describing the embodiments of the present application in more detail in conjunction with the accompanying drawings, the above and other purposes, features and advantages of the present application will become more apparent. The accompanying drawings are used to provide a further understanding of the embodiments of the present application and constitute a part of the specification. Together with the embodiments of the present application, they are used to explain the present application and do not constitute a limitation of the present application. In the accompanying drawings, the same reference numerals generally represent the same components or steps.

[0067] Figure 1 It is a schematic diagram illustrating the architecture of a communication system according to an embodiment of the present application.

[0068] Figure 2 is a flowchart illustrating the application of the model loading method according to an embodiment of the present application.

[0069] Figure 3 It is a schematic diagram illustrating the application of model loading according to an embodiment of the present application.

[0070] Figure 4 It is a schematic diagram illustrating a method for determining model parameters according to an embodiment of the present application.

[0071] Figure 5 It is a schematic diagram illustrating the application of the model saving method according to an embodiment of the present application.

[0072] Figure 6It is a schematic diagram illustrating application of a model index file according to an embodiment of the present application.

[0073] Figure 7 It is a schematic diagram illustrating the application of a tensor index file according to an embodiment of the present application.

[0074] Figure 8 It is a schematic diagram illustrating the application of a model file according to an embodiment of the present application.

[0075] Fig. 9 It is a schematic diagram illustrating the principle of a user loading a model through a terminal device in an inference scenario, a training scenario, and a transfer learning scenario according to an embodiment of the present application.

[0076] Fig.10 is a structural diagram illustrating the application of a model loading device according to an embodiment of the present application.

[0077] Fig.11 It is a structural diagram illustrating an apparatus for determining model parameters according to an embodiment of the present application.

[0078] Fig.12 It is a structural diagram illustrating the application of a model storage device according to an embodiment of the present application.

[0079] Fig.13 is a schematic diagram illustrating an electronic device according to an embodiment of the present application.

[0080] Fig.14 is a schematic diagram illustrating a computer-readable storage medium according to an embodiment of the present application. DETAILED DESCRIPTION

[0081] To make the purpose, technical scheme and advantages of the embodiments of the present application clearer, the technical scheme in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all of the embodiments. The components of the embodiments of the present application generally described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the application claimed for protection, but merely represents the selected embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without making creative work belong to the scope of protection of the present application.

[0082] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, further definition and explanation thereof is not required in subsequent drawings.

[0083] The term "and / or" herein only describes an association relationship, indicating that three relationships may exist. For example, A and / or B may represent the following three situations: A exists alone, A and B exist at the same time, and B exists alone. In addition, the term "at least one" herein represents any combination of at least two of any one or more of a plurality of. For example, including at least one of A, B, and C may represent including any one or more elements selected from the set consisting of A, B, and C.

[0084] In the related art, all parameters of the trained model are saved in one file, which results in the inability to flexibly load only part of the parameters or load on demand when using the model, which brings inconvenience to the use of the model.

[0085] The embodiments of the present application provide a method for determining model parameters, a method for saving and loading models, and a device. The model loading method includes sending a model parameter loading request to a first computing device, wherein the first computing device stores model parameters of multiple target types in the target model; the model parameter loading request is used to indicate the model parameters of one or more target types that need to be loaded; and receiving the model parameters of one or more target types returned by the first computing device. The model parameters of the target model are saved in the first computing device according to the target type classification, so that when the terminal device has a model parameter loading request, the first computing device can respond to the model parameter loading request and return the model parameters of one or more target types stored locally to the terminal device, so that the terminal device can flexibly load only part of the model parameters of the target model or load on demand.

[0086] See also Figure 1 , Figure 1 Schematic diagram of the architecture of a communication system disclosed in the embodiment of the present application. Figure 1 The communication system shown includes a terminal device 1 and a computing device 2 .

[0087] The terminal device 1 can transmit data with the computing device 2. An application for the user to load model parameters can be deployed on the terminal device 1. After the application is started, the model parameters of multiple target application scenarios or multiple target types to be selected can be displayed to the user through the human-computer interaction interface. The terminal device sends a model parameter loading request to the computing device 2.

[0088] After receiving the model parameters returned by the computing device 2, the terminal device can use the model parameters to run the model locally to complete tasks in scenarios such as reasoning scenarios, training scenarios, or transfer learning scenarios.

[0089] The computing device 2 is used to save the model parameters and return the model parameters to the terminal device in response to the model parameter loading request of the terminal device. In some embodiments, the computing device 2 may further include a first computing device and a second computing device, and the first computing device returns the model parameters to the terminal device in response to the model parameter loading request of the terminal device, and the second computing device saves the model parameters. Setting two computing devices to respectively save the model parameters and respond to the model parameter loading request can reduce the pressure on the computing device.

[0090] See also Figure 2 , Figure 2 The flowchart of the model loading method provided in the embodiment of the present application can be applied to a terminal device, such as Figure 2 As shown, the method may include the following steps:

[0091] Step 201: Send a model parameter loading request to a first computing device, wherein the first computing device stores model parameters of multiple target types in a target model; the model parameter loading request is used to indicate model parameters of one or more target types that need to be loaded.

[0092] In this embodiment, multiple target types can be configured in advance according to actual needs. For example, considering that the model parameters that need to be loaded are not exactly the same when the model is used in different application scenarios, it can be configured in advance based on the application scenarios in which the model may be applied. The application scenarios of the model include but are not limited to training scenarios, reasoning scenarios, transfer learning scenarios, model fine-tuning scenarios, model distillation scenarios, and / or model pruning scenarios.

[0093] Usually, when using a model in a training scenario, all parameters of the model need to be loaded. When using a model in an inference scenario, the first type of parameters of the model need to be loaded. When using a model in a transfer learning scenario, the third type of parameters of the model need to be loaded. The first type of parameters are the parameters that will be iterated during the model training process, and the third type of parameters are the parameters corresponding to a specific functional network layer. Therefore, in order to flexibly load only some parameters or load on demand, multiple target types can be pre-configured based on the application scenario of the model.

[0094] As an example, the target types pre-configured based on the application scenario of the model may include a hyperparameter type (epoch), a basic parameter type (model), an optimizer type (optimizer) and / or a specific functional network layer type, etc.

[0095] Among them, model parameters of basic parameter type are parameters that will be iterated during model training. In deep learning, such parameters usually refer to weights and biases in neural networks. Parameters of hyperparameter type are parameters that need to be manually set before the model runs, which are used to control the behavior and performance of the model. For example, parameters such as learning rate, number of iterations, regularization parameters, number of neurons in hidden layers, etc. are usually hyperparameter type parameters. Model parameters of optimizer type include optimizer parameters. In the process of deep learning backpropagation, the optimizer guides the various parameters of the loss function to update the appropriate size in the right direction, so that the updated parameters make the loss function value approach the global minimum.

[0096] In some optional embodiments, the model parameter loading request may indicate the model parameters of one or more target types that need to be loaded by carrying a model parameter tag of a selected target application scenario or a selected target type.

[0097] In the case where the model parameter loading request carries a selected target application scenario, before sending the model parameter loading request to the first computing device, multiple target application scenarios to be selected sent by the first computing device can be obtained, and a model parameter loading request can be generated based on the target application scenario selected from the multiple target application scenarios to be selected.

[0098] It should be understood that the terminal device can display multiple target application scenarios to be selected through the human-computer interaction interface, and then obtain the selected target application scenario based on the selection operation detected on the human-computer interaction interface. Among them, the multiple target application scenarios to be selected displayed on the human-computer interaction interface include but are not limited to training scenarios, reasoning scenarios, transfer learning scenarios, model fine-tuning scenarios, model distillation scenarios and / or model pruning scenarios.

[0099] In the case where the model parameter loading request carries a selected model parameter tag, before sending the model parameter loading request to the first computing device, a plurality of model parameter tags of target types to be selected sent by the first computing device can be obtained, and a model parameter loading request can be generated based on the model parameter tag of the target type selected from the plurality of model parameter tags of target types to be selected.

[0100] It should be understood that the terminal device can display multiple model parameter labels of target types to be selected through the human-computer interaction interface, and then obtain the model parameter label of the selected target type based on the selection operation detected on the human-computer interaction interface. Each model parameter label is used to uniquely indicate a target type.

[0101] In actual applications, an application for model loading may be deployed in the terminal device, and when detecting that the application is started, the first computing device may send multiple model parameter tags to be selected to the terminal device. Alternatively, the user may actively read the address index relationship through the terminal device, and obtain multiple model parameter tags of target types to be selected from the multiple address index relationships.

[0102] Step 202: Receive one or more target type model parameters returned by the first computing device.

[0103] In some embodiments, it is supported to load the model parameters in the first computing device to the terminal device based on multithreading. Figure 3 , Figure 3 Schematic diagram of model loading provided for implementation of this application.

[0104] like Figure 3 As shown, the terminal device loads one or more target type model parameters into the hard disk through I / O threads based on the user's operation, and then loads these model parameters into the memory through multi-threading, thereby speeding up the loading speed.

[0105] The following is a detailed introduction to a method for determining a model parameter disclosed in an embodiment of the present application. The execution subject of the method for determining a model parameter provided in an embodiment of the present application is generally a first computing device with a certain computing capability, and the first computing device includes, for example: a terminal device, a server, a hard disk of a computing device, or other processing device. In some possible implementations, the method for determining a model parameter can be implemented by a processor calling a computer-readable instruction stored in a memory.

[0106] See also Figure 4 FIG. 1 is a flowchart of a method for determining model parameters provided in an embodiment of the present application. The method may include the following steps:

[0107] Step 401: In response to a model parameter loading request sent by a terminal device, locally stored address index relationships and model parameters of multiple target types are obtained.

[0108] Among them, the model parameter loading request is used to indicate the model parameters of one or more target types that need to be loaded; the address index relationship can be in the form of a table or in the form of a function. In the embodiment of the present application, the table form is used as an example for explanation, and the address index table is used as an example of the address index relationship below. The address index table is generated based on the storage address of the model parameter of each target type, and the address index table is used to indicate the index address corresponding to the storage address of the model parameter of each target type.

[0109] In some optional embodiments, the index address in the address index table can be directly represented by the storage address of the model parameters of each target type. For example, after storing the model parameters of each target type, the starting address and ending address of storing each type of model parameters can be obtained, and the starting address and ending address can be used as the index address corresponding to the storage address of the model parameters of the target type in the address index table.

[0110] In other optional embodiments, considering that the size of the model parameters of each target type can be estimated based on experience, the size of the storage space occupied by the model parameters of each target type can be estimated based on the size of the model parameters of each target type. Therefore, after storing the model parameters of each target type, the starting address of each type of model parameters can be obtained, and then based on the starting address and the estimated size of the storage space occupied by the model parameters of this type, the index address corresponding to the storage address of the model parameters of each target type in the address index table is generated.

[0111] Step 402: Based on the address index relationship, obtain model parameters of one or more target types from model parameters of multiple target types.

[0112] In some optional embodiments, the model parameter loading request may indicate the model parameters of one or more target types to be loaded by carrying the model parameter tags of the selected target application scenario or the selected target type. The following describes step 402 based on these two situations.

[0113] In the case where the model parameter loading request carries the selected target application scenario, step 402 may include the following steps:

[0114] Based on the selected target application scenario, determining a model parameter label for obtaining model parameters of one or more target types;

[0115] The address index relationship is queried using the model parameter tag to determine the index address corresponding to the storage address of the model parameter of one or more target types;

[0116] Get one or more target type model parameters from multiple target type model parameters based on the index address.

[0117] Here, each model parameter tag is used to uniquely indicate a target type. The address index relationship includes multiple index entries, each of which includes an index item and an index value stored in a dictionary form, where the index item is a model parameter tag, and the index value is the index address corresponding to the storage address of the model parameter represented by the model parameter tag.

[0118] When the model parameter tag is used to query the address index relationship, an index entry including the model parameter tag in the address index relationship can be obtained, and the index address in the index entry is the index address corresponding to the storage address of the model parameter of one or more target types.

[0119] In the case where the model parameter loading request carries the model parameter tag of the selected target type, step 402 may include the following steps:

[0120] The address index relationship is queried using the model parameter tag to determine the index address corresponding to the storage address of the model parameter of one or more target types;

[0121] Get one or more target type model parameters from multiple target type model parameters based on the index address.

[0122] In some optional embodiments, in order to facilitate calling and fast loading, the address index table that implements the address index relationship is divided into a first address index table and a second address index table, so that when querying the address index table, you can first select the target address index table to be queried from the first address index table and the second address index table, and then traverse the target address index table to obtain the index address corresponding to the storage address of the model parameters of one or more target types.

[0123] Among them, the first address index table is generated based on the storage addresses of the first type of parameters, the second type of parameters and the fourth type of parameters, and the second address index table is generated based on the storage address of the third type of parameters. The first type of parameters are parameters that will be iterated during the model training process, the second type of parameters are model training configuration parameters, the third type of parameters are parameters corresponding to specific functional network layers, and the fourth type of parameters are optimizer parameters that update the parameters of the model during the model training process.

[0124] Step 403: Send one or more target type model parameters to the terminal device.

[0125] The following is a detailed introduction to a model saving method disclosed in an embodiment of the present application. The execution subject of the model saving method provided in the embodiment of the present application is generally a second computing device with certain computing capabilities, and the second computing device includes, for example, a terminal device, a server or other processing device. In some possible implementations, the model saving method can be implemented by a processor calling a computer-readable instruction stored in a memory.

[0126] In some embodiments, the second computing device and the first computing device may be the same computing device. In this case, the model parameters are saved and the model parameters are determined during the model loading process on the same computing device.

[0127] In other embodiments, the second computing device and the first computing device may also be different computing devices. In this case, after saving the model parameters of multiple target types and generating address index relationships on the second computing device, the model parameters of the multiple target types and address index relationships saved on the second computing device are migrated to the first computing device.

[0128] In other embodiments, the first computing device and the second computing device may also be different storage devices on the same computing device, such as the first computing device is a hard disk and the second computing device is a memory. In this case, the model parameters of the target model are saved in the memory, and the model parameters saved in the memory are migrated to the hard disk for further storage.

[0129] See also Figure 5 As shown, it is a flowchart of the model saving method provided in an embodiment of the present application, and the method may include the following steps:

[0130] Step 501: Based on the target model, determine model parameters of multiple target types in the target model.

[0131] In practical applications, after the model is trained, the model parameters will change. Here, the model parameters refer to all parameters involved in the model training process, including but not limited to basic parameters, hyperparameters, optimizer parameters, etc. Therefore, the target model in this step refers to one or more models that have at least been trained.

[0132] In this embodiment, multiple target types can be pre-configured according to actual needs. For example, considering that when the model is used in different application scenarios, the model parameters that need to be loaded are not exactly the same, so they can be pre-configured based on the application scenarios in which the model may be applied.

[0133] In some optional embodiments, models of multiple target types include but are not limited to first-category parameters, second-category parameters, third-category parameters and / or fourth-category parameters, wherein the first-category parameters are parameters that will be iterated during the model training process, the second-category parameters are model training configuration parameters, the third-category parameters are parameters corresponding to specific functional network layers, and the fourth-category parameters are optimizer parameters that update the parameters of the model during the model training process.

[0134] As an example, the first category of parameters may include the weights and biases of the target model, the second category of parameters may include hyperparameters of the target model (such as the number of iterations, regularization parameters, etc.), and the third category of parameters may include parameters of network layers such as convolution layer 1...convolution layer n, pooling layer, normalization layer, etc.

[0135] Step 502: Store the model parameters of each target type respectively, and generate an address index relationship based on the storage address of the model parameters of each target type; wherein the address index relationship is used to indicate the index address corresponding to the storage address of the model parameters of each target type.

[0136] In some embodiments, for the convenience of management and calling, models of different target types may be stored in completely different storage spaces.

[0137] In other embodiments, when the model parameters of multiple target types include first-category parameters and third-category parameters, since the parameters corresponding to the specific functional network layer in the third-category parameters are the parameters that will be iterated in the specific functional network layer during the model training process, in simple terms, the parameters corresponding to the specific functional network layer can be understood as the weights and biases of the specific functional network layer. Therefore, in this case, in order to save storage space, the storage space corresponding to the storage address of the third-category parameters can be set to part of the storage space corresponding to the storage address of the first-category parameters.

[0138] As an example, if the first category parameters are stored in storage space 1, since the third category parameters (such as model parameters corresponding to convolution layer 1...convolution layer n and the normalization layer) themselves belong to the model parameters corresponding to the first category parameters, the model parameters corresponding to convolution layer 1...convolution layer n and the normalization layer are no longer stored separately, but the index address corresponding to the storage address of the first category parameters and the index address corresponding to the storage address of the third category parameters still need to be recorded in the address index relationship.

[0139] In some optional embodiments, in order to facilitate calling and fast loading based on different application scenarios, when the address index relationship is an address index table, two address index tables can be generated based on the first category parameters, the second category parameters, the third category parameters and the fourth category parameters, namely, a first address index table and a second address index table, wherein the first address index table is generated based on the storage addresses of the first category parameters, the second category parameters and the fourth category parameters, and the second address index table is generated based on the storage address of the third category parameters.

[0140] In some optional embodiments, when the storage space corresponding to the storage address of the third type of parameter is part of the storage space corresponding to the storage address of the first type of parameter, and the address index table includes a first address index table and a second address index table, the index address in the first address index table includes a start address and an offset length;

[0141] The index address in the second address index table includes a tensor shape and an offset address. The tensor shape is used to characterize the size of the storage space occupied by the third type of parameters, and the offset address is used to characterize the offset size relative to the starting address of the first type of parameters.

[0142] It should be understood that the sum of the starting address of the first type of parameter and the offset address of the third type of parameter is the starting address of the third type of parameter. The sum of the starting address of the third type of parameter and the length of the storage space calculated based on the tensor shape is the end address of the third type of parameter.

[0143] Regarding the tensor shape, taking the convolution layer as an example, the convolution layer can be calculated by inputting the number of channels c in , the number of output channels c out , convolution kernel size (convolution kernel height) H and (convolution kernel width W), calculate the size of the convolution layer parameters = c in *c out *H*W+c out , so the tensor shape included in the address information in the mapping relationship between the convolution layer type and the address information can be calculated based on the number of input channels c included in the convolution layer. in , the number of output channels c out , and the convolution kernel sizes H and W. Another example is the normalization layer, which usually only refers to the number of output channels c out Calculate the size of the normalization layer parameters. If the number of output channels is c out is c, then the size of the normalization layer parameter = 2c.

[0144] Step 503: Save the index table.

[0145] The model saving method of the present application is introduced below with a specific embodiment.

[0146] Saving the model parameters of the target model requires generating three files, namely the model index file, the model file, and the tensor index file.

[0147] Please refer to Figure 6 , Figure 6 This is a schematic diagram of the generated model index file. Figure 6 As shown, the model index file includes multiple index entries, each index entry includes an index item and an index value, the index item is the model parameter label of each target type, and the index value is the index address corresponding to the storage address of the model parameter of each target type. Figure 6 In , model parameter labels of various target types include hyperparameter labels, basic parameter labels, and optimizer labels, etc. The index address is represented by the starting address and offset length.

[0148] It should be understood that the hyperparameters corresponding to the hyperparameter labels belong to the second category of parameters, the basic parameters corresponding to the basic parameter labels belong to the first category of parameters, and the parameters corresponding to the optimizer labels belong to the fourth category of parameters.

[0149] Please refer to Figure 7 , Figure 7 This is a schematic diagram of the generated tensor index file. Figure 7As shown, the tensor index file also includes multiple index entries, each of which includes an index item and an index value. The index item is the model parameter label of each target type, and the index value is the index address corresponding to the storage address of the model parameter of each target type. Figure 7 In , model parameter labels of various target types include convolution layer 1 label...convolution layer n label, normalization layer label, etc., and the index address is represented by the offset address and tensor shape.

[0150] Please refer to Figure 8 , Figure 8 This is a schematic diagram of the generated model file. Figure 8 As shown in the figure, the model parameters are saved in blocks according to hyperparameters, basic parameters (such as weights and biases), optimizer parameters, etc.

[0151] At the same time, the basic parameters include the basic parameters in the convolution layers 1-n and the basic parameters in the normalization layer. Therefore, when storing the basic parameters, the storage space of the basic parameters is further divided based on the network layer, that is, the basic parameters corresponding to the convolution layer n are stored in the same storage space, and the basic parameters of the model corresponding to the normalization layer are stored in the same storage space. The storage space corresponding to the convolution layer n and the storage space corresponding to the normalization layer together constitute the storage space of the basic parameters.

[0152] The following takes the training scenario, inference scenario, and transfer learning scenario as examples to describe the process of users loading model parameters through terminal devices in these scenarios.

[0153] Please refer to Fig. 9 , Fig. 9 The present application provides an embodiment in which user B loads a model through a terminal device in an inference scenario, user C loads a model through a terminal device in a training scenario, and user D loads a model through a terminal device in a transfer learning scenario.

[0154] User A trained a model and generated the following files through a computing device: model file, model index file, and model tensor file. These files will be used for other users' reasoning, training, and transfer learning.

[0155] User B uses the model in the inference scenario through the terminal device. The specific process is as follows:

[0156] User B actively reads the model index file through the inference API of the terminal device, and the terminal device displays model parameter labels of multiple target types to be selected based on the model index file.

[0157] The terminal device generates a model parameter loading request based on the model parameter tag of the target type selected by user B, and sends it to the computing device;

[0158] The terminal device receives the model parameters returned by the computing device, and uses these model parameters to complete the reasoning task using the model trained by user A.

[0159] User C uses the model in the training scenario through the terminal device. The specific process is as follows:

[0160] User C actively reads the model index file through the training API of the terminal device, and the terminal device displays model parameter labels of multiple target types to be selected based on the model index file.

[0161] The terminal device generates a model parameter loading request based on the model parameter tag of the target type selected by user C, and sends it to the computing device;

[0162] The terminal device receives the model parameters returned by the computing device, and uses these model parameters to complete the training task using the model trained by user A.

[0163] User D uses the model in the training scenario through the terminal device. The specific process is as follows:

[0164] User D actively reads the model index file and the tensor index file through the training API of the terminal device, and the terminal device displays model parameter labels of multiple target types to be selected based on the model index file and the tensor index file.

[0165] The terminal device generates a model parameter loading request based on the model parameter tag of the target type selected by user D, and sends it to the computing device;

[0166] The terminal device receives the model parameters returned by the computing device, and uses these model parameters to complete the transfer learning task using the model trained by user A.

[0167] The present application also provides a model loading device, which is applied to a terminal device, such as Fig.10 As shown, the device comprises:

[0168] The first sending module 11 is used to send a model parameter loading request to the first computing device, wherein the first computing device stores model parameters of multiple target types in the target model; the model parameter loading request is used to indicate the model parameters of one or more target types that need to be loaded;

[0169] The receiving module 12 is used to receive one or more model parameters of the target type returned by the first computing device.

[0170] In some optional embodiments, the device is also used for:

[0171] Before sending the model parameter loading request to the first computing device, obtaining a plurality of target application scenarios to be selected sent by the first computing device;

[0172] A model parameter loading request is generated based on a selected target application scenario, where the selected target application scenario is a scenario to which model parameters of one or more target types will be applied.

[0173] In some optional embodiments, the device is also used for:

[0174] Before sending the model parameter loading request to the first computing device, obtaining model parameter tags of multiple target types to be selected sent by the first computing device;

[0175] Generates a model parameter loading request based on the model parameter tags of the selected target type.

[0176] The present application also provides a device for determining model parameters, which is applied to a first computing device, such as Fig.11 As shown, the device comprises:

[0177] The first acquisition module 21 is used to obtain the locally stored address index relationship and the model parameters of multiple target types in response to the model parameter loading request sent by the terminal device; the model parameter loading request is used to indicate the model parameters of one or more target types that need to be loaded; the address index relationship is generated based on the storage address of the model parameter of each target type, and the address index relationship is used to indicate the index address corresponding to the storage address of the model parameter of each target type;

[0178] A second acquisition module 22, configured to acquire model parameters of one or more target types from model parameters of multiple target types based on the address index relationship;

[0179] The second sending module 23 is used to send one or more target type model parameters to the terminal device.

[0180] In some optional embodiments, the model parameter loading request carries a selected target application scenario, where the selected target application scenario is a scenario to which one or more target type model parameters will be applied;

[0181] The second acquisition module 22 is used for:

[0182] Based on the selected target application scenario, determining a model parameter label for obtaining model parameters of one or more target types;

[0183] The address index relationship is queried using the model parameter tag to determine the index address corresponding to the storage address of the model parameter of one or more target types;

[0184] Get one or more target type model parameters from multiple target type model parameters based on the index address.

[0185] In some optional embodiments, the model parameter loading request carries a model parameter tag of the selected target type;

[0186] The second acquisition module 22 is used for:

[0187] The address index relationship is queried using the model parameter tag to determine the index address corresponding to the storage address of the model parameter of one or more target types;

[0188] Get one or more target type model parameters from multiple target type model parameters based on the index address.

[0189] In some optional embodiments, the address index relationship is an address index table, and the second acquisition module 22 is used to:

[0190] Determine a target address index table including a model parameter label from a first address index table and a second address index table included in the address index table; the first address index table is generated based on the storage addresses of the first type of parameters, the second type of parameters and the fourth type of parameters, and the second address index table is generated based on the storage address of the third type of parameters, the first type of parameters are parameters that will be iterated during the model training process, the second type of parameters are model training configuration parameters, the third type of parameters are parameters corresponding to specific functional network layers, and the fourth type of parameters are optimizer parameters that update the parameters of the model during the model training process;

[0191] Traversing the target address index table to determine the index entry including the model parameter tag;

[0192] The index address in the index entry is used as the index address corresponding to the storage address of the model parameter of one or more target types.

[0193] The present application also provides a model storage device, which is applied to a second computing device, such as Fig.12 As shown, the device comprises:

[0194] A determination module 31 is used to determine model parameters of multiple target types in the target model based on the target model;

[0195] The processing module 32 is used to store the model parameters of each target type respectively, and generate an address index relationship based on the storage address of the model parameters of each target type; wherein the address index relationship is used to indicate the index address corresponding to the storage address of the model parameters of each target type;

[0196] The saving module 33 is used to save the index table.

[0197] In some optional embodiments, the model parameters of multiple target types include first-category parameters and second-category parameters, the first-category parameters are parameters that will be iterated during the model training process, and the second-category parameters are model training configuration parameters.

[0198] In some optional embodiments, the first type of parameters includes weights and biases of the target model, and the second type of parameters includes hyperparameters of the target model.

[0199] In some optional embodiments, the model parameters of multiple target types include third-type parameters, which are parameters corresponding to specific functional network layers.

[0200] In some optional embodiments, the model parameters of multiple target types include fourth type parameters, which are optimizer parameters for updating the parameters of the model during the model training process.

[0201] In some optional embodiments, when the model parameters of the multiple target types include first-category parameters and third-category parameters, the storage space corresponding to the storage address of the third-category parameters is part of the storage space corresponding to the storage address of the first-category parameters;

[0202] The first type of parameters are the parameters that will be iterated during the model training process, and the third type of parameters are the parameters corresponding to specific functional network layers.

[0203] In some optional embodiments, the address index relationship is an address index table, and in the case where the model parameters of the multiple target types further include second-category parameters and fourth-category parameters, the address index table includes a first address index table generated based on the storage addresses of the first-category parameters, the second-category parameters, and the fourth-category parameters, and a second address index table generated based on the storage addresses of the third-category parameters;

[0204] The second type of parameters are model training configuration parameters, and the fourth type of parameters are optimizer parameters that update the model parameters during model training.

[0205] In some optional embodiments, the index address in the first address index table includes a start address and an offset length;

[0206] The index address in the second address index table includes a tensor shape and an offset address. The tensor shape is used to characterize the size of the storage space occupied by the third type of parameters, and the offset address is used to characterize the offset size relative to the starting address of the first type of parameters.

[0207] The present application also provides an electronic device to execute the above-mentioned model saving method, model loading method or model parameter determination method. Fig.13 It shows a schematic diagram of an electronic device provided by some embodiments of the present application. Fig.13As shown, the electronic device 8 includes: a processor 800, a memory 801, a bus 802 and a communication interface 803, and the processor 800, the communication interface 803 and the memory 801 are connected via the bus 802; the memory 801 stores a computer program that can be run on the processor 800, and when the processor 800 runs the computer program, it executes the model saving method, model loading method or model parameter determination method provided in any of the aforementioned embodiments of the present application.

[0208] The memory 801 may include a high-speed random access memory (RAM), and may also include a non-volatile memory, such as at least one disk storage. The communication connection between the device network element and at least one other network element is realized through at least one communication interface 803 (which may be wired or wireless), and the Internet, wide area network, local area network, metropolitan area network, etc. may be used.

[0209] The bus 802 may be an ISA bus, a PCI bus, or an EISA bus, etc. The bus may be divided into an address bus, a data bus, a control bus, etc. The memory 801 is used to store programs, and the processor 800 executes the programs after receiving the execution instruction. The model saving method, the model loading method, or the model parameter determination method disclosed in any implementation of the aforementioned embodiment of the present application may be applied to the processor 800, or implemented by the processor 800.

[0210] The processor 800 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the hardware integrated logic circuit or software instructions in the processor 800. The above processor 800 can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a readily available programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The methods, steps and logic block diagrams disclosed in the embodiments of the present application can be implemented or executed. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. The steps of the method disclosed in the embodiments of the present application can be directly embodied as a hardware decoding processor to be executed, or the hardware and software modules in the decoding processor can be executed. The software module can be located in a mature storage medium in the field such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory 801, and the processor 800 reads the information in the memory 801 and completes the steps of the above method in combination with its hardware.

[0211] The present application also provides a computer-readable storage medium corresponding to the model saving method, model loading method or model parameter determination method provided in the above-mentioned embodiment. Fig.14 The computer-readable storage medium shown is a CD 30 on which a computer program (i.e., a program product) is stored. When the computer program is run by the processor, it will execute the model saving method, model loading method, or model parameter determination method provided in any of the aforementioned embodiments.

[0212] It should be noted that examples of computer-readable storage media may also include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other optical or magnetic storage media, which are not listed here one by one.

[0213] It should be noted that:

[0214] In the description provided herein, a large number of specific details are described. However, it is understood that the embodiments of the present application can be practiced without these specific details. In some instances, well-known structures and technologies are not shown in detail so as not to obscure the understanding of this description.

[0215] Similarly, it should be understood that in order to streamline the present application and help understand one or more of the various inventive aspects, in the above description of the exemplary embodiments of the present application, the various features of the present application are sometimes grouped together into a single embodiment, figure, or description thereof. However, the disclosed method should not be interpreted as reflecting the following schematic diagram: the claimed application requires more features than the features clearly stated in each claim. More specifically, as reflected in the claims below, the inventive aspects are less than all the features of the single embodiment disclosed above. Therefore, the claims following the specific embodiment are hereby expressly incorporated into the specific embodiment, wherein each claim itself serves as a separate embodiment of the present application.

[0216] In addition, those skilled in the art will appreciate that, although some embodiments herein include certain features included in other embodiments but not other features, the combination of features of different embodiments is meant to be within the scope of the present application and form different embodiments. For example, in the claims below, any one of the claimed embodiments may be used in any combination.

[0217] The above are only preferred specific implementations of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed in the present application should be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be based on the protection scope of the claims.

Claims

1. A method for determining model parameters, characterized in that: Applied to a first computing device, the method includes: In response to a model parameter loading request sent by a terminal device, a locally stored address index relationship and model parameters of multiple target types are obtained; the model parameter loading request is used to indicate the model parameters of one or more target types that need to be loaded; the address index relationship is used to indicate the index address corresponding to the storage address of the model parameters of each target type; Based on the address index relationship, obtain model parameters of one or more target types from model parameters of multiple target types; Send one or more target type model parameters to the terminal device.

2. The method according to claim 1, characterized in that The model parameter loading request carries a selected target application scenario, where the selected target application scenario is a scenario to which one or more target type model parameters will be applied; Based on the address index relationship, obtain one or more target type model parameters from multiple target type model parameters, including: Based on the selected target application scenario, determining a model parameter label for obtaining model parameters of one or more target types; The address index relationship is queried using the model parameter tag to determine the index address corresponding to the storage address of the model parameter of one or more target types; Get one or more target type model parameters from multiple target type model parameters based on the index address.

3. The method according to claim 1, characterized in that The model parameter loading request carries the model parameter tag of the selected target type; Based on the address index relationship, obtain one or more target type model parameters from multiple target type model parameters, including: The address index relationship is queried using the model parameter tag to determine the index address corresponding to the storage address of the model parameter of one or more target types; Get one or more target type model parameters from multiple target type model parameters based on the index address.

4. The method according to claim 1 or 2, characterized in that The address index relationship is an address index table. The address index relationship is queried using a model parameter tag to determine the index address corresponding to the storage address of the model parameter of one or more target types, including: Determine a target address index table including a model parameter label from a first address index table and a second address index table included in the address index table; the first address index table is generated based on the storage addresses of the first type of parameters, the second type of parameters and the fourth type of parameters, and the second address index table is generated based on the storage address of the third type of parameters, the first type of parameters are parameters that will be iterated during the model training process, the second type of parameters are model training configuration parameters, the third type of parameters are parameters corresponding to specific functional network layers, and the fourth type of parameters are optimizer parameters that update the parameters of the model during the model training process; Traversing the target address index table to determine the index entry including the model parameter tag; The index address in the index entry is used as the index address corresponding to the storage address of the model parameter of one or more target types.

5. A model saving method, characterized in that: Applied to a second computing device, the method includes: Based on the target model, determining model parameters of multiple target types in the target model; The model parameters of each target type are stored respectively, and an address index relationship is generated based on the storage address of the model parameters of each target type; wherein the address index relationship is used to indicate the index address corresponding to the storage address of the model parameters of each target type; Save index relations.

6. The method according to claim 5, characterized in that Model parameters of various target types include first-category parameters and second-category parameters. The first-category parameters are parameters that will be iterated during the model training process, and the second-category parameters are model training configuration parameters.

7. A model loading method, characterized in that: Applied to terminal equipment, the method includes: Sending a model parameter loading request to a first computing device, wherein the first computing device stores model parameters of multiple target types in the target model; the model parameter loading request is used to indicate model parameters of one or more target types that need to be loaded; Receive one or more target type model parameters returned by the first computing device.

8. The method according to claim 7, characterized in that Before sending the model parameter loading request to the first computing device, the method further includes: Acquire multiple target application scenarios to be selected sent by the first computing device; A model parameter loading request is generated based on a selected target application scenario, where the selected target application scenario is a scenario to which model parameters of one or more target types will be applied.

9. The method according to claim 7, characterized in that Before the step of sending a model parameter loading request to the first computing device, the method further includes: Obtain model parameter tags of multiple target types to be selected sent by the first computing device; Generates a model parameter loading request based on the model parameter tags of the selected target type.

10. An electronic device comprising a memory, a processor and a computer program stored in the memory, characterized in that: The processor executes the computer program to implement the steps of any one of claims 1-4, any one of claims 5-6, or any one of claims 7-9.

Images