Model parameter adjustment method, device, electronic device and storage medium

Abstract

The present application provides a model parameter adjustment method, device, electronic device and storage medium, which includes determining a first parameter category of the model according to the task scenario of the model to be trained; determining a first target dynamic parameter corresponding to the first parameter category from a preset parameter list based on the first parameter category; initializing the model based on the first target dynamic parameter, and recording the storage location information of the initialized first target dynamic parameter; in response to determining that the task scenario has changed, determining a second parameter type of the model based on the changed task scenario; determining a second target dynamic parameter corresponding to the second parameter type from a preset parameter list based on the second parameter type, and replacing the first target dynamic parameter with the second target dynamic parameter based on the storage location information, so that the model can support multiple downstream task scenarios by adjusting the dynamic parameters, thereby improving the efficiency of model adjustment.

Description

Technical Field

[0001] The present application relates to the field of model parameter adjustment technology, and in particular to a model parameter adjustment method, device, electronic device and storage medium. Background Art

[0002] This section is intended to provide a background or context to the embodiments of the application that are recited in the claims. No statement herein is admitted to be prior art by virtue of its inclusion in this section.

[0003] In recent years, with the rapid development of artificial intelligence (AI), particularly deep learning, deep learning has been able to autonomously learn data identifiers from large amounts of training data, eliminating tedious feature engineering tasks, significantly improving application performance, and enabling transfer learning capabilities. However, the increasing size of models also presents challenges for downstream deployment. After training, ultra-large-scale pre-trained models need to be applied to downstream tasks in diverse domains and types, requiring fine-tuning of the pre-trained model parameters. However, as the size of pre-trained model parameters increases, directly fine-tuning large pre-trained models faces a series of challenges, including high computational resource overhead. Summary of the Invention

[0004] In view of this, the purpose of this application is to provide a model parameter adjustment method, device, electronic device and storage medium.

[0005] Based on the above objectives, this application provides a model parameter adjustment method, including:

[0006] Determining a first parameter category to be configured for the model based on a task scenario of the model to be trained;

[0007] Determining a first target dynamic parameter corresponding to the first parameter category from a preset parameter list based on the first parameter category;

[0008] Initializing the model based on the first target dynamic parameter, and recording storage location information of the initialized first target dynamic parameter;

[0009] In response to determining that the task scenario has changed, determining a second parameter type of the model to be adjusted based on the changed task scenario;

[0010] A second target dynamic parameter corresponding to the second parameter type is determined from a preset parameter list based on the second parameter type, and the first target dynamic parameter is replaced with the second target dynamic parameter based on the storage location information.

[0011] In some embodiments, after initializing the model based on the first target dynamic parameter, the method further includes:

[0012] Obtaining multiple static parameters of the initialized model;

[0013] A target static parameter of the model is determined from the plurality of static parameters based on the first parameter category, so that the target static parameter participates in the training process of the model.

[0014] In some embodiments, the preset parameter list is set in a memory; before initializing the model based on the first target dynamic parameter, the method further includes:

[0015] The first target dynamic parameter is transferred from the preset parameter list to the target video memory.

[0016] In some embodiments, the first target dynamic parameters include a plurality of parameter matrices; and transferring the first target dynamic parameters from the preset parameter list to the target video memory specifically includes:

[0017] For each parameter matrix in the plurality of parameter matrices, obtaining a first size of the parameter matrix in the first dimension and a second size of the parameter matrix in the second dimension;

[0018] splicing the first parameter matrices of the same size along a second dimension into a merged parameter matrix, and transferring the merged parameter matrix from the preset parameter list to a target video memory;

[0019] The merged parameter matrix transmitted to the target video memory is split into a plurality of parameter matrices having the same first size based on the second size.

[0020] In some embodiments, before splicing the first parameter matrices of the same size into a merged parameter matrix along the second dimension, transferring the first target dynamic parameter from the preset parameter list to a target video memory specifically further includes:

[0021] It is determined that a first number of parameter matrices having the same first size is greater than a second number of parameter matrices having the same second size.

[0022] In some embodiments, the preset parameter list includes a first preset list and a second preset list, the first preset list is set in a video memory, and the second preset list is set in a memory; after determining a first target dynamic parameter corresponding to the first parameter category from the preset parameter list based on the first parameter category, the method further includes:

[0023] Determine a third target dynamic parameter in the first preset list that has participated in the initialization the least number of times;

[0024] In response to determining that the first target dynamic parameter is in the second preset list, the third target dynamic parameter is replaced with the storage location of the first target dynamic parameter.

[0025] In some embodiments, after determining a second target dynamic parameter corresponding to the second parameter type from a preset parameter list based on the second parameter type, the method further includes:

[0026] In response to determining that the second target dynamic parameter is in the second preset list, the third target dynamic parameter is replaced with the storage location of the first target dynamic parameter.

[0027] Based on the same inventive concept, the exemplary embodiment of the present application further provides a model parameter adjustment device, comprising:

[0028] A first determination module determines a first parameter category of the model based on a task scenario of the model to be trained;

[0029] a second determining module, determining a first target dynamic parameter corresponding to the first parameter category from a preset parameter list based on the first parameter category;

[0030] an initialization module, configured to initialize the model based on the first target dynamic parameter and record storage location information of the initialized first target dynamic parameter;

[0031] a third determining module, in response to determining that the task scenario has changed, determining a second parameter type of the model based on the changed task scenario;

[0032] A parameter replacement module determines a second target dynamic parameter corresponding to the second parameter type from a preset parameter list based on the second parameter type, and replaces the first target dynamic parameter with the second target dynamic parameter based on the storage location information.

[0033] Based on the same inventive concept, an exemplary embodiment of the present application also provides an electronic device, comprising a memory, a processor, and a computer program stored on the memory and executable by the processor, wherein the processor implements the model parameter adjustment method as described above when executing the program.

[0034] Based on the same inventive concept, an exemplary embodiment of the present application further provides a non-transitory computer-readable storage medium, which stores computer instructions for causing a computer to execute the model parameter adjustment method as described above.

[0035] From the above, it can be seen that the model parameter adjustment method, device, electronic device and storage medium provided in the present application determine the first parameter category to be configured of the model according to the task scenario of the model to be trained; determine the first target dynamic parameter corresponding to the first parameter category from the preset parameter list based on the first parameter category; initialize the model based on the first target dynamic parameter, and record the storage location information of the initialized first target dynamic parameter; in response to determining that the task scenario has changed, determine the second parameter type to be adjusted of the model based on the changed task scenario; determine the second target dynamic parameter corresponding to the second parameter type from the preset parameter list based on the second parameter type, and replace the first target dynamic parameter with the second target dynamic parameter based on the storage location information, so that the model can support multiple downstream task scenarios by adjusting the dynamic parameters, thereby improving the efficiency of model adjustment. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] In order to more clearly illustrate the technical solutions in this application or related technologies, the following briefly introduces the drawings required for use in the embodiments or related technical descriptions. Obviously, the drawings described below are merely embodiments of this application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0037] Figure 1 A flow chart of a model parameter adjustment method according to an embodiment of the present application;

[0038] Figure 2 A schematic diagram of a splicing parameter matrix according to an embodiment of the present application;

[0039] Figure 3 This is a structural diagram of a model parameter adjustment device according to an embodiment of the present application;

[0040] Figure 4 This is a schematic structural diagram of a specific electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0041] The principles and spirit of the present application will be described below with reference to several exemplary embodiments. It should be understood that these embodiments are provided solely to enable those skilled in the art to better understand and implement the present application, and are not intended to limit the scope of the present application in any way. Rather, these embodiments are provided to make the present application more thorough and complete, and to fully convey the scope of the present application to those skilled in the art.

[0042] According to the embodiments of the present application, a model parameter adjustment method, device, electronic device and storage medium are proposed.

[0043] It should be understood herein that any number of elements in the drawings is for illustration only and not for limitation, and any naming is only for distinction and does not have any limiting meaning.

[0044] The principles and spirit of the present application are explained in detail below with reference to several representative implementations of the present application.

[0045] In the existing technology, the Pytorch framework or the Tensorflow deep learning framework is mainly used as the basis, and the preset Transformer model in the source code is directly modified. The locked parameter gradient method is used to fine-tune (a method of retraining the model using data not used in pre-training). After obtaining the fine-tuned model, the framework is directly used for inference. Therefore, generally speaking, a basic pre-trained large model inference can only adapt to one fine-tuning technology inference calculation, that is, it can only support one downstream subtask at the same time. Usually, the inference service of deep learning first needs to define a model structure; then initialize the model with the trained model parameters, allocate video memory or memory to the model, and form a calculation graph; when it is ready, the calculation and inference can be started. Among them, the model structure defined in the first step and the initialization of the model are immutable in the subsequent calculation process. The parameters used by the model are loaded into the video memory or memory during the initialization process for subsequent calculations. When downstream subtasks change, the model needs to be reinitialized based on the trained model parameters. This means that all the parameters loaded into the video memory or main memory after the model is initialized need to be replaced. However, this number is usually very large, which consumes a lot of computing power resources and also causes the same parameters to be unable to be reused, resulting in a waste of computing power resources.

[0046] In order to solve the above problems, this application provides a model parameter adjustment method, which specifically includes:

[0047] Determine the first parameter category of the model to be configured according to the task scenario of the model to be trained; determine the first target dynamic parameter corresponding to the first parameter category from the preset parameter list based on the first parameter category; initialize the model based on the first target dynamic parameter, and record the storage location information of the initialized first target dynamic parameter; in response to determining that the task scenario has changed, determine the second parameter type of the model to be adjusted based on the changed task scenario; determine the second target dynamic parameter corresponding to the second parameter type from the preset parameter list based on the second parameter type, and replace the first target dynamic parameter with the second target dynamic parameter based on the storage location information, so that the model can support multiple downstream task scenarios by adjusting the dynamic parameters, thereby improving the efficiency of model adjustment. Since in the prior art all parameters are replaced when adjusting parameters, that is, all parameters are replaced first and then initialized with the replaced parameters, when initializing the model, the storage location information of each parameter is not recorded after each parameter is loaded into the video memory or internal memory. The present application only replaces the dynamic parameters in all model parameters. Therefore, the storage location information of the dynamic parameters is recorded in advance, and the dynamic parameters can be directly replaced through the storage location information. This improves the efficiency of model parameter adjustment by adjusting a small number of dynamic parameters, while retaining those reusable static parameters, so that the model can be applied to multiple downstream tasks through a small amount of parameter adjustment.

[0048] After introducing the basic principles of the present application, various non-limiting implementation methods of the present application are described in detail below.

[0049] In some specific application scenarios, the model parameter adjustment method of the present application can be applied to any system and device involving model parameter adjustment.

[0050] In some specific application scenarios, the model parameter adjustment method of the present application can be directly applied and run locally, or it can be run on a cloud server. When running on a cloud server, the acquired data to be processed is sent to the cloud server via the network. The server processes the data to be processed using the model parameter adjustment method of the present application and sends the processing results to the local computer via the network.

[0051] The following describes the model parameter adjustment method according to the exemplary embodiment of the present application in conjunction with specific application scenarios. It should be noted that the above application scenarios are only shown to facilitate understanding of the spirit and principles of the present application, and the embodiments of the present application are not limited in this respect. On the contrary, the embodiments of the present application can be applied to any applicable scenario.

[0052] refer to Figure 1, an embodiment of the present application provides a model parameter adjustment method, comprising the following steps:

[0053] S101, determining a first parameter category to be configured for the model based on a task scenario of the model to be trained.

[0054] In specific implementation, since the model generally requires different parameters to be configured when processing different tasks, when training the model, the parameters that the model needs to be configured will first be selected based on the current task scenario. These parameters determine the model structure, initial weights and other information. Optionally, the task scenario of the model can be directly input by the user as needed. It should be noted that multiple parameters corresponding to the current task scenario can be determined through this first parameter category. These parameters are collectively classified as first-category parameters. The corresponding multiple parameters can be dynamic parameters or static parameters.

[0055] S102: Determine a first target dynamic parameter corresponding to the first parameter category from a preset parameter list based on the first parameter category.

[0056] In specific implementation, after determining the first parameter category of the model, the first target dynamic parameter can be found from the preset parameter list based on the first parameter category. It should be noted that the preset parameter list stores target dynamic parameters corresponding to multiple parameter types, and each target dynamic parameter corresponds to at least one task scenario that needs to be solved. The above-mentioned first target dynamic parameter is a dynamic parameter corresponding to the first parameter category. The dynamic parameter refers to those parameters that change during model training, such as weight parameters, which will continue to change when training the model with a large amount of sample data.

[0057] S103: Initialize the model based on the first target dynamic parameter, and record storage location information of the initialized first target dynamic parameter.

[0058] In a specific implementation, after determining the first target dynamic parameter, the model to be trained is initialized according to the first target parameter, and the storage location information of the initialized first target parameter is saved. The initialization process is to load the determined first target dynamic parameter into the video memory or memory for subsequent calculation use.

[0059] In some embodiments, after initializing the model based on the first target dynamic parameter, the method further includes:

[0060] Obtaining multiple static parameters of the initialized model;

[0061] A target static parameter of the model is determined from the plurality of static parameters based on the first parameter category, so that the target static parameter participates in the training process of the model.

[0062] During specific implementation, there are many static parameters in the model, which determine the structural characteristics of the model, and these static parameters will not change during model training. In order to make the model parameter adjustment method of the present application not only applicable to different task scenarios by adjusting only the dynamic parameters of the model when the required model structure is the same, but also to complete the change of the model structure by adjusting the static parameters when the required model structure is different. In some embodiments, the present application loads multiple static parameters that may be used for different model structures into the video memory or memory at the initialization of the model, and then determines the target static parameters of the model according to the first parameter category, and only allows these target static parameters to participate in training and calculation in the subsequent model training process, while other static parameters will not participate in the calculation, which is equivalent to directly shielding the static parameters other than the target static parameters.

[0063] In some embodiments, the preset parameter list is set in a memory; before initializing the model based on the first target dynamic parameter, the method further includes:

[0064] The first target dynamic parameter is transferred from the preset parameter list to the target video memory.

[0065] In specific implementation, in order to improve computing efficiency, the model initialization is generally run in the GPU. At this time, the dynamic parameters need to be loaded into the target video memory of the GPU. When the preset parameter list is set in the memory, in order to increase the parameter loading speed, the first target dynamic parameter will be first transferred from the preset parameter list to the target video memory.

[0066] In some embodiments, the first target dynamic parameters include a plurality of parameter matrices; and transferring the first target dynamic parameters from the preset parameter list to the target video memory specifically includes:

[0067] For each parameter matrix in the plurality of parameter matrices, obtaining a first size of the parameter matrix in the first dimension and a second size of the parameter matrix in the second dimension;

[0068] splicing the first parameter matrices of the same size along a second dimension into a merged parameter matrix, and transferring the merged parameter matrix from the preset parameter list to a target video memory;

[0069] The merged parameter matrix transmitted to the target video memory is split into a plurality of parameter matrices having the same first size based on the second size.

[0070] During specific implementation, the first target dynamic parameters generally include multiple parameter matrices. Under normal circumstances, these parameter matrices need to be transferred one by one from the memory to the GPU. In order to improve the efficiency of data transmission, when transmitting the parameter matrix, the multiple parameter matrices can be spliced ​​into a merged parameter matrix, and then when the data transmission is completed, the individual parameter matrices are split out of the merged parameter matrix. During operation, first obtain the first size of each parameter matrix in the first dimension and the second size in the second dimension. The first dimension and the second dimension represent the rows and columns of the parameter matrix, respectively. Optionally, the order of the first dimension and the second dimension can be interchanged, which is not limited here. After determining the first dimension and the second dimension of each data matrix, the parameter matrices with the same first size are spliced ​​along the second dimension into a merged parameter matrix, and the merged parameter matrix is ​​transferred from the preset parameter list to the target video memory, and then the merged parameter matrix transferred to the target video memory is split into multiple parameter matrices with the same first size according to the second dimension. Reference Figure 2 , where the number of columns of the two parameter matrices on the left are both 2, so the first size in the first dimension is the same. Therefore, the two parameter matrices on the left can be spliced ​​into the middle combined parameter matrix along the second dimension, and then the middle combined parameter matrix is ​​transmitted as a whole for data transmission. After the data transmission is completed, the middle combined parameter matrix is ​​split into the two parameter matrices on the right.

[0071] To further improve data transmission efficiency, in some embodiments, the first target dynamic parameter includes a plurality of parameter matrices, and transmitting the first target dynamic parameter from the preset parameter list to the target video memory specifically includes:

[0072] For each parameter matrix in the plurality of parameter matrices, obtaining a first size of the parameter matrix in the first dimension and a second size of the parameter matrix in the second dimension;

[0073] In response to determining that a first number of the first parameter matrices having the same size is less than a second number of the second parameter matrices having the same size, concatenating the second parameter matrices having the same size along a first dimension into a first merged parameter matrix, and transferring the first merged parameter matrix from the preset parameter list to a target video memory;

[0074] The first merged parameter matrix transmitted to the target video memory is split into a plurality of parameter matrices having the same first size based on the first size.

[0075] In some embodiments, transferring the first target dynamic parameter from the preset parameter list to the target video memory specifically includes:

[0076] In response to determining that a first number of the first parameter matrices having the same size is greater than a second number of the second parameter matrices having the same size, concatenating the first parameter matrices having the same size along a second dimension into a merged parameter matrix, and transferring the second merged parameter matrix from the preset parameter list to a target video memory;

[0077] The second merge parameter matrix transmitted to the target video memory is split into a plurality of parameter matrices having the same first size based on the second size.

[0078] In a specific implementation, since when transmitting data, the fewer the number of merged matrices, the fewer the number of transmissions, and thus the higher the transmission efficiency, when splicing the parameter matrices into a merged parameter matrix, it is first determined whether the first number of parameter matrices with the same first size or the second number of parameter matrices with the same second size is larger, and then splicing is performed based on the determination result to obtain a merged parameter matrix with the smallest possible total number.

[0079] S104 : In response to determining that the task scenario has changed, determining a second parameter type of the model based on the changed task scenario.

[0080] In specific implementation, during model training, when the task scenario is changed, the corresponding model parameters will also change. It should be noted that the change in the task scenario refers to the current task scenario being different from the last received task scenario.

[0081] S105: Determine a second target dynamic parameter corresponding to the second parameter type from a preset parameter list based on the second parameter type, and replace the first target dynamic parameter with the second target dynamic parameter based on the storage location information.

[0082] In specific implementation, after determining the second parameter type of the model, first determine the second target dynamic parameter corresponding to the second parameter type from the preset parameter list according to the second parameter type, and then replace the first target dynamic parameter with the second target dynamic parameter according to the storage location information of the previously recorded first target dynamic parameter, thereby completing the replacement of the dynamic parameter without changing other parameters.

[0083] In some embodiments, the preset parameter list includes a first preset list and a second preset list, the first preset list is set in the GPU, and the second preset list is set in the memory; after determining the first target dynamic parameter corresponding to the first parameter category from the preset parameter list based on the first parameter category, the method further includes:

[0084] Determine a third target dynamic parameter in the first preset list that has participated in the initialization the least number of times;

[0085] In response to determining that the first target dynamic parameter is in the second preset list, the third target dynamic parameter is replaced with the storage location of the first target dynamic parameter.

[0086] In specific implementation, in general, in order to improve the efficiency of model operation, the model initialization is run in the GPU, so if the preset list is set in the GPU's video memory, the speed of parameter acquisition will increase, but the GPU video memory capacity is generally not too large, so the number of dynamic parameters stored in the GPU is limited. In order to reduce the number of times the target dynamic parameters are transferred from the memory to the GPU, the dynamic parameters in the second preset list set in the GPU are preferably the most frequently used dynamic parameters. When it is necessary to free up a position for a dynamic parameter for the current task scenario, it is preferred that the third target dynamic parameter that has participated in the initialization the least number of times is used as the replacement object.

[0087] In some embodiments, after determining a second target dynamic parameter corresponding to the second parameter type from a preset parameter list based on the second parameter type, the method further includes:

[0088] In response to determining that the second target dynamic parameter is in the second preset list, the third target dynamic parameter is replaced with the storage location of the first target dynamic parameter.

[0089] The model parameter adjustment method provided in the present application determines the first parameter category of the model according to the task scenario of the model to be trained; determines the first target dynamic parameter corresponding to the first parameter category from a preset parameter list based on the first parameter category; initializes the model based on the first target dynamic parameter, and records the storage location information of the initialized first target dynamic parameter; in response to determining that the task scenario has changed, determines the second parameter type of the model based on the changed task scenario; determines the second target dynamic parameter corresponding to the second parameter type from a preset parameter list based on the second parameter type, and replaces the first target dynamic parameter with the second target dynamic parameter based on the storage location information, so that the model can support multiple downstream task scenarios by adjusting the dynamic parameters, thereby improving the efficiency of model adjustment.

[0090] Based on the same inventive concept, corresponding to any of the above-mentioned embodiment methods, the present application also provides a model parameter adjustment device.

[0091] refer to Figure 3 , the model parameter adjustment device comprises:

[0092] A first determining module 201 determines a first parameter category of the model based on a task scenario of the model to be trained;

[0093] A second determining module 202 determines, based on the first parameter category, a first target dynamic parameter corresponding to the first parameter category from a preset parameter list;

[0094] An initialization module 203 initializes the model based on the first target dynamic parameter and records storage location information of the initialized first target dynamic parameter;

[0095] A third determining module 204, in response to determining that the task scenario has changed, determines a second parameter type of the model based on the changed task scenario;

[0096] The parameter replacement module 205 determines a second target dynamic parameter corresponding to the second parameter type from a preset parameter list based on the second parameter type, and replaces the first target dynamic parameter with the second target dynamic parameter based on the storage location information.

[0097] For the convenience of description, the above devices are described as being divided into various modules according to their functions. Of course, when implementing this application, the functions of each module can be implemented in the same or multiple software and / or hardware.

[0098] The apparatus of the above embodiment is used to implement the corresponding model parameter adjustment method in any of the above embodiments, and has the beneficial effects of the corresponding method embodiment, which will not be described in detail here.

[0099] Based on the same inventive concept, corresponding to any of the above-mentioned embodiments and methods, the present application also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and runnable on the processor, wherein when the processor executes the program, the model parameter adjustment method described in any of the above embodiments is implemented.

[0100] Figure 4 10 is a schematic diagram showing a more specific hardware structure of an electronic device provided in this embodiment. The device may include: a processor 1010, a memory 1020, an input / output interface 1030, a communication interface 1040, and a bus 1050. The processor 1010, the memory 1020, the input / output interface 1030, and the communication interface 1040 are communicatively connected to each other within the device via the bus 1050.

[0101] The processor 1010 can be implemented using a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this specification.

[0102] The memory 1020 can be implemented in the form of ROM (Read Only Memory), RAM (Random Access Memory), static storage devices, dynamic storage devices, etc. The memory 1020 can store an operating system and other application programs. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program code is stored in the memory 1020 and is called and executed by the processor 1010.

[0103] The input / output interface 1030 is used to connect input / output modules to implement information input and output. The input / output modules can be configured as components within the device (not shown in the figure) or can be externally connected to the device to provide corresponding functions. Input devices may include a keyboard, mouse, touch screen, microphone, various sensors, etc., and output devices may include a display, speaker, vibrator, indicator light, etc.

[0104] The communication interface 1040 is used to connect to a communication module (not shown) to enable communication between the device and other devices. The communication module can communicate via a wired method (such as USB, network cable, etc.) or a wireless method (such as mobile network, WiFi, Bluetooth, etc.).

[0105] The bus 1050 comprises a path for transmitting information between the various components of the device (eg, the processor 1010 , the memory 1020 , the input / output interface 1030 , and the communication interface 1040 ).

[0106] It should be noted that although the above device only shows the processor 1010, the memory 1020, the input / output interface 1030, the communication interface 1040, and the bus 1050, in a specific implementation, the device may also include other components necessary for normal operation. In addition, it will be understood by those skilled in the art that the above device may only include the components necessary to implement the embodiments of this specification, and does not necessarily include all the components shown in the figure.

[0107] The electronic device of the above embodiment is used to implement the corresponding model parameter adjustment method in any of the above embodiments, and has the beneficial effects of the corresponding method embodiment, which will not be described in detail here.

[0108] Based on the same inventive concept, corresponding to any of the above-mentioned embodiment methods, the present application also provides a non-transitory computer-readable storage medium, wherein the non-transitory computer-readable storage medium stores computer instructions, and the computer instructions are used to enable the computer to execute the model parameter adjustment method described in any of the above embodiments.

[0109] The computer-readable media of this embodiment include permanent and non-permanent, removable and non-removable media that can be used to store information by any method or technology. The information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, read-only compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, tape disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device.

[0110] The computer instructions stored in the storage medium of the above embodiment are used to enable the computer to execute the model parameter adjustment method described in any of the above embodiments, and have the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0111] Those skilled in the art should understand that the discussion of any of the above embodiments is merely illustrative and is not intended to imply that the scope of the present application (including the claims) is limited to these examples. Within the scope of the present application, the technical features in the above embodiments or different embodiments may be combined, the steps may be implemented in any order, and there are many other variations of the different aspects of the embodiments of the present application as described above, which are not provided in detail for the sake of simplicity.

[0112] In addition, for simplicity of description and discussion, and in order not to make the embodiment of the application difficult to understand, the known power supply / ground connection with integrated circuit (IC) chip and other components may or may not be shown in the accompanying drawings provided. In addition, the device can be shown in the form of a block diagram to avoid making the embodiment of the application difficult to understand, and this also takes into account the following fact, that is, the details of the embodiment of these block diagram devices are highly dependent on the platform to be implemented in the embodiment of the application (that is, these details should be fully within the scope of understanding of those skilled in the art). When specific details (for example, circuit) are set forth to describe exemplary embodiments of the application, it will be apparent to those skilled in the art that the embodiment of the application can be implemented without these specific details or when these specific details are changed. Therefore, these descriptions should be considered to be illustrative rather than restrictive.

[0113] Although the present invention has been described in conjunction with specific embodiments thereof, many alternatives, modifications, and variations of these embodiments will be apparent to those skilled in the art based on the foregoing description. For example, other memory architectures (e.g., dynamic RAM (DRAM)) may utilize the embodiments discussed.

[0114] The embodiments of the present application are intended to cover all such substitutions, modifications, and variations that fall within the broad scope of the appended claims. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the embodiments of the present application should be included in the scope of protection of this application.

Claims

1. A model parameter adjustment method, characterized in that: include: Determining a first parameter category to be configured for the model based on a task scenario of the model to be trained; Determining a first target dynamic parameter corresponding to the first parameter category from a preset parameter list based on the first parameter category; The preset parameter list is set in the memory; The first target dynamic parameter includes a plurality of parameter matrices; for each parameter matrix in the plurality of parameter matrices, obtaining a first size of the first dimension and a second size of the second dimension of each parameter matrix; splicing the first parameter matrices of the same size along a second dimension into a merged parameter matrix, and transferring the merged parameter matrix from the preset parameter list to a target video memory; splitting the merged parameter matrix transmitted to the target video memory into a plurality of parameter matrices having the same first size based on the second size; Initializing the model based on the first target dynamic parameter, and recording storage location information of the initialized first target dynamic parameter; In response to determining that the task scenario has changed, determining a second parameter type of the model to be adjusted based on the changed task scenario; A second target dynamic parameter corresponding to the second parameter type is determined from the preset parameter list based on the second parameter type, and the first target dynamic parameter is replaced with the second target dynamic parameter based on the storage location information.

2. The method according to claim 1, characterized in that After initializing the model based on the first target dynamic parameter, the method further includes: Obtaining multiple static parameters of the initialized model; A target static parameter of the model is determined from the plurality of static parameters based on the first parameter category, so that the target static parameter participates in the training process of the model.

3. The method according to claim 1, characterized in that Before splicing the first parameter matrices of the same size into a merged parameter matrix along the second dimension, transferring the first target dynamic parameter from the preset parameter list to a target video memory, specifically further comprising: It is determined that a first number of parameter matrices having the same first size is greater than a second number of parameter matrices having the same second size.

4. The method according to claim 1, wherein The preset parameter list includes a first preset list and a second preset list, the first preset list is set in the video memory, and the second preset list is set in the internal memory; After determining a first target dynamic parameter corresponding to the first parameter category from a preset parameter list based on the first parameter category, the method further includes: Determine a third target dynamic parameter in the first preset list that has participated in the initialization the least number of times; In response to determining that the first target dynamic parameter is in the second preset list, the third target dynamic parameter is replaced with the storage location of the first target dynamic parameter.

5. The method according to claim 4, characterized in that After determining a second target dynamic parameter corresponding to the second parameter type from a preset parameter list based on the second parameter type, the method further includes: In response to determining that the second target dynamic parameter is in the second preset list, the third target dynamic parameter is replaced with the storage location of the first target dynamic parameter.

6. A model parameter adjustment device, characterized in that: include: A first determining module determines a first parameter category to be configured for the model based on a task scenario of the model to be trained; a second determining module, determining a first target dynamic parameter corresponding to the first parameter category from a preset parameter list based on the first parameter category; The preset parameter list is set in the memory; the first target dynamic parameter includes a plurality of parameter matrices; for each parameter matrix in the plurality of parameter matrices, a first size of the first dimension and a second size of the second dimension of each parameter matrix are obtained; splicing the first parameter matrices of the same size along a second dimension into a merged parameter matrix, and transferring the merged parameter matrix from the preset parameter list to a target video memory; splitting the merged parameter matrix transmitted to the target video memory into a plurality of parameter matrices having the same first size based on the second size; an initialization module, configured to initialize the model based on the first target dynamic parameter and record storage location information of the initialized first target dynamic parameter; a third determining module, in response to determining that the task scenario has changed, determining a second parameter type of the model to be adjusted based on the changed task scenario; A parameter replacement module determines a second target dynamic parameter corresponding to the second parameter type from a preset parameter list based on the second parameter type, and replaces the first target dynamic parameter with the second target dynamic parameter based on the storage location information.

7. An electronic device, characterized in that: The method comprises a memory, a processor, and a computer program stored in the memory and executable by the processor, wherein the processor implements the method according to any one of claims 1 to 5 when executing the program.

8. A non-transitory computer-readable storage medium, characterized in that The non-transitory computer-readable storage medium stores computer instructions for causing a computer to execute the method according to any one of claims 1 to 5.

Images