Model bypass tuning method and device

Abstract

The present disclosure relates to the field of machine learning technology and provides a model bypass tuning method and device. The method includes: determining a small-scale target model corresponding to a large-scale target model used to perform a target task, wherein the small-scale target model and the large-scale target model are different versions of the target model, and the model scale of the small-scale target model is smaller than the model scale of the large-scale target model; constructing a dual-path target model using the small-scale target model and the large-scale target model, wherein the small-scale target model is a side branch network of the dual-path target model and the large-scale target model is a main branch network of the dual-path target model; training the dual-path target model: freezing the model parameters of the large-scale target model in the dual-path target model, and training the small-scale target model in the dual-path target model; and performing the target task using the trained dual-path target model.

Description

Technical Field

[0001] The present disclosure relates to the field of machine learning technology, and in particular to a model bypass tuning method and device. Background Art

[0002] In recent years, with the advancement of machine learning technology, an increasing number of large-scale models have been applied to various fields, such as computer vision, natural language processing, and text processing. To ensure that large-scale models meet requirements and improve training efficiency, it is common to further train pre-trained large-scale models. However, because large-scale models often have high computational complexity and a large number of parameters, further training of pre-trained large-scale models also faces the problem of being time-consuming and requiring a large number of parameters to be tuned.

[0003] In the process of realizing the concept of the present disclosure, the inventors discovered that there are at least the following technical problems in the related art: large-scale model training faces the problem of a large number of parameters to be tuned and a long time consumption. Summary of the Invention

[0004] In view of this, the embodiments of the present disclosure provide a model bypass tuning method, device, electronic device and computer-readable storage medium to solve the problems in the prior art of large-scale model training faced with a large number of parameters to be tuned and a long time consumption.

[0005] According to a first aspect of an embodiment of the present disclosure, a model bypass tuning method is provided, comprising: determining a small-scale target model corresponding to a large-scale target model used to perform a target task, wherein the small-scale target model and the large-scale target model are different versions of the target model, and the model scale of the small-scale target model is smaller than the model scale of the large-scale target model; constructing a dual-path target model using the small-scale target model and the large-scale target model, wherein the small-scale target model is a side branch network of the dual-path target model, and the large-scale target model is a main branch network of the dual-path target model; training the dual-path target model: freezing the model parameters of the large-scale target model in the dual-path target model, and training the small-scale target model in the dual-path target model; and performing the target task using the trained dual-path target model.

[0006] According to a second aspect of an embodiment of the present disclosure, a model bypass tuning device is provided, comprising: a determination module configured to determine a small-scale target model corresponding to a large-scale target model used to perform a target task, wherein the small-scale target model and the large-scale target model are different versions of the target model, and the model scale of the small-scale target model is smaller than the model scale of the large-scale target model; a construction module configured to construct a two-path target model using the small-scale target model and the large-scale target model, wherein the small-scale target model is a side branch network of the two-path target model, and the large-scale target model is a main branch network of the two-path target model; a training module configured to train the two-path target model: freeze the model parameters of the large-scale target model in the two-path target model, and train the small-scale target model in the two-path target model; and an execution module configured to execute the target task using the trained two-path target model.

[0007] According to a third aspect of an embodiment of the present disclosure, an electronic device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above method when executing the computer program.

[0008] According to a fourth aspect of the embodiments of the present disclosure, a computer-readable storage medium is provided, which stores a computer program. When the computer program is executed by a processor, the steps of the above method are implemented.

[0009] Compared with the prior art, the embodiments of the present disclosure have the following beneficial effects: because the embodiments of the present disclosure determine the small-scale target model corresponding to the large-scale target model used to perform the target task, wherein the small-scale target model and the large-scale target model are different versions of the target model, and the model scale of the small-scale target model is smaller than the model scale of the large-scale target model; the small-scale target model and the large-scale target model are used to construct a dual-path target model, wherein the small-scale target model is a side branch network of the dual-path target model, and the large-scale target model is a main branch network of the dual-path target model; the dual-path target model is trained: the model parameters of the large-scale target model in the dual-path target model are frozen, and the small-scale target model in the dual-path target model is trained; the trained dual-path target model is used to perform the target task. Therefore, the above-mentioned technical means can solve the problem of a large number of parameters to be tuned and a long time in large-scale model training in the prior art, thereby reducing the number of parameters to be tuned and the training time. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] In order to more clearly illustrate the technical solutions in the embodiments of the present disclosure, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0011] Figure 1 This is a flow chart of a model bypass tuning method provided by an embodiment of the present disclosure (I);

[0012] Figure 2 This is a flow chart (II) of a model bypass tuning method provided by an embodiment of the present disclosure;

[0013] Figure 3 Schematic diagram of the structure of a model bypass tuning device provided by an embodiment of the present disclosure;

[0014] Figure 4 It is a structural diagram of an electronic device provided by an embodiment of the present disclosure. DETAILED DESCRIPTION

[0015] In the following description, specific details such as specific system structures and techniques are provided for purposes of illustration rather than limitation to facilitate a thorough understanding of the embodiments of the present disclosure. However, it will be apparent to those skilled in the art that the present disclosure may be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to avoid obscuring the description of the present disclosure with unnecessary detail.

[0016] Figure 1 This is a flow chart (1) of a model bypass tuning method provided in an embodiment of the present disclosure. Figure 1 The model bypass tuning method can be executed by a computer or server, or software on the computer or server. Figure 1 As shown in Figure 2, the model bypass tuning method includes:

[0017] S101, determining a small-scale target model corresponding to a large-scale target model for executing a target task, wherein the small-scale target model and the large-scale target model are different versions of the target model, and the model scale of the small-scale target model is smaller than the model scale of the large-scale target model;

[0018] S102, constructing a dual-path target model using the small-scale target model and the large-scale target model, wherein the small-scale target model is a side branch network of the dual-path target model, and the large-scale target model is a main branch network of the dual-path target model;

[0019] S103, training the dual-path target model: freezing the model parameters of the large-scale target model in the dual-path target model, and training the small-scale target model in the dual-path target model;

[0020] S104, using the trained dual-path target model to perform the target task.

[0021] The large-scale target model has been pre-trained, but the small-scale target model has not been pre-trained; or both the large-scale target model and the small-scale target model have been pre-trained.

[0022] In the prior art, in order to ensure that the large-scale model meets the requirements, the pre-trained large-scale model is often further trained. Since the large-scale model has a large number of parameters to be tuned, further training of the pre-trained large-scale model is actually also faced with the problem of being time-consuming. The reason why further training of the large-scale model takes a long time is that the large-scale model has a large number of parameters to be tuned, while the small-scale model has fewer parameters to be tuned. If the small-scale target model is trained, the training time can be reduced. Therefore, based on the above considerations, the embodiment of the present disclosure uses the small-scale target model as a side network and the large-scale target model as the main network to construct a dual-path target model. During training, the model parameters of the large-scale target model are frozen and the small-scale target model is trained. Since the network layer inside the large-scale target model only provides input to the network layer inside the small-scale target model, the complexity of backpropagation depends on the scale of the small-scale target model. It is not necessary to perform backpropagation directly on the large-scale target model, so the training efficiency can be significantly improved.

[0023] The target task may be a text classification task, and the target model may be a text classification model. Of course, it is foreseeable that the target task may be other tasks, and the target model may be other models, such as an object detection task, and the target model may be an object detection model.

[0024] The small-scale target model and the large-scale target model only need to meet the conditions that they are different versions of the target model and the model scale of the small-scale target model is smaller than the model scale of the large-scale target model. The small-scale target model and the large-scale target model can be selected according to the situation.

[0025] For example, if the target task is text classification and the target model is BERT, BERT-base and BERT-large are commonly used for large-scale target models, while BERT-small, BERT-tiny, and BERT-mini are commonly used for small-scale target models. BERT-base has 12 encoding layers, BERT-large has 24 encoding layers, BERT-small and BERT-mini have 4 encoding layers, and BERT-tiny has 2 encoding layers. In practice, the number of attention heads per encoding layer of BERT-base, BERT-large, BERT-small, BERT-tiny, and BERT-mini can vary. For example, BERT-large can have 16 attention heads per encoding layer, while BERT-base can have 12 attention heads per encoding layer.

[0026] For example, BERT-base is often used for large-scale target models, while BERT-small is often used for small-scale target models. Training a large-scale target model alone reduces the workload by at least half compared to training a small-scale target model.

[0027] According to the technical solution provided by the embodiments of the present disclosure, a small-scale target model corresponding to a large-scale target model for executing a target task is determined, wherein the small-scale target model and the large-scale target model are different versions of the target model, and the model scale of the small-scale target model is smaller than the model scale of the large-scale target model; a dual-path target model is constructed using the small-scale target model and the large-scale target model, wherein the small-scale target model is a side branch network of the dual-path target model, and the large-scale target model is a main branch network of the dual-path target model; the dual-path target model is trained: the model parameters of the large-scale target model in the dual-path target model are frozen, and the small-scale target model in the dual-path target model is trained; the trained dual-path target model is used to execute the target task. Therefore, the above-mentioned technical means can be used to solve the problems in the prior art of large-scale model training facing a large number of parameters to be tuned and a long time, thereby reducing the number of parameters to be tuned and the training time.

[0028] A dual-path target model is constructed using a small-scale target model and a large-scale target model, including: determining the number N of network layers of the small-scale target model and the number M of network layers of the large-scale target model; dividing the M network layers of the large-scale target model into N groups according to the order of the network layers from front to back, thereby obtaining N network layer groups; on the basis of ensuring the connection relationship between the original internal network layers of the small-scale target model and the large-scale target model, connecting the output of the i-th network layer group of the large-scale target model to the input of the i-th network layer of the small-scale target model, thereby obtaining a dual-path target model; wherein, the inputs of the small-scale target model and the large-scale target model are the same, both being the inputs of the dual-path target model, and the output of the small-scale target model being the output of the dual-path target model, M is greater than N, i is at least 1, and i is at most N.

[0029] The output of the i-th network layer group of the large-scale target model is added to the input of the i-1-th network layer of the small-scale target model as the input of the i-th network layer of the small-scale target model.

[0030] Determining a small-scale target model corresponding to a large-scale target model used to perform a target task includes: obtaining expected indicators for training the dual-path target model, wherein the expected indicators include: training time for completing the dual-path target model training under preset conditions, and the accuracy, precision, and recall of the dual-path target model after training; determining the number of network layers of the large-scale target model; determining the number of network layers of the small-scale target model based on the expected indicators and the number of network layers of the large-scale target model; and determining a version from multiple versions of the target model as the small-scale target model based on the number of network layers of the small-scale target model.

[0031] The number of network layers of the version determined from multiple versions of the target model is the one that is closest to the number of network layers of the small-scale target model.

[0032] For example, the number of network layers of the large-scale target model is certain. Under the preset conditions, the shorter the training time to complete the dual-path target model training, the fewer network layers the small-scale target model has. The preset conditions can be the conventional computing power of the computer at the current time. For example, the corresponding preset conditions during 2021-2022 are the computing power of the computer with a CPU of i7-12700 and a GPU of RTX3060. The number of network layers of the large-scale target model is certain. The higher the precision, accuracy or recall rate of the dual-path target model, the more network layers the small-scale target model has. The expected indicators are certain, and the number of network layers of the large-scale target model is proportional to the number of network layers of the small-scale target model.

[0033] In an optional embodiment, another method for training a dual-path target model is provided, including: freezing the model parameters of the small-scale target model in the dual-path target model, and training the large-scale target model in the dual-path target model; wherein both the large-scale target model and the small-scale target model have been pre-trained.

[0034] In this optional embodiment, training the large-scale target model in the dual-path target model is actually the same as the workload of training the large-scale target model alone, but training the large-scale target model in the dual-path target model can achieve a better effect, that is, under the same training, the dual-path target model has a better effect.

[0035] In this optional embodiment, on the basis of ensuring the connection relationship between the original internal network layers of the small-scale target model and the large-scale target model, the output of the i-th network layer of the small-scale target model is connected to the input of the i-th network layer group of the large-scale target model to obtain a dual-path target model; wherein, the inputs of the small-scale target model and the large-scale target model are the same, both are the inputs of the dual-path target model, and the output of the large-scale target model is the output of the dual-path target model; the output of the i-th network layer of the small-scale target model is added to the input of the i-1-th network layer group of the large-scale target model as the input of the i-th network layer group of the large-scale target model.

[0036] The input of the dual-path target model is input into both the small-scale target model and the large-scale target model at the same time, that is, the dual-path target model has two input ports.

[0037] Figure 2 This is a flow chart (II) of a model bypass tuning method provided in an embodiment of the present disclosure.

[0038] like Figure 2 As shown, including:

[0039] S201, determining a small-scale target model corresponding to a large-scale target model for executing a target task;

[0040] S202, constructing a dual-path target model using the small-scale target model and the large-scale target model;

[0041] S203, training the dual-path target model: freezing the model parameters of the small-scale target model in the dual-path target model, and training the large-scale target model in the dual-path target model;

[0042] S204, using the trained dual-path target model to perform the target task.

[0043] According to the technical solution provided in the embodiments of the present disclosure, a small-scale target model corresponding to a large-scale target model used to perform a target task is determined; a dual-path target model is constructed using the small-scale target model and the large-scale target model; the dual-path target model is trained: the model parameters of the small-scale target model in the dual-path target model are frozen, and the large-scale target model in the dual-path target model is trained; and the trained dual-path target model is used to perform the target task. Therefore, the above-mentioned technical means can solve the problem of poor effect of large-scale model training in the existing technology, thereby improving the training effect of the large-scale model.

[0044] All of the above optional technical solutions can be combined in any way to form optional embodiments of the present application, and will not be described in detail here.

[0045] The following are embodiments of the apparatus disclosed herein, which can be used to implement the method embodiments disclosed herein. For details not disclosed in the apparatus embodiments disclosed herein, please refer to the method embodiments disclosed herein.

[0046] Figure 3 Schematic diagram of a model bypass tuning device provided by an embodiment of the present disclosure. Figure 3 As shown, the model bypass tuning device includes:

[0047] a determination module 301 configured to determine a small-scale target model corresponding to a large-scale target model for performing a target task, wherein the small-scale target model and the large-scale target model are different versions of the target model, and a model scale of the small-scale target model is smaller than a model scale of the large-scale target model;

[0048] A construction module 302 is configured to construct a two-path target model using the small-scale target model and the large-scale target model, wherein the small-scale target model is a side branch network of the two-path target model and the large-scale target model is a main branch network of the two-path target model;

[0049] The training module 303 is configured to train the two-path target model: freeze the model parameters of the large-scale target model in the two-path target model, and train the small-scale target model in the two-path target model;

[0050] The execution module 304 is configured to execute the target task using the trained two-path target model.

[0051] The large-scale target model has been pre-trained, but the small-scale target model has not been pre-trained; or both the large-scale target model and the small-scale target model have been pre-trained.

[0052] In the prior art, in order to ensure that the large-scale model meets the requirements, the pre-trained large-scale model is often further trained. Since the large-scale model has a large number of parameters to be tuned, further training of the pre-trained large-scale model is actually also faced with the problem of being time-consuming. The reason why further training of the large-scale model takes a long time is that the large-scale model has a large number of parameters to be tuned, while the small-scale model has fewer parameters to be tuned. If the small-scale target model is trained, the training time can be reduced. Therefore, based on the above considerations, the embodiment of the present disclosure uses the small-scale target model as a side network and the large-scale target model as the main network to construct a dual-path target model. During training, the model parameters of the large-scale target model are frozen and the small-scale target model is trained. Since the network layer inside the large-scale target model only provides input to the network layer inside the small-scale target model, the complexity of backpropagation depends on the scale of the small-scale target model. It is not necessary to perform backpropagation directly on the large-scale target model, so the training efficiency can be significantly improved.

[0053] The target task may be a text classification task, and the target model may be a text classification model. Of course, it is foreseeable that the target task may be other tasks, and the target model may be other models, such as an object detection task, and the target model may be an object detection model.

[0054] The small-scale target model and the large-scale target model only need to meet the conditions that they are different versions of the target model and the model scale of the small-scale target model is smaller than the model scale of the large-scale target model. The small-scale target model and the large-scale target model can be selected according to the situation.

[0055] For example, if the target task is text classification and the target model is BERT, BERT-base and BERT-large are commonly used for large-scale target models, while BERT-small, BERT-tiny, and BERT-mini are commonly used for small-scale target models. BERT-base has 12 encoding layers, BERT-large has 24 encoding layers, BERT-small and BERT-mini have 4 encoding layers, and BERT-tiny has 2 encoding layers. In practice, the number of attention heads per encoding layer of BERT-base, BERT-large, BERT-small, BERT-tiny, and BERT-mini can vary. For example, BERT-large can have 16 attention heads per encoding layer, while BERT-base can have 12 attention heads per encoding layer.

[0056] For example, BERT-base is often used for large-scale target models, while BERT-small is often used for small-scale target models. Training a large-scale target model alone reduces the workload by at least half compared to training a small-scale target model.

[0057] According to the technical solution provided by the embodiments of the present disclosure, a small-scale target model corresponding to a large-scale target model for executing a target task is determined, wherein the small-scale target model and the large-scale target model are different versions of the target model, and the model scale of the small-scale target model is smaller than the model scale of the large-scale target model; a dual-path target model is constructed using the small-scale target model and the large-scale target model, wherein the small-scale target model is a side branch network of the dual-path target model, and the large-scale target model is a main branch network of the dual-path target model; the dual-path target model is trained: the model parameters of the large-scale target model in the dual-path target model are frozen, and the small-scale target model in the dual-path target model is trained; the trained dual-path target model is used to execute the target task. Therefore, the above-mentioned technical means can be used to solve the problems in the prior art of large-scale model training facing a large number of parameters to be tuned and a long time, thereby reducing the number of parameters to be tuned and the training time.

[0058] Optionally, the construction module 302 is also configured to determine the number N of network layers of the small-scale target model and the number M of network layers of the large-scale target model; divide the M network layers of the large-scale target model into N groups according to the order of the network layers from front to back to obtain N network layer groups; on the basis of ensuring the connection relationship between the original internal network layers of the small-scale target model and the large-scale target model, connect the output of the i-th network layer group of the large-scale target model to the input of the i-th network layer of the small-scale target model to obtain a two-path target model; wherein, the inputs of the small-scale target model and the large-scale target model are the same, both are the inputs of the two-path target model, the output of the small-scale target model is the output of the two-path target model, M is greater than N, i is at least 1, and i is at most N; add the output of the i-th network layer group of the large-scale target model to the input of the i-1-th network layer of the small-scale target model as the input of the i-th network layer of the small-scale target model.

[0059] Optionally, the determination module 301 is also configured to obtain expected indicators for training the dual-path target model, wherein the expected indicators include: the training time for completing the dual-path target model training under preset conditions, the accuracy, precision and recall rate of the dual-path target model after training; determining the number of network layers of the large-scale target model; determining the number of network layers of the small-scale target model based on the expected indicators and the number of network layers of the large-scale target model; and determining a version from multiple versions of the target model as the small-scale target model based on the number of network layers of the small-scale target model.

[0060] The number of network layers of the version determined from multiple versions of the target model is the one that is closest to the number of network layers of the small-scale target model.

[0061] For example, the number of network layers of the large-scale target model is certain. Under the preset conditions, the shorter the training time to complete the dual-path target model training, the fewer network layers the small-scale target model has. The preset conditions can be the conventional computing power of the computer at the current time. For example, the corresponding preset conditions during 2021-2022 are the computing power of the computer with a CPU of i7-12700 and a GPU of RTX3060. The number of network layers of the large-scale target model is certain. The higher the precision, accuracy or recall rate of the dual-path target model, the more network layers the small-scale target model has. The expected indicators are certain, and the number of network layers of the large-scale target model is proportional to the number of network layers of the small-scale target model.

[0062] Optionally, the training module 303 is further configured to freeze the model parameters of the small-scale target model in the dual-path target model and train the large-scale target model in the dual-path target model; wherein both the large-scale target model and the small-scale target model have been pre-trained.

[0063] In this optional embodiment, training the large-scale target model in the dual-path target model is actually the same as the workload of training the large-scale target model alone, but training the large-scale target model in the dual-path target model can achieve a better effect, that is, under the same training, the dual-path target model has a better effect.

[0064] Optionally, the training module 303 is also configured to connect the output of the i-th network layer of the small-scale target model to the input of the i-th network layer group of the large-scale target model on the basis of ensuring the connection relationship between the original internal network layers of the small-scale target model and the large-scale target model, so as to obtain a dual-path target model; wherein, the inputs of the small-scale target model and the large-scale target model are the same, both are the inputs of the dual-path target model, and the output of the large-scale target model is the output of the dual-path target model; the output of the i-th network layer of the small-scale target model is added to the input of the i-1-th network layer group of the large-scale target model as the input of the i-th network layer group of the large-scale target model; the input of the dual-path target model will be input into the small-scale target model and the large-scale target model at the same time, that is, the dual-path target model has two input ports.

[0065] Optionally, the determination module 301 is further configured to determine a small-scale target model corresponding to a large-scale target model for performing the target task;

[0066] Optionally, the construction module 302 is further configured to construct a two-path target model using the small-scale target model and the large-scale target model;

[0067] Optionally, the training module 303 is further configured to train the two-path target model: freeze the model parameters of the small-scale target model in the two-path target model, and train the large-scale target model in the two-path target model;

[0068] Optionally, the execution module 304 is further configured to execute the target task using the trained two-path target model.

[0069] According to the technical solution provided in the embodiments of the present disclosure, a small-scale target model corresponding to a large-scale target model used to perform a target task is determined; a dual-path target model is constructed using the small-scale target model and the large-scale target model; the dual-path target model is trained: the model parameters of the small-scale target model in the dual-path target model are frozen, and the large-scale target model in the dual-path target model is trained; and the trained dual-path target model is used to perform the target task. Therefore, the above-mentioned technical means can solve the problem of poor effect of large-scale model training in the existing technology, thereby improving the training effect of the large-scale model.

[0070] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present disclosure.

[0071] Figure 4 FIG. 4 is a schematic diagram of an electronic device 4 provided in an embodiment of the present disclosure. Figure 4 As shown, the electronic device 4 of this embodiment includes: a processor 401, a memory 402, and a computer program 403 stored in the memory 402 and executable by the processor 401. When the processor 401 executes the computer program 403, the steps of the above-described method embodiments are implemented. Alternatively, when the processor 401 executes the computer program 403, the functions of the modules / units in the above-described device embodiments are implemented.

[0072] The electronic device 4 may be a desktop computer, a notebook, a PDA, a cloud server, or other electronic device. The electronic device 4 may include but is not limited to a processor 401 and a memory 402. Those skilled in the art will appreciate that Figure 4 This is merely an example of the electronic device 4 and does not limit the electronic device 4 . The electronic device 4 may include more or fewer components than shown in the figure, or different components.

[0073] The processor 401 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc.

[0074] Memory 402 can be an internal storage unit of electronic device 4, such as a hard disk or memory of electronic device 4. Memory 402 can also be an external storage device of electronic device 4, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) card, a flash memory card, etc. equipped on electronic device 4. Memory 402 can also include both an internal storage unit of electronic device 4 and an external storage device. Memory 402 is used to store computer programs and other programs and data required by the electronic device.

[0075] Those skilled in the art will clearly understand that for the sake of convenience and brevity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In actual applications, the above-mentioned functions can be distributed and completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The above-mentioned integrated units can be implemented in the form of hardware or in the form of software functional units.

[0076] If the integrated module / unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present disclosure implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and when the computer program is executed by the processor, it can implement the steps of the above-mentioned various method embodiments. The computer program may include computer program code, which may be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may include: any entity or device capable of carrying computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium. It should be noted that the content contained in the computer-readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media do not include electric carrier signals and telecommunication signals.

[0077] The above embodiments are only used to illustrate the technical solutions of the present disclosure, rather than to limit them. Although the present disclosure has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present disclosure, and should all be included in the scope of protection of the present disclosure.

Claims

1. A model bypass tuning method, characterized in that: include: Determining a small-scale target model corresponding to a large-scale target model for performing a target task, wherein the small-scale target model and the large-scale target model are different versions of the target model, and the model scale of the small-scale target model is smaller than the model scale of the large-scale target model; the target task is a text classification task, and the target model is a text classification model, or the target task is an object detection task, and the target model is an object detection model; Constructing a two-path target model using the small-scale target model and the large-scale target model, wherein the small-scale target model is a side branch network of the two-path target model and the large-scale target model is a main branch network of the two-path target model; Training the two-path target model: freezing the model parameters of the large-scale target model in the two-path target model, and training the small-scale target model in the two-path target model; Utilizing the trained dual-path target model to perform the target task; Constructing a dual-path target model using the small-scale target model and the large-scale target model includes: Determine the number of network layers N of the small-scale target model and the number of network layers M of the large-scale target model; Dividing the M network layers of the large-scale target model into N groups according to the order of the network layers from front to back, thereby obtaining N network layer groups; On the basis of ensuring the connection relationship between the original internal network layers of the small-scale target model and the large-scale target model, the output of the i-th network layer group of the large-scale target model and the input of the i-1-th network layer of the small-scale target model are added as the input of the i-th network layer of the small-scale target model to obtain the two-path target model; Among them, the inputs of the small-scale target model and the large-scale target model are the same, both of which are the inputs of the dual-path target model. The output of the small-scale target model is used as the output of the dual-path target model. M is greater than N, i is at least 1, and i is at most N.

2. The method according to claim 1, characterized in that The large-scale target model has passed pre-training, and the small-scale target model has not passed the pre-training; or both the large-scale target model and the small-scale target model have passed the pre-training.

3. The method according to claim 1, characterized in that Determine the small-scale target model corresponding to the large-scale target model used to perform the target task, including: Obtaining expected indicators for training the dual-path target model, wherein the expected indicators include: a training time for completing the dual-path target model training under preset conditions, and the accuracy, precision, and recall of the dual-path target model after training; Determining the number of network layers of the large-scale target model; Determining the number of network layers of the small-scale target model according to the expected indicator and the number of network layers of the large-scale target model; According to the number of network layers of the small-scale target model, one version is determined from multiple versions of the target model as the small-scale target model.

4. The method according to claim 1, wherein Training the two-path target model includes: Freezing model parameters of the small-scale target model in the two-path target model, and training the large-scale target model in the two-path target model; Wherein, both the large-scale target model and the small-scale target model have been pre-trained.

5. A model bypass tuning device, characterized in that: include: a determination module configured to determine a small-scale target model corresponding to a large-scale target model for performing a target task, wherein the small-scale target model and the large-scale target model are different versions of the target model, and the model scale of the small-scale target model is smaller than the model scale of the large-scale target model; the target task is a text classification task, and the target model is a text classification model, or the target task is an object detection task, and the target model is an object detection model; A construction module is configured to construct a two-path target model using the small-scale target model and the large-scale target model, wherein the small-scale target model is a side branch network of the two-path target model and the large-scale target model is a main branch network of the two-path target model; A training module is configured to train the two-path target model: freeze the model parameters of the large-scale target model in the two-path target model, and train the small-scale target model in the two-path target model; an execution module, configured to execute the target task using the trained two-path target model; The construction module is specifically configured to: determine the number N of network layers of the small-scale target model and the number M of network layers of the large-scale target model; divide the M network layers of the large-scale target model into N groups in the order of network layers from front to back to obtain N network layer groups; on the basis of ensuring the connection relationship between the original internal network layers of the small-scale target model and the large-scale target model, add the output of the i-th network layer group of the large-scale target model and the input of the i-1-th network layer of the small-scale target model as the input of the i-th network layer of the small-scale target model to obtain the two-way target model; wherein, the inputs of the small-scale target model and the large-scale target model are the same, both are the inputs of the two-way target model, the output of the small-scale target model is used as the output of the two-way target model, M is greater than N, i is a minimum of 1, and i is a maximum of N.

6. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 4 are implemented.

7. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 4 are implemented.

Images