Training method and device of deep neural network based on transfer learning
By readjusting the parameters after training of the image processing network, the problem of too fast fitting of feature extraction networks in deep neural networks is solved, and the feature learning ability and transfer learning accuracy are improved.
Patent Information
- Application Number
- CN202010590593.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2020-06-24
- Publication Date
- 2024-11-05
- Estimated Expiration
- 2040-06-24
AI Technical Summary
In deep neural networks based on transfer learning, the pre-trained feature extraction network is fitted too quickly, resulting in underfitting the feature extraction network, affecting the feature learning ability for the target task.
By re-adjusting the parameters of the image processing network after training, the amplitude of gradient update is increased, so that the feature extraction network is more suitable for the initial parameters during the next training, avoid overfitting, and improve the learning ability of the feature extraction network.
It effectively improves the feature learning ability of the feature extraction network, enhances the adaptability to the target tasks, and improves the accuracy of transfer learning.
Smart Images

Figure CN111783949B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to artificial intelligence technology, and in particular to the fields of transfer learning, deep learning and neural network technology. Background Art
[0002] In recent years, deep learning and transfer learning have achieved great technological breakthroughs and rapid application popularization. Transfer learning can improve the training effect of small-scale sample sets in business with the help of source models fully trained by big data. The deep neural network based on transfer learning includes the migrated feature extraction network and the processing network suitable for the target task.
[0003] When training a deep neural network based on transfer learning, the parameters are fine-tuned for the target task within the entire network. Since the pre-trained feature extraction network itself has a certain transfer ability, it can provide valuable deep features for the target task in the early stage of fine-tuning or even before fine-tuning. At this time, the target network can easily fit the training samples quickly, causing the gradient used to update the parameters to weaken or even disappear from the source of back propagation, and it is impossible to fully update the feature extraction network to make it more suitable for classification tasks.
[0004] In short, the problem of over-fitting of the deep neural network mentioned above leads to under-fitting of the feature extraction network. The main advantage of deep learning is that the deep feature extraction network has a strong feature learning ability. This under-fitting affects the feature learning of the target task by the feature extraction network in the transfer learning task. Summary of the invention
[0005] The embodiments of the present application provide a method, apparatus, device and readable storage medium for training a deep neural network based on transfer learning to train a deep neural network.
[0006] In a first aspect, an embodiment of the present application provides a training method for a deep neural network based on transfer learning, comprising:
[0007] Obtaining a deep neural network to be trained, wherein the deep neural network includes a pre-trained image feature extraction network and an untrained image processing network;
[0008] Training the image feature extraction network and the image processing network;
[0009] During the training process, the trained parameters of the image processing network are readjusted.
[0010] In a second aspect, the embodiment of the present application further provides a training device for a deep neural network based on transfer learning, comprising:
[0011] An acquisition module, used to acquire a deep neural network to be trained, wherein the deep neural network includes a pre-trained image feature extraction network and an untrained image processing network;
[0012] A training module, used for training the image feature extraction network and the image processing network;
[0013] The readjustment module is used to readjust the trained parameters of the image processing network during the training process.
[0014] In a third aspect, an embodiment of the present application provides an electronic device, including:
[0015] at least one processor; and
[0016] a memory communicatively connected to the at least one processor; wherein,
[0017] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute a deep neural network training method based on transfer learning provided in any embodiment.
[0018] In a fourth aspect, an embodiment of the present application provides a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to enable the computer to execute a deep neural network training method based on transfer learning provided in any embodiment.
[0019] The technology of the present application can improve the feature extraction capability of the feature extraction network.
[0020] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS
[0021] The accompanying drawings are used to better understand the present solution and do not constitute a limitation of the present application.
[0022] Figure 1a It is a flowchart of a first method for training a deep neural network based on transfer learning in an embodiment of the present application;
[0023] Figure 1b is a schematic diagram of multiple rounds of training provided in an embodiment of the present application;
[0024] Figure 2a is a flowchart of a second deep neural network training method based on transfer learning in an embodiment of the present application;
[0025] Figure 2bis a schematic diagram of a training cycle provided in an embodiment of the present application;
[0026] Figure 3 is a flowchart of a third deep neural network training method based on transfer learning in an embodiment of the present application;
[0027] Figure 4 is a structural diagram of a deep neural network training device based on transfer learning in an embodiment of the present application;
[0028] Figure 5 It is a structural diagram of an electronic device in an embodiment of the present application. DETAILED DESCRIPTION
[0029] The following is a description of exemplary embodiments of the present application in conjunction with the accompanying drawings, including various details of the embodiments of the present application to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the present application. Similarly, for the sake of clarity and conciseness, the description of well-known functions and structures is omitted in the following description.
[0030] According to an embodiment of the present application, Figure 1a This is a flowchart of the first method for training a deep neural network based on transfer learning in the embodiment of the present application. The embodiment of the present application is applicable to the case of training a deep neural network based on transfer learning. The method is executed by a training device for a deep neural network based on transfer learning, which is implemented by software and / or hardware and is specifically configured in an electronic device with certain data computing capabilities.
[0031] like Figure 1a The training method of the deep neural network based on transfer learning shown in the figure includes:
[0032] S110, obtaining a deep neural network to be trained, where the deep neural network includes a pre-trained image feature extraction network and an untrained image processing network.
[0033] In this embodiment, the image feature extraction network is used to extract deep features from the input image to obtain deep features, which are obtained through transfer learning, including the structure and parameters of the image feature extraction network. The image processing network is used to process the deep features to obtain processing results. The image processing network is determined according to the image processing task, and the image processing task includes but is not limited to image detection tasks, image segmentation tasks, and image classification tasks. Accordingly, the image processing network outputs the image detection results, image segmentation results, and image classification results corresponding to the image processing tasks.
[0034] Specifically, first obtain a model pre-trained using a large data set, such as a ResNet-50 model pre-trained using the ImageNet data set. Among them, the function of the feature extraction network (feature extractor, FE) is general and has migration value; the structure and parameters of the image processing network are adapted to specific image processing tasks and generally do not have migration value. If the image processing network of the pre-trained model is a classification network, it is necessary to replace the classification network of the pre-trained model, that is, the FC layer (Fully connected layer) with a structure adapted to the image processing task of this embodiment, such as the structure of the image segmentation task, and then initialize the image processing network.
[0035] It should be noted that in order to adapt to more scenarios, the deep neural network can also include an input layer and an output layer. The input layer can process the color channel, reduce the dimension, remove noise, etc. of the image, and output the processed image to the image feature extraction layer. The output layer normalizes the output of the image processing network to output the result within the appropriate value range.
[0036] S120, training the image feature extraction network and the image processing network.
[0037] First, an image sample set matching the processing task is constructed, and the image sample set includes multiple image samples with annotations matching the processing task. Taking the image segmentation task as an example, the image sample set includes multiple image samples with segmentation annotations.
[0038] Each image sample is input into the deep neural network, and the processing results are output through the image feature extraction network and the image processing network. The processing results are close to the annotations by optimizing the parameters of the image feature extraction network and the image processing network. The parameters of the image feature extraction network are optimized based on the parameters obtained during pre-training, and the parameters of the image processing network are optimized based on the initialized parameters. Optionally, if the deep neural network includes an input layer and an output layer, the parameters of the input layer and the output layer can be pre-set and not participate in the training; or they can participate in the training after initialization.
[0039] In the field of transfer learning, in order to fully retain the generalization ability of the pre-trained network and prevent overfitting on the target task with a small number of samples, we can use the deep neural network as a starting point, continue to fine-tune the parameters for the image processing task, and constrain the parameters of the feature extraction network to be close to the parameters during pre-training. This method is called the SPAR (starting point as reference) method.
[0040] S130. During the training process, the trained parameters of the image processing network are readjusted.
[0041] According to the training method provided by S120, the parameters of the image processing network are continuously trained (or adjusted), and the parameters after each training are called post-training parameters. The post-training parameters will become the initial values for the next training. In addition to adjusting the parameters during the training process, the post-training parameters of the image processing network are readjusted, and the number of adjustments is at least once. Specifically, the training process includes multiple rounds of training, and the post-training parameters of the image processing network are readjusted between two adjacent rounds of training in the multiple rounds of training.
[0042] Figure 1b It is a schematic diagram of multiple rounds of training provided in an embodiment of the present application. Figure 1b 100 rounds of training are shown, and an adjustment can be made between the 20th and 21st rounds of training, and between the 50th and 51st rounds of training. Optionally, in each adjustment, all or part of the parameters of the image processing network can be adjusted. The adjustment method can be set independently, and it is sufficient to change the trained parameters of the image processing network.
[0043] In this embodiment, the trained parameters of the image processing network are readjusted during the training process, so that the next time training can be performed with the readjusted parameters as the initial values, the amplitude of the gradient update of the feature extraction network is increased, so as to fully fit and deeply explore the feature learning ability of the feature extraction network to better adapt to the target task; at the same time, since the image processing network is easy to fit, readjusting the parameters will not lead to insufficient training of the image processing network, so with the same amount of training time, the method proposed in this embodiment can obtain a higher transfer learning accuracy.
[0044] In the above embodiment and the following embodiment, during the training process, the trained parameters of the image processing network are readjusted, including at least one of the following operations: 1) initializing the trained parameters of the image processing network; 2) adjusting the trained parameters of the image processing network to custom values; 3) adjusting the trained parameters of the image processing network to the parameters of the image processing network at the historical training moment. It should be noted that when multiple readjustments are performed, each adjustment may use different operations as described above, or may use the same operation.
[0045] Specifically, this embodiment does not limit the initialization method, such as the Kaiming initialization method. The parameters of the image processing network at the historical training time refer to the parameters of the image processing network after a certain historical training.
[0046] This embodiment restores the image processing network to the state at the beginning of training by initializing the trained parameters to maximize the amplitude of gradient updates, thereby more fully exploring the feature learning ability of the feature extraction network; by adjusting the trained parameters to custom values, the amplitude of gradient updates is random due to the randomness of the custom values, so that the feature extraction network has stronger flexibility and generalization ability; by adjusting the trained parameters to the parameters of the historical training time, it is equivalent to restoring the image processing network to the historical training time, while improving the training efficiency, appropriately increasing the amplitude of gradient updates, and improving the feature learning ability of the feature extraction network.
[0047] According to an embodiment of the present application, Figure 2a It is a flowchart of the second deep neural network training method based on transfer learning in the embodiment of the present application. The embodiment of the present application optimizes the timing of parameter adjustment based on the technical solutions of the above embodiments.
[0048] Figure 2a The training method of the deep neural network based on transfer learning shown in FIG. 1 specifically includes the following operations:
[0049] S210: Obtain a deep neural network to be trained, where the deep neural network includes a pre-trained image feature extraction network and an untrained image processing network.
[0050] S220: training the image feature extraction network and the image processing network.
[0051] S230: Divide the training process into at least two training cycles.
[0052] S240: between two adjacent training cycles, readjust the trained parameters of the image processing network.
[0053] The training process includes multiple rounds of training, and at least one consecutive round of training constitutes a training cycle. The number of training rounds included in at least two training cycles can be the same or different. The number of training cycles is the number of readjustments plus 1.
[0054] Optional, Figure 2b It is a schematic diagram of the training cycle provided in an embodiment of the present application. Figure 2b A total of 100 rounds of training are included, which are evenly divided into 4 training cycles, namely the first training cycle, the second training cycle, the third training cycle and the fourth training cycle. Between the first training cycle and the second training cycle, the trained parameters of the image processing network obtained after the first training cycle are readjusted, so that the second training cycle continues training based on the readjusted parameters; similarly, between the second training cycle and the third training cycle, and between the third training cycle and the fourth training cycle, the trained parameters of the image processing network are readjusted.
[0055] This embodiment determines the timing of parameter readjustment by dividing the training cycle, so that when a new training cycle arrives, training can be continued with the readjusted parameters as initial values; by adjusting the number of training rounds included in the training cycle, the timing and number of parameter readjustments can be conveniently adjusted; when the training process is evenly divided into at least two training cycles, regular readjustment of parameters can be achieved, further improving the feature learning ability of the feature extraction network.
[0056] According to an embodiment of the present application, Figure 3 It is a flowchart of the third deep neural network training method based on transfer learning in the embodiments of the present application. Based on the above embodiments, this embodiment optimizes the training process.
[0057] like Figure 3 The training method of the deep neural network based on transfer learning shown in the figure includes:
[0058] S310: Obtain a deep neural network to be trained, where the deep neural network includes a pre-trained image feature extraction network and an untrained image processing network.
[0059] S320, using a back propagation algorithm to train the image feature extraction network and the image processing network.
[0060] S330: During the training process, readjust the trained parameters of the image processing network.
[0061] In a specific application scenario, assume that the deep neural network is trained for a total of M rounds, that is, the image samples are trained for M rounds, M is a natural number. Set a readjustment number R, which is generally 2, 3 or 4, then each training cycle is T = M / R.
[0062] The overall process of network training is as follows: In each round of training, the image samples are input into the deep neural network and the output of each image sample is calculated. The loss value, such as the cross entropy loss, is calculated based on the output and the sample label. Then, the gradient of each parameter is calculated based on the loss value, and the parameters are updated by back propagation until each image sample has been trained for M rounds.
[0063] In the overall process of the above network training, after each training cycle, that is, after the 1st, 2nd, ..., M / R training cycles, all parameters of the image processing layer are reinitialized, while the feature extraction layer remains unchanged; so that when the next training cycle comes, the training continues with the initialized image processing layer and the feature extraction layer obtained in the previous training cycle as the starting point.
[0064] This embodiment readjusts the parameters after training to increase the amplitude of gradient update; and the back propagation algorithm is essentially a gradient descent method. It can be seen that this embodiment can increase the space for gradient descent, avoid the phenomenon of training "pause" when the objective function is complex, and give full play to the optimization ability of the back propagation algorithm for complex objective functions.
[0065] According to an embodiment of the present application, Figure 4 It is a structural diagram of a training device for a deep neural network based on transfer learning in an embodiment of the present application. The embodiment of the present application is suitable for training a deep neural network based on transfer learning. The device is implemented using software and / or hardware and is specifically configured in an electronic device with certain data computing capabilities.
[0066] like Figure 4 A training device 400 of a deep neural network based on transfer learning is shown, comprising: an acquisition module 401, a training module 402 and a readjustment module 403; wherein,
[0067] The acquisition module 401 is used to acquire a deep neural network to be trained, where the deep neural network includes a pre-trained image feature extraction network and an untrained image processing network.
[0068] The training module 402 is used to train the image feature extraction network and the image processing network.
[0069] The readjustment module 403 is used to readjust the trained parameters of the image processing network during the training process.
[0070] In this embodiment, the trained parameters of the image processing network are readjusted during the training process, so that the next time training can be performed with the readjusted parameters as the initial values, the amplitude of the gradient update of the feature extraction network is increased, so as to fully fit and deeply explore the feature learning ability of the feature extraction network to better adapt to the target task; at the same time, since the image processing network is easy to fit, readjusting the parameters will not lead to insufficient training of the image processing network, so with the same amount of training time, the method proposed in this embodiment can obtain a higher transfer learning accuracy.
[0071] Furthermore, the readjustment module 403 includes at least one of the following units: an initialization unit, used to initialize the post-trained parameters of the image processing network; a customization unit, used to adjust the post-trained parameters of the image processing network to custom values; and a historical moment unit, used to adjust the post-trained parameters of the image processing network to the parameters of the image processing network at a historical training moment.
[0072] Furthermore, the readjustment module 403 includes: a division unit, used to divide the training process into at least two training cycles; and a readjustment unit, used to readjust the trained parameters of the image processing network between two adjacent training cycles.
[0073] Furthermore, the training module 402 is specifically used to train the image feature extraction network and the image processing network using a back propagation algorithm.
[0074] The above-mentioned deep neural network training device based on transfer learning can execute the deep neural network training method based on transfer learning provided in any embodiment of the present application, and has the corresponding functional modules and beneficial effects for executing the deep neural network training method based on transfer learning.
[0075] According to an embodiment of the present application, the present application also provides an electronic device and a readable storage medium.
[0076] like Figure 5 , is a block diagram of an electronic device for implementing a training method of a deep neural network based on transfer learning of an embodiment of the present application. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workbenches, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present application described and / or required herein.
[0077] like Figure 5 As shown, the electronic device includes: one or more processors 501, a memory 502, and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. The various components are connected to each other using different buses, and can be installed on a common mainboard or installed in other ways as needed. The processor can process instructions executed in the electronic device, including instructions stored in or on the memory to display graphical information of the GUI on an external input / output device (such as a display device coupled to the interface). In other embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories and multiple memories. Similarly, multiple electronic devices can be connected, and each terminal provides some necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system). Figure 5 A processor 501 is taken as an example.
[0078] The memory 502 is a non-transient computer-readable storage medium provided in the present application. The memory stores instructions executable by at least one processor to enable at least one processor to execute the training method of a deep neural network based on transfer learning provided in the present application. The non-transient computer-readable storage medium of the present application stores computer instructions, which are used to enable a computer to execute the training method of a deep neural network based on transfer learning provided in the present application.
[0079] The memory 502 is a non-transient computer-readable storage medium that can be used to store non-transient software programs, non-transient computer executable programs and modules, such as the program instructions / modules corresponding to the training method of the deep neural network based on transfer learning in the embodiment of the present application (for example, the attached Figure 4 The processor 501 executes various functional applications and data processing of the server by running the non-transient software programs, instructions and modules stored in the memory 502, that is, the training method of the deep neural network based on transfer learning in the above method embodiment is implemented.
[0080] The memory 502 may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created by the use of an electronic device that implements a training method for a deep neural network based on transfer learning, etc. In addition, the memory 502 may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory 502 may optionally include a memory remotely disposed relative to the processor 501, and these remote memories may be connected to an electronic device that executes a training method for a deep neural network based on transfer learning via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.
[0081] The electronic device for executing the training method of the deep neural network based on transfer learning may further include: an input device 503 and an output device 504. The processor 501, the memory 502, the input device 503 and the output device 504 may be connected via a bus or other means. Figure 5 The example of connecting through bus is taken in the following.
[0082] The input device 503 can receive input digital or character information, and generate key signal input related to user settings and function control of the electronic device that performs the training method of the deep neural network based on transfer learning, such as a touch screen, a keypad, a mouse, a trackpad, a touchpad, an indicator rod, one or more mouse buttons, a trackball, a joystick and other input devices. The output device 504 may include a display device, an auxiliary lighting device (e.g., an LED) and a tactile feedback device (e.g., a vibration motor), etc. The display device may include, but is not limited to, a liquid crystal display (LCD), a light emitting diode (LED) display and a plasma display. In some embodiments, the display device may be a touch screen.
[0083] Various implementations of the systems and techniques described herein can be realized in digital electronic circuit systems, integrated circuit systems, dedicated ASICs (application specific integrated circuits), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.
[0084] These computer programs (also referred to as programs, software, software applications, or code) include machine instructions for programmable processors and can be implemented using high-level procedural and / or object-oriented programming languages, and / or assembly / machine languages. As used herein, the terms "machine-readable medium" and "computer-readable medium" refer to any computer program product, device, and / or means (e.g., disk, optical disk, memory, programmable logic device (PLD)) for providing machine instructions and / or data to a programmable processor, including a machine-readable medium that receives machine instructions as a machine-readable signal. The term "machine-readable signal" refers to any signal for providing machine instructions and / or data to a programmable processor.
[0085] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).
[0086] The systems and techniques described herein can be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), the Internet, and a blockchain network.
[0087] A computer system may include clients and servers. Clients and servers are generally remote from each other and usually interact through a communication network. The relationship of client and server is generated by computer programs running on respective computers and having a client-server relationship to each other.
[0088] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this application can be executed in parallel, sequentially or in different orders, as long as the expected results of the technical solution disclosed in this application can be achieved, and this document is not limited here.
[0089] The above specific implementations do not constitute a limitation on the protection scope of this application. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions and improvements made within the spirit and principles of this application should be included in the protection scope of this application.
Claims
1. A training method for a deep neural network based on transfer learning, comprising: Obtaining a deep neural network to be trained, wherein the deep neural network includes a pre-trained image feature extraction network and an untrained image processing network; Training the image feature extraction network and the image processing network; Divide the training process into at least two training cycles; Between two adjacent training cycles, the trained parameters of the image processing network are readjusted; wherein the trained parameters are parameters after each training, and the trained parameters are used as initial values for the next training; the readjustment is an adjustment other than the parameter adjustment during the training process; During the training process, the trained parameters of the image processing network are readjusted, including at least one of the following operations: Initializing the trained parameters of the image processing network; Adjusting the trained parameters of the image processing network to custom values; The trained parameters of the image processing network are adjusted to the parameters of the image processing network at the historical training moment.
2. The method according to claim 1, wherein: The training of the image feature extraction network and the image processing network comprises: The image feature extraction network and the image processing network are trained using a back propagation algorithm.
3. A training device for a deep neural network based on transfer learning, comprising: An acquisition module, used to acquire a deep neural network to be trained, wherein the deep neural network includes a pre-trained image feature extraction network and an untrained image processing network; A training module, used for training the image feature extraction network and the image processing network; Re-adjusted modules, including: A division unit, used for dividing the training process into at least two training cycles; A readjustment unit, used to readjust the trained parameters of the image processing network between two adjacent training cycles; wherein the trained parameters are parameters after each training, and the trained parameters are used as initial values for the next training; and the readjustment is an adjustment other than the parameter adjustment during the training process; The readjustment module includes at least one of the following units: An initialization unit, used to initialize the trained parameters of the image processing network; A custom unit, used to adjust the trained parameters of the image processing network to custom values; The historical moment unit is used to adjust the trained parameters of the image processing network to the parameters of the image processing network at the historical training moment.
4. The device according to claim 3, wherein: The training module is specifically used to train the image feature extraction network and the image processing network using a back propagation algorithm.
5. An electronic device comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute a deep neural network training method based on transfer learning as described in any one of claims 1-2.
6. A non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to enable the computer to execute a deep neural network training method based on transfer learning as described in any one of claims 1-2.
