Methods, apparatus, electronic devices and storage media for neural network training

Abstract

This disclosure provides a method, apparatus, electronic device, and storage medium for training a neural network. The method includes: acquiring training data and multiple trained target neural networks; extracting features from the training data using the multiple target neural networks respectively, obtaining feature information output by the feature extraction layers of each target neural network; determining the loss function value of a joint neural network to be trained based on the feature information output by the feature extraction layers of each target neural network, wherein the joint neural network consists of multiple target neural networks and connection layers between the feature extraction layers of each target neural network; and performing at least one round of network training on the joint neural network to be trained based on the loss function value to obtain a trained joint neural network. This disclosure not only considers the characteristics of different tasks but also utilizes connection layers to achieve feature exchange between tasks, improving the versatility of the trained joint neural network.

Description

Technical Field

[0001] This disclosure relates to the field of artificial intelligence technology, and more specifically, to a method, apparatus, electronic device, and storage medium for training a neural network. Background Technology

[0002] With the rapid development of artificial intelligence technology, end-to-end deep learning technology is becoming increasingly mature. In most cases, a neural network needs to be learned for each task. For example, an image classification neural network can be learned for an image classification task, and an object detection neural network can be learned for an object detection task.

[0003] To achieve better generalization performance through joint task learning, a multi-task learning method is provided in related technologies. This method can learn a pre-trained neural network for various tasks upstream, and the neural network can directly share the pre-trained early weights when facing different tasks downstream.

[0004] Since different tasks usually have different learning directions, there are significant heterogeneous conflicts between tasks. These heterogeneous issues will interfere with the overall target direction of multi-task learning, resulting in poor multi-task learning performance. Summary of the Invention

[0005] This disclosure provides at least one method, apparatus, electronic device, and storage medium for neural network training to improve the training performance of multi-task learning.

[0006] In a first aspect, embodiments of this disclosure provide a method for training a neural network, the method comprising:

[0007] Acquire training data and multiple target neural networks trained therefrom; the training data includes multiple image samples; the multiple target neural networks are used to perform different target tasks respectively;

[0008] The training data is extracted using the multiple target neural networks respectively, and the feature information output by the feature extraction layer of each target neural network is obtained.

[0009] Based on the feature information output by the feature extraction layers of each of the target neural networks, the loss function value of the joint neural network to be trained is determined, wherein the joint neural network is composed of the plurality of target neural networks and the connection layers between the feature extraction layers of each of the target neural networks;

[0010] Based on the loss function value, perform at least one round of network training on the joint neural network to be trained to obtain a trained joint neural network.

[0011] Using the aforementioned neural network training method, given training data and multiple trained target neural networks, features can first be extracted from the training data using these target neural networks designed to perform different target tasks. Based on the feature information output by the feature extraction layers of each target neural network, the loss function value of the joint neural network to be trained is determined. Then, the joint neural network is trained based on the loss function value. In the process of learning the joint neural network, this embodiment not only considers the characteristics of different tasks but also utilizes connection layers to achieve feature exchange between different tasks. This can improve the versatility of the trained joint neural network to a certain extent, thereby facilitating related downstream applications.

[0012] In one possible implementation, one of the plurality of target neural networks serves as the main neural network of the joint neural network, and the other target neural networks among the plurality of target neural networks serve as the sub-neural networks of the joint neural network; determining the loss function value of the joint neural network to be trained based on the feature information output by the feature extraction layers included in each of the target neural networks includes:

[0013] Based on the first feature information output by the first feature extraction layer of the sub-neural network, the second feature information output by the second feature extraction layer of the main neural network is updated to obtain the updated second feature information.

[0014] Based on the updated second feature information, the loss function value of the joint neural network to be trained is determined.

[0015] Here, we consider using the feature assistance of the secondary neural network to the main neural network to train the joint neural network. The resulting joint neural network will have stronger task adaptability, that is, it can be better applied to various business scenarios.

[0016] In one possible implementation, the main neural network includes a task layer; determining the loss function value of the joint neural network to be trained based on the updated second feature information includes:

[0017] The updated second feature information is input into the task layer of the main neural network to determine the task output result of the main neural network.

[0018] The loss function value of the main neural network is determined based on the difference between the task output and the task labeling result of the corresponding input training data.

[0019] The loss function value of the main neural network is determined as the loss function value of the joint neural network to be trained.

[0020] Here, the loss function value of the joint neural network to be trained can be determined based on the difference between the task output of the main neural network and the task labeling result of the corresponding input training data. It can be seen that the loss function value determined here starts from the main neural network layer. More feature information can be mined through the updated second feature information. Furthermore, the task characteristics can be more accurately mined based on the matching between the two results (i.e., the task output result and the task labeling result), thereby improving the performance of network training.

[0021] In one possible implementation, the sub-neural network includes multiple first feature extraction layers with different output sizes, and the main neural network includes multiple second feature extraction layers with different output sizes; updating the second feature information output by the second feature extraction layers of the main neural network based on the first feature information output by the first feature extraction layers of the sub-neural network includes:

[0022] For the target second feature extraction layer among the multiple second feature extraction layers included in the main neural network, the second feature information output by the target second feature extraction layer is updated based on the first feature information output by at least one first feature extraction layer included in the sub-neural network, so as to obtain the updated second feature information for the target second feature extraction layer.

[0023] Here, during the process of updating the second feature information, the corresponding first feature information can be selected based on the specific network structure of the main neural network and the sub-neural network, so as to mine richer fusion features through more diverse feature selection schemes.

[0024] In one possible implementation, updating the second feature information output by the target second feature extraction layer based on the first feature information output by at least one first feature extraction layer included in the sub-neural network includes:

[0025] For a target second feature extraction layer among the multiple second feature extraction layers included in the main neural network, a first feature extraction layer whose output size matches that of the target second feature extraction layer is selected from the multiple first feature extraction layers included in the sub-neural network; and,

[0026] Based on the selected first feature information output by the first feature extraction layer, the second feature information output by the target second feature extraction layer is updated to obtain the updated second feature information for the target second feature extraction layer.

[0027] Here, the first feature extraction layer can be selected at the level of output size matching, thereby achieving feature fusion at the same or similar size to achieve more accurate network training.

[0028] In one possible implementation, updating the second feature information output by the target second feature extraction layer based on a selected first feature information output by the first feature extraction layer includes:

[0029] Based on a selected connection layer between the first feature extraction layer and the target second feature extraction layer, the first feature information output by the selected first feature extraction layer is fused with the second feature information output by the target second feature extraction layer to obtain the updated second feature information for the target second feature extraction layer.

[0030] In one possible implementation, where a selected first feature extraction layer comprises multiple feature extraction blocks, and each feature extraction block has a connection layer with the target second feature extraction layer; the fusion of the first feature information output by the selected first feature extraction layer and the second feature information output by the target second feature extraction layer includes:

[0031] The first feature information output by each feature extraction block of the selected first feature extraction layer is fused with the second feature information output by the target second feature extraction layer to obtain the updated second feature information for the target second feature extraction layer.

[0032] In one possible implementation, updating the second feature information output by the target second feature extraction layer based on the first feature information output by at least one first feature extraction layer included in the sub-neural network includes:

[0033] For a target second feature extraction layer among the multiple second feature extraction layers included in the main neural network, multiple first feature extraction layers corresponding to the target second feature extraction layer are selected from the multiple first feature extraction layers included in the sub-neural network; and,

[0034] The second feature information output by the second feature extraction layer of the target is updated based on the first feature information selected from multiple first feature extraction layers.

[0035] Here, the selection of the first feature extraction layer can be made at the level of output size diversity, thereby realizing feature fusion under multiple sizes and improving the network training performance under target tasks at different scales.

[0036] In one possible implementation, after obtaining the trained joint neural network, the method further includes:

[0037] The training data is used to extract features using a distillation neural network to be trained, thereby obtaining the feature information output by the feature extraction layer of the distillation neural network.

[0038] Based on the feature similarity between the feature information output by the feature extraction layer of the distillation neural network and the feature information output by the feature extraction layer of each target neural network in the joint neural network, the loss function value of the distillation neural network to be trained is determined.

[0039] Based on the loss function value, at least one round of network training is performed on the distillation neural network to be trained to obtain a trained distillation neural network.

[0040] Here, to facilitate the application of downstream tasks, a corresponding distillation neural network can be trained for the joint neural network. This distillation neural network distills the feature knowledge learned from the relatively complex joint neural network based on the feature similarity between the two networks (i.e., the distillation neural network and the target neural network), making it more lightweight while ensuring network accuracy.

[0041] In one possible implementation, after obtaining the trained joint neural network, the method further includes:

[0042] The test data collected under the target task and the pruning weight mask constructed for the target task are obtained, wherein the pruning weight mask is used to characterize whether each computing unit in the joint neural network participates in the network training of the target task.

[0043] The collected test data is input into the trained joint neural network to determine the loss function value of the joint neural network;

[0044] Repeat the following steps until the loss function value is less than a preset threshold:

[0045] The pruning weight mask is adjusted based on the loss function value, and the joint neural network is pruned based on the adjusted pruning weight mask.

[0046] The collected test data is input into the pruned joint neural network for the next round of network training.

[0047] Here, during the joint neural network process, a pruning weight mask constructed for the target task can be learned simultaneously. Based on the pruning weight mask, corresponding pruning operations can be performed on the joint neural network, which makes the pruning operation more targeted, thereby ensuring that the consumption of network resources is greatly reduced without significantly reducing the network accuracy.

[0048] In one possible implementation, the target neural network includes a feature extraction layer for feature extraction and a task layer for task processing; the target neural network is trained according to the following steps:

[0049] Select training data that matches the current task from the acquired training data;

[0050] The selected training data is input into the feature extraction layer of the target neural network to be trained, and the feature information output by the feature extraction layer is obtained.

[0051] The feature information output by the feature extraction layer is input into the task layer of the target neural network to be trained, and the task output result of the task layer is obtained.

[0052] If the task output and the task labeling results for the selected training data do not match, the target neural network to be trained is trained again until the task output and the task labeling results for the selected training data match.

[0053] Here, we can learn common features within a target task based on the feature extraction layer, and we can also learn some task-specific features based on the task layer, ensuring the accuracy of the trained target neural network in applying the target task.

[0054] Secondly, embodiments of this disclosure also provide an apparatus for training a neural network, the apparatus comprising:

[0055] An acquisition module is used to acquire training data and multiple trained target neural networks; the training data includes multiple image samples; the multiple target neural networks are used to perform different target tasks.

[0056] The extraction module is used to extract features from the training data using the multiple target neural networks respectively, and to obtain the feature information output by the feature extraction layer of each target neural network.

[0057] A generation module is used to determine the loss function value of the joint neural network to be trained based on the feature information output by the feature extraction layers of each of the target neural networks, wherein the joint neural network is composed of the plurality of target neural networks and the connection layers between the feature extraction layers of each of the target neural networks;

[0058] The training module is used to perform at least one round of network training on the joint neural network to be trained based on the loss function value, so as to obtain a trained joint neural network.

[0059] Thirdly, embodiments of this disclosure also provide an electronic device, including: a processor, a memory, and a bus, wherein the memory stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor communicates with the memory via the bus, and when the machine-readable instructions are executed by the processor, they perform the steps of the neural network training method as described in the first aspect and any of its various embodiments.

[0060] Fourthly, embodiments of this disclosure also provide a computer-readable storage medium storing a computer program that, when executed by a processor, performs the steps of the neural network training method as described in the first aspect and its various embodiments.

[0061] For a description of the effects of the aforementioned neural network training apparatus, electronic equipment, and computer-readable storage medium, please refer to the description of the aforementioned neural network training method; these details will not be repeated here.

[0062] To make the above-mentioned objects, features and advantages of this disclosure more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings. Attached Figure Description

[0063] To more clearly illustrate the technical solutions of the embodiments of this disclosure, the accompanying drawings used in the embodiments will be briefly described below. These drawings are incorporated in and constitute a part of this specification. They illustrate embodiments conforming to this disclosure and, together with the specification, serve to explain the technical solutions of this disclosure. It should be understood that the following drawings only show some embodiments of this disclosure and should not be considered as limiting the scope. Those skilled in the art can obtain other related drawings based on these drawings without creative effort.

[0064] Figure 1 A flowchart of a neural network training method provided by an embodiment of this disclosure is shown;

[0065] Figure 2The flowchart illustrates a specific method for joint neural network training in the neural network training method provided in this embodiment of the present disclosure.

[0066] Figure 3(a) shows a schematic diagram of the connection method of the first connection layer in the neural network training method provided in the embodiments of this disclosure;

[0067] Figure 3(b) shows a schematic diagram of the connection method of the second connection layer in the neural network training method provided in the embodiments of this disclosure;

[0068] Figure 3(c) shows a schematic diagram of the connection method of the third connection layer in the neural network training method provided in the embodiments of this disclosure;

[0069] Figure 4 A schematic diagram of a neural network training apparatus provided in an embodiment of this disclosure is shown;

[0070] Figure 5 A schematic diagram of an electronic device provided in an embodiment of the present disclosure is shown. Detailed Implementation

[0071] To make the objectives, technical solutions, and advantages of the embodiments of this disclosure clearer, the technical solutions of the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this disclosure, and not all of them. The components of the embodiments of this disclosure described and shown in the accompanying drawings can generally be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of this disclosure provided in the accompanying drawings is not intended to limit the scope of the claimed disclosure, but merely represents selected embodiments of this disclosure. All other embodiments obtained by those skilled in the art based on the embodiments of this disclosure without inventive effort are within the scope of protection of this disclosure.

[0072] It should be noted that similar labels and letters in the following figures indicate similar items. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures.

[0073] In this document, the term "and / or" merely describes a relationship, indicating that three relationships can exist. For example, A and / or B can represent three cases: A alone, A and B simultaneously, and B alone. Furthermore, the term "at least one" in this document means any combination of at least two of any one or more elements. For example, including at least one of A, B, and C can mean including any one or more elements selected from the set consisting of A, B, and C.

[0074] Research has found that, in order to achieve better generalization performance through joint task learning, a multi-task learning method is provided in related technologies. This method can learn a pre-trained neural network for various tasks upstream, and the neural network can directly share the pre-trained early weights when facing various different tasks downstream.

[0075] Since different tasks usually have different learning directions, there are significant heterogeneous conflicts between tasks. These heterogeneous issues will interfere with the overall target direction of multi-task learning, resulting in poor multi-task learning performance.

[0076] Based on the above research, this disclosure provides a method, apparatus, electronic device, and storage medium for neural network training to improve the training performance of multi-task learning.

[0077] To facilitate understanding of this embodiment, a method for training a neural network disclosed in this disclosure will first be described in detail. The execution entity of the neural network training method provided in this disclosure is generally an electronic device with a certain computing capability. This electronic device may include, for example, a terminal device, a server, or other processing devices. The terminal device may be a user equipment (UE), mobile device, cellular phone, cordless phone, personal digital assistant (PDA), handheld device, computing device, in-vehicle device, wearable device, etc. In some possible implementations, this neural network training method can be implemented by a processor calling computer-readable instructions stored in memory.

[0078] See Figure 1 The diagram shows a flowchart of a neural network training method provided in an embodiment of this disclosure. The method includes steps S101 to S104, wherein:

[0079] S101: Acquire training data and multiple target neural networks trained from it; the training data includes multiple image samples; the multiple target neural networks are used to perform different target tasks.

[0080] S102: Use multiple target neural networks to extract features from the training data respectively, and obtain the feature information output by the feature extraction layer of each target neural network;

[0081] S103: Based on the feature information output by the feature extraction layers of each target neural network, determine the loss function value of the joint neural network to be trained, wherein the joint neural network consists of multiple target neural networks and connection layers between the feature extraction layers of each target neural network.

[0082] S104: Perform at least one round of network training on the joint neural network to be trained based on the loss function value to obtain a trained joint neural network.

[0083] To facilitate understanding of the neural network training method provided in this disclosure, the application scenarios of this method will be briefly described first. The neural network training method in this disclosure can mainly be applied to the field of vision. For example, a well-trained joint neural network can be applied to tasks such as object detection, image classification, and depth estimation.

[0084] Multi-task learning methods in related technologies can learn a pre-trained neural network upstream for various tasks, allowing this network to directly share pre-trained early weights when facing different downstream tasks. However, since different tasks typically have different learning directions, this leads to significant heterogeneous conflicts between tasks. These heterogeneous issues interfere with the overall target direction of multi-task learning, resulting in poor performance. Furthermore, the performance of multi-task learning largely depends on the existing task dictionary; the dictionary size, task type, label source, etc., all affect the final result, further negatively impacting performance.

[0085] To address the aforementioned issues, this disclosure provides a scheme for joint training of neural networks based on the connection layers between the feature extraction layers of each target neural network, thereby improving the performance of multi-task learning.

[0086] Here, the loss function value of the joint neural network to be trained can be determined by combining the feature information output by the feature extraction layers of each target neural network. Since there are connection layers between the feature extraction layers of each target neural network, the feature information output by the feature extraction layers of each target neural network can be fused using these connection layers, thereby enabling the trained joint neural network to have the task characteristics of each target neural network.

[0087] Each target neural network in this embodiment can be used to perform different target tasks. Taking a network with three target neural networks as an example, these three target neural networks may include a detection neural network for performing a target detection task, a classification neural network for performing a target image classification task, and a segmentation neural network for performing a target semantic segmentation task.

[0088] Before using multiple target neural networks to extract feature information, multiple target neural networks can be trained first. Before training the target neural network, appropriate training data can be selected from the acquired training data based on the task characteristics of the target task corresponding to the target neural network. For example, the corresponding detection neural network can be trained based on the training data of the target detection task, and the corresponding classification neural network can be trained based on the training data of the target image classification task.

[0089] The training data in this embodiment can mainly be image samples collected in the field of computer vision. These image samples can be labeled samples with task annotation results, and the task annotations of the image samples used by different target neural networks are also different. Some image samples can be information about the location, size, etc. of the target object labeled for the target detection task, while some image samples can be semantic information of the object labeled for the target semantic segmentation task.

[0090] Here, different target neural networks can be trained using corresponding training data, resulting in multiple trained target neural networks for performing different target tasks. These target neural networks can include feature extraction layers for feature extraction and task layers for task processing. Specifically, the target neural networks can be trained using the following steps:

[0091] Step 1: Select training data that matches the current task from the acquired training data;

[0092] Step 2: Input the selected training data into the feature extraction layer of the target neural network to be trained, and obtain the feature information output by the feature extraction layer;

[0093] Step 3: Input the feature information output by the feature extraction layer into the task layer of the target neural network to be trained, and obtain the task output result of the task layer;

[0094] Step 4: If the task output and the task labeling results for the selected training data do not match, retrain the target neural network until the task output and the task labeling results for the selected training data match.

[0095] Here, features can be extracted based on the training data matched to the current task, and then through the feature extraction layer included in the target neural network. When the feature information output by the feature extraction layer is input into the task layer included in the target neural network to be trained, multiple rounds of training of the target neural network can be performed based on the task output results and the matching results of the task labeling results for the selected training data.

[0096] In this embodiment of the disclosure, if the task output result and the task labeling result for the selected training data do not match, it indicates that the current network performance is poor and the network parameter values ​​need to be adjusted for the next round of training until the two results match.

[0097] To achieve joint training of multiple target neural networks, we first use each pre-trained target neural network to extract features from the training data, obtaining the feature information output by the feature extraction layer of each target neural network. Then, based on the feature information output by the feature extraction layers of each target neural network, we determine the loss function value of the joint neural network to be trained. Since the feature extraction layers of each target neural network are connected, feature exchange between them is possible. Therefore, the determined loss function value is a loss function value that reflects the fused features, resulting in better performance of the trained joint neural network.

[0098] In practical applications, one of the target neural networks in a joint neural network can serve as the main neural network, while the other target neural networks can serve as the sub-neural networks. Here, the loss function value of the joint neural network to be trained can be determined by following these steps:

[0099] Step 1: Update the second feature information output by the second feature extraction layer of the main neural network based on the first feature information output by the first feature extraction layer of the sub-neural network, to obtain the updated second feature information;

[0100] Step 2: Input the updated second feature information into the task layer of the main neural network to obtain the task output of the main neural network;

[0101] Step 3: Based on the task output and the task labeling results of the corresponding input training data, determine the loss function value of the main neural network;

[0102] Step 4: Determine the loss function value of the main neural network as the loss function value of the joint neural network to be trained.

[0103] Here, the loss function value of the joint neural network can be determined based on the comparison between the task output result determined by the updated second feature information and the corresponding task annotation result.

[0104] In practical applications, the main neural network and the sub-neural network can correspond to heterogeneous tasks, such as object detection and object classification tasks, respectively.

[0105] The main neural network can be trained using homogeneous data, which refers to data that performs similar tasks. For example, the main neural network can be used to perform object detection task 1 for pedestrian detection and object detection task 2 for vehicle detection. Similarly, the secondary neural network can also be trained using homogeneous data. For example, the secondary neural network can be used to perform object classification tasks 1 and 2 for image classification.

[0106] In this context, similar tasks can share the same few hidden layers of the network, but the network branches off near the output layer to perform different tasks, such as object detection task 1 and object detection task 2 mentioned above. Different tasks (i.e., heterogeneous tasks) learn some common low-level features by sharing a few hidden layers at the bottom of the network, and the parameters shared at the bottom layer can be exactly the same.

[0107] Furthermore, considering the characteristics of each target task, a task-specific layer can be designed for each task to learn higher-level abstract features. All target tasks can share some relevant hidden layers while retaining their task-specific output layers.

[0108] In the training of joint neural networks for heterogeneous multi-task learning, multiple tasks (such as classification and detection) have feature extraction layers with the same size as the parameter space. Here, the feature information output by the feature extraction layer of the sub-neural network corresponding to the sub-task can be fused with the feature extraction layer output by the feature extraction layer of the main neural network corresponding to the main task.

[0109] In the above feature fusion process, embodiments of this disclosure introduce a connection layer T to facilitate feature exchange between various neural networks. For example... Figure 2 The diagram illustrates the joint training of two target neural networks according to an embodiment of this disclosure. The left branch corresponds to the main neural network, and the right branch corresponds to the sub-neural network. The connection layer T connects the main neural network and the sub-neural network. Stage corresponds to the feature extraction layer of the neural network, and Head corresponds to the task-related task layer. For example, Head1 and Head2 correspond to target detection task 1 and target detection task 2, respectively, and Head3 and Head4 correspond to target classification task 1 and target classification task 2, respectively.

[0110] When training data is input into the main neural network and the sub-neural network, the stages included in the main neural network can extract the corresponding second feature information, and the stages included in the sub-neural network can extract the corresponding first feature information. The fusion between the first feature information and the second feature information can be achieved through the connection layer T.

[0111] In practical applications, the aforementioned sub-neural network may include multiple first feature extraction layers with different output sizes, and the main neural network may include multiple second feature extraction layers with different output sizes. The aforementioned connection layer T can specifically realize the link between the first feature extraction layers and the second feature extraction layers.

[0112] In this embodiment of the disclosure, the target second feature extraction layer can be updated based on the first feature information output by at least one first feature extraction layer of the sub-neural network, thereby obtaining the updated second feature information for the target second feature extraction layer.

[0113] The aforementioned target second feature extraction layer may refer to any randomly selected or designated second feature extraction layer among a plurality of second feature extraction layers. That is, the present invention may perform feature fusion on only one second feature extraction layer. In addition, the aforementioned target second feature extraction layer may also refer to some or all of the second feature extraction layers among a plurality of second feature extraction layers. That is, the present invention may perform feature fusion on a plurality of second feature extraction layers.

[0114] The placement of the connection layer T in this embodiment can be varied. This results in a variety of feature fusion schemes. These can be further explained in detail from the following aspects:

[0115] Firstly, regarding the target second feature extraction layer included in the neural network, one first feature extraction layer that matches the output size of the target second feature extraction layer can be selected from the multiple first feature extraction layers included in the sub-neural network, and the second feature information output by the target second feature extraction layer can be updated based on this selected first feature extraction layer.

[0116] In practical applications, the fusion of two feature information can be achieved by using a connection layer between a selected first feature extraction layer and a target second feature extraction layer.

[0117] Furthermore, in the case where a selected first feature extraction layer includes multiple feature extraction blocks, and each feature extraction block has a connection layer with the target second feature extraction layer, feature fusion is achieved by fusing the first feature information output by each feature extraction block of the selected first feature extraction layer with the second feature information output by the target second feature extraction layer.

[0118] It should be noted that, in the embodiments of this disclosure, output size matching may refer to the output sizes of the two feature extraction layers being completely identical, or the output sizes being not significantly different.

[0119] Secondly, regarding the target second feature extraction layer included in the neural network, multiple first feature extraction layers corresponding to the target second feature extraction layer can be selected from the multiple first feature extraction layers included in the sub-neural network, and then feature fusion is performed based on the selected multiple first feature extraction layers.

[0120] The selected first feature extraction layers can be first feature extraction layers with the same or similar output size as the target second feature extraction layer, or they can be first feature extraction layers with a certain degree of correlation with the target second feature extraction layer. There is no specific limit to the number of first feature extraction layers selected.

[0121] To better understand the placement of the aforementioned connecting layer T, we can... Figures 3(a) to 3(c) The three connection methods shown will be explained in detail.

[0122] Figure 3(a) shows that features can be placed in the middle of a specific layer for feature transfer in a feature extraction layer of the same size. Figure 3(b) shows feature transfer between different sizes. Figure 3(c) shows fusion of blocks in different feature extraction layers of the same size.

[0123] The neural network training method provided in this disclosure facilitates feature exchange between various target neural networks by introducing a connection layer T. By introducing the concept of a transfer layer, different target neural networks are divided into main branches (corresponding to the main task) and auxiliary branches (corresponding to the secondary task) according to different tasks. The purpose of the connection layer is to learn new features to perform inference in the main branch with auxiliary branches, thus organizing the main neural network E into a joint neural network G. The calculation formula for the joint neural network is: G = f(E, T).

[0124] In this embodiment of the disclosure, after training the joint neural network, network compression can be achieved through knowledge distillation. Specifically, this can be achieved through the following steps:

[0125] Step 1: Use the distillation neural network to be trained to extract features from the training data, and obtain the feature information output by the feature extraction layer of the distillation neural network.

[0126] Step 2: Based on the feature similarity between the feature information output by the feature extraction layer of the distillation neural network and the feature information output by the feature extraction layer of each target neural network in the joint neural network, determine the loss function value of the distillation neural network to be trained;

[0127] Step 3: Based on the loss function value, perform at least one round of network training on the distillation neural network to be trained to obtain the trained distillation neural network.

[0128] Here, the distillation neural network can be trained based on the feature similarity between the feature information output by the feature extraction layers of the distillation neural network and the feature information output by the feature extraction layers of each target neural network in the joint neural network. A sufficiently high feature similarity indicates that the characteristics of the distillation neural network closely approximate those of the joint neural network; conversely, a low feature similarity indicates that the characteristics of the distillation neural network deviate further from those of the joint neural network. In this case, convergence of the distillation neural network can be achieved through multiple rounds of iterative training.

[0129] In specific applications, embodiments of this disclosure can employ a teacher-student framework to implement knowledge distillation, distilling the feature representation "knowledge" learned from a complex, highly capable joint neural network and passing it to a standard parameterized network (i.e., a distilled neural network).

[0130] After obtaining the joint neural network G, we can first define a randomly initialized distillation network D. t Then, based on the number of branches in the joint neural network, a distillation-assisted conversion layer d is defined. t During distillation, to make the distillation neural network more versatile than other networks, the training objective here is to increase the connection between the features of the distillation neural network and the various branches of the joint neural network. Therefore, after a specific stage of the distillation neural network, this feature can be input into the distillation auxiliary transformation layer d. t In the process, the distillation auxiliary feature f is obtained. t , where f t =d t (E t ).

[0131] Here, the auxiliary loss function for training the distillation neural network can be determined by defining the L2 distance between the distillation auxiliary features and the branch features of the joint neural network G. The auxiliary loss function is defined as follows:

[0132]

[0133] It should be noted that the loss function of the distillation neural network includes not only the aforementioned auxiliary loss function, but also the main loss function determined by the task of the distillation neural network itself.

[0134] Considering that joint neural networks cover the characteristics of multi-task tasks to a certain extent, this results in a large number of parameters and computational resources being consumed. In order to facilitate the flexible application of joint neural networks to various downstream target tasks, we can perform network part-time operations for different downstream tasks to obtain a more lightweight downstream pruning model (corresponding to the pruned joint neural network).

[0135] In this embodiment, test data collected under the target task and a pruning weight mask constructed for the target task can be obtained. The test data can be input into the trained joint neural network to determine the loss function value of the joint neural network. If the loss function value is determined to be not less than a preset threshold, the pruning weight mask can be adjusted based on the loss function value, and the joint neural network can be pruned based on the adjusted pruning weight mask. The collected test data can then be input into the pruned joint neural network for the next round of network training. This process is repeated until the obtained loss function value is less than the preset threshold.

[0136] The aforementioned pruning weight mask characterizes whether each computational unit in the joint neural network needs to participate in the network training for the target task. Since the loss function value is determined from downstream test data, the effectiveness of pruning using the pruning weight mask can be evaluated.

[0137] In practical applications, different downstream tasks can be referred to as S. t S will not be trained separately here. t Instead, it defines a set of pruning weight masks M corresponding to the target task t. t This weight mask M t This represents a flag indicating whether the corresponding computational unit in the joint neural network G participates in the computation of the downstream model.

[0138] Among them, M t There are two possible values: 0 and 1. 0 indicates that the computational unit will be discarded during pruning, while 1 indicates that the computational unit will be retained and participate in the pruning calculation. In this case, the calculation formula of the pruning model is derived from the S of the joint neural network. t =f(x,G) transformed into S t =f(x, G, M) t ).

[0139] For each target task, this embodiment of the disclosure first sets all M to 1, and then, after a specific number of network iterations i, obtains G. i Then G in the downstream task i The computational units that change least in the original G are pruned according to a specific pruning probability α (e.g., 0.7). The M value of the pruned computational units becomes 0. This process is repeated until G is fully processed. i When the similarity to the original G reaches a pre-set threshold, the loss function value of the corresponding pruning model is sufficiently small.

[0140] It should be noted that when multiple downstream tasks migrate simultaneously, multiple M can be defined, and the pruning tasks of multiple tasks can be synchronized. Here, gradient optimization can be shared among the corresponding tasks, so as to achieve the optimal pruning operation simultaneously on multiple tasks.

[0141] Those skilled in the art will understand that, in the above-described method of the specific implementation, the order in which each step is written does not imply a strict execution order and does not constitute any limitation on the implementation process. The specific execution order of each step should be determined by its function and possible internal logic.

[0142] Based on the same inventive concept, this disclosure also provides a neural network training apparatus corresponding to the neural network training method. Since the principle of the apparatus in this disclosure for solving the problem is similar to the neural network training method described above in this disclosure, the implementation of the apparatus can refer to the implementation of the method, and the repeated parts will not be described again.

[0143] Reference Figure 4 The diagram shown is a schematic representation of a neural network training apparatus according to an embodiment of this disclosure. The apparatus includes: an acquisition module 401, an extraction module 402, a generation module 403, and a training module 404; wherein,

[0144] The acquisition module 401 is used to acquire training data and multiple target neural networks trained therefrom; the training data includes multiple image samples; the multiple target neural networks are used to perform different target tasks respectively;

[0145] The extraction module 402 is used to extract features from the training data using multiple target neural networks respectively, and to obtain the feature information output by the feature extraction layer of each target neural network.

[0146] The generation module 403 is used to determine the loss function value of the joint neural network to be trained based on the feature information output by the feature extraction layers of each target neural network. The joint neural network consists of multiple target neural networks and connection layers between the feature extraction layers of each target neural network.

[0147] Training module 404 is used to perform at least one round of network training on the joint neural network to be trained based on the loss function value, so as to obtain a trained joint neural network.

[0148] The apparatus for training the aforementioned neural network, upon acquiring training data and multiple trained target neural networks, can first extract features from the training data using these target neural networks designed to perform different target tasks. Based on the feature information output by the feature extraction layers of each target neural network, the loss function value of the joint neural network to be trained is determined. Then, the joint neural network is trained based on the loss function value. In the process of learning the joint neural network, this embodiment not only considers the characteristics of different tasks but also utilizes connection layers to achieve feature exchange between different tasks. This can improve the versatility of the trained joint neural network to a certain extent, thereby facilitating related downstream applications.

[0149] In one possible implementation, one of the multiple target neural networks serves as the main neural network of the joint neural network, and the other target neural networks serve as the sub-neural networks of the joint neural network; the generation module 403 is used to determine the loss function value of the joint neural network to be trained based on the feature information output by the feature extraction layers included in each target neural network according to the following steps:

[0150] The second feature information output by the second feature extraction layer of the main neural network is updated based on the first feature information output by the first feature extraction layer of the sub-neural network to obtain the updated second feature information.

[0151] Based on the updated second feature information, the loss function value of the joint neural network to be trained is determined.

[0152] In one possible implementation, the main neural network includes a task layer; a generation module 403 is configured to determine the loss function value of the joint neural network to be trained based on the updated second feature information according to the following steps:

[0153] The updated second feature information is input into the task layer of the main neural network to determine the task output of the main neural network;

[0154] The loss function value of the main neural network is determined by calculating the difference between the task output and the task labeling result of the corresponding input training data.

[0155] The loss function value of the main neural network is used as the loss function value of the joint neural network to be trained.

[0156] In one possible implementation, the sub-neural network includes multiple first feature extraction layers with different output sizes, and the main neural network includes multiple second feature extraction layers with different output sizes; the generation module 403 is used to update the second feature information output by the second feature extraction layer of the main neural network based on the first feature information output by the first feature extraction layer of the sub-neural network according to the following steps:

[0157] For the target second feature extraction layer among the multiple second feature extraction layers included in the main neural network, the second feature information output by the target second feature extraction layer is updated based on the first feature information output by at least one first feature extraction layer included in the sub-neural network, so as to obtain the updated second feature information for the target second feature extraction layer.

[0158] In one possible implementation, the generation module 403 is configured to update the second feature information output by the target second feature extraction layer based on the first feature information output by at least one first feature extraction layer included in the sub-neural network, according to the following steps:

[0159] For a target second feature extraction layer among the multiple second feature extraction layers included in the main neural network, a first feature extraction layer whose output size matches that of the target second feature extraction layer is selected from the multiple first feature extraction layers included in the sub-neural network; and,

[0160] Based on the first feature information output by the selected first feature extraction layer, the second feature information output by the second feature extraction layer of the target is updated to obtain the updated second feature information for the second feature extraction layer of the target.

[0161] In one possible implementation, the generation module 403 is configured to update the second feature information output by the target second feature extraction layer based on the first feature information output by a selected first feature extraction layer, according to the following steps:

[0162] Based on the connection layer between the selected first feature extraction layer and the target second feature extraction layer, the first feature information output by the selected first feature extraction layer and the second feature information output by the target second feature extraction layer are fused to obtain the updated second feature information for the target second feature extraction layer.

[0163] In one possible implementation, where a selected first feature extraction layer comprises multiple feature extraction blocks, and each feature extraction block has a connection layer with the target second feature extraction layer; the generation module 403 is configured to fuse the first feature information output by the selected first feature extraction layer with the second feature information output by the target second feature extraction layer according to the following steps:

[0164] The first feature information output by each feature extraction block of the selected first feature extraction layer is fused with the second feature information output by the target second feature extraction layer to obtain the updated second feature information for the target second feature extraction layer.

[0165] In one possible implementation, the generation module 403 is configured to update the second feature information output by the target second feature extraction layer based on the first feature information output by at least one first feature extraction layer included in the sub-neural network, according to the following steps:

[0166] For the target second feature extraction layer among the multiple second feature extraction layers included in the main neural network, multiple first feature extraction layers corresponding to the target second feature extraction layer are selected from the multiple first feature extraction layers included in the sub-neural network; and,

[0167] The second feature information output by the second feature extraction layer of the target is updated based on the first feature information output by the selected first feature extraction layers.

[0168] In one possible implementation, the above-described apparatus further includes:

[0169] The distillation module 405 is used to extract features from the training data using the distillation neural network to be trained after obtaining the trained joint neural network, thereby obtaining the feature information output by the feature extraction layer of the distillation neural network; to determine the loss function value of the distillation neural network to be trained based on the feature similarity between the feature information output by the feature extraction layer of the distillation neural network and the feature information output by the feature extraction layer of each target neural network in the joint neural network; and to perform at least one round of network training on the distillation neural network to be trained based on the loss function value, thereby obtaining the trained distillation neural network.

[0170] In one possible implementation, the above-described apparatus further includes:

[0171] The pruning module 406, after obtaining the trained joint neural network, acquires test data collected under the target task and a pruning weight mask constructed for the target task. The pruning weight mask characterizes whether each computational unit in the joint neural network participates in the network training for the target task. It then inputs the collected test data into the trained joint neural network to determine the loss function value. The following steps are repeated until the loss function value is less than a preset threshold:

[0172] The pruning weight mask is adjusted based on the loss function value, and the joint neural network is pruned based on the adjusted pruning weight mask; the collected test data is then input into the pruned joint neural network for the next round of network training.

[0173] In one possible implementation, the target neural network includes a feature extraction layer for feature extraction and a task layer for task processing; the acquisition module 401 is configured to train the target neural network according to the following steps:

[0174] Select training data that matches the current task from the acquired training data;

[0175] The selected training data is input into the feature extraction layer of the target neural network to be trained, and the feature information output by the feature extraction layer is obtained.

[0176] The feature information output by the feature extraction layer is input into the task layer of the target neural network to be trained, and the task output result of the task layer is obtained.

[0177] If the task output does not match the task labeling results for the selected training data, the target neural network to be trained is retrained until the task output matches the task labeling results for the selected training data.

[0178] The processing flow of each module in the device and the interaction flow between each module can be referred to the relevant descriptions in the above method embodiments, and will not be detailed here.

[0179] This disclosure also provides an electronic device, such as... Figure 5 The diagram shown is a schematic representation of an electronic device structure provided in this embodiment of the present disclosure, including: a processor 501, a memory 502, and a bus 503. The memory 502 stores machine-readable instructions executable by the processor 501 (e.g., ...). Figure 4 The device includes the acquisition module 401, extraction module 402, generation module 403, and training module 404 (and their corresponding execution instructions). When the electronic device is running, the processor 501 and memory 502 communicate via bus 503. When machine-readable instructions are executed by the processor 501, the following processing is performed:

[0180] Acquire training data and multiple target neural networks trained from it; the training data includes multiple image samples; the multiple target neural networks are used to perform different target tasks.

[0181] Multiple target neural networks are used to extract features from the training data to obtain the feature information output by the feature extraction layer of each target neural network.

[0182] Based on the feature information output by the feature extraction layers of each target neural network, the loss function value of the joint neural network to be trained is determined. The joint neural network consists of multiple target neural networks and connection layers between the feature extraction layers of each target neural network.

[0183] Based on the loss function value, perform at least one round of network training on the joint neural network to be trained to obtain a trained joint neural network.

[0184] This disclosure also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, performs the steps of the neural network training method described in the above-described method embodiments. The storage medium may be a volatile or non-volatile computer-readable storage medium.

[0185] This disclosure also provides a computer program product carrying program code. The program code includes instructions that can be used to execute the steps of the neural network training method described in the above method embodiments. For details, please refer to the above method embodiments, which will not be repeated here.

[0186] The aforementioned computer program product can be implemented through hardware, software, or a combination thereof. In one optional embodiment, the computer program product is specifically embodied in a computer storage medium; in another optional embodiment, the computer program product is specifically embodied in a software product, such as a software development kit (SDK), etc.

[0187] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems and devices described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here. In the several embodiments provided in this disclosure, it should be understood that the disclosed systems, devices, and methods can be implemented in other ways. The device embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. Furthermore, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Another point is that the displayed or discussed mutual coupling or direct coupling or communication connection may be through some communication interfaces; the indirect coupling or communication connection of devices or units may be electrical, mechanical, or other forms.

[0188] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0189] In addition, the functional units in the various embodiments of this disclosure can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0190] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a processor-executable, non-volatile, computer-readable storage medium. Based on this understanding, the technical solution of this disclosure, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause an electronic device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this disclosure. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0191] Finally, it should be noted that the above-described embodiments are merely specific implementations of this disclosure, used to illustrate the technical solutions of this disclosure, and not to limit it. The protection scope of this disclosure is not limited thereto. Although this disclosure has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that any person skilled in the art can still modify or easily conceive of changes to the technical solutions described in the foregoing embodiments, or make equivalent substitutions for some of the technical features, within the scope of the technology disclosed in this disclosure. Such modifications, changes, or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this disclosure, and should all be covered within the protection scope of this disclosure. Therefore, the protection scope of this disclosure should be determined by the protection scope of the claims.

Claims

1. A method for training a neural network, characterized in that, The method includes: Acquire training data and multiple target neural networks trained therefrom; the training data includes multiple image samples; the multiple target neural networks are used to perform different target tasks respectively; The training data is extracted using the multiple target neural networks respectively, and the feature information output by the feature extraction layer of each target neural network is obtained. Based on the feature information output by the feature extraction layers of each of the target neural networks, the loss function value of the joint neural network to be trained is determined, wherein the joint neural network is composed of the plurality of target neural networks and the connection layers between the feature extraction layers of each of the target neural networks; Based on the loss function value, perform at least one round of network training on the joint neural network to be trained to obtain a trained joint neural network. One of the multiple target neural networks serves as the main neural network of the joint neural network, and the other target neural networks serve as the sub-neural networks of the joint neural network. The sub-neural network includes multiple first feature extraction layers with different output sizes, and the main neural network includes multiple second feature extraction layers with different output sizes. The connection layer realizes the link between the first feature extraction layers and the second feature extraction layers, and is used to fuse the first feature information output by the first feature extraction layer with the second feature information output by the second feature extraction layer.

2. The method according to claim 1, characterized in that, The step of determining the loss function value of the joint neural network to be trained based on the feature information output by the feature extraction layers of each of the target neural networks includes: The second feature information output by the second feature extraction layer of the main neural network is updated based on the first feature information output by the first feature extraction layer of the sub-neural network to obtain the updated second feature information. Based on the updated second feature information, the loss function value of the joint neural network to be trained is determined.

3. The method according to claim 2, characterized in that, The main neural network includes a task layer; determining the loss function value of the joint neural network to be trained based on the updated second feature information includes: The updated second feature information is input into the task layer of the main neural network to determine the task output result of the main neural network. The loss function value of the main neural network is determined based on the difference between the task output and the task labeling result of the corresponding input training data. The loss function value of the main neural network is determined as the loss function value of the joint neural network to be trained.

4. The method according to claim 2 or 3, characterized in that, The sub-neural network includes multiple first feature extraction layers with different output sizes, and the main neural network includes multiple second feature extraction layers with different output sizes; updating the second feature information output by the second feature extraction layer of the main neural network based on the first feature information output by the first feature extraction layer of the sub-neural network includes: For the target second feature extraction layer among the multiple second feature extraction layers included in the main neural network, the second feature information output by the target second feature extraction layer is updated based on the first feature information output by at least one first feature extraction layer included in the sub-neural network, so as to obtain the updated second feature information for the target second feature extraction layer.

5. The method according to claim 4, characterized in that, The step of updating the second feature information output by the second feature extraction layer of the target based on the first feature information output by at least one first feature extraction layer of the sub-neural network includes: For a target second feature extraction layer among the multiple second feature extraction layers included in the main neural network, a first feature extraction layer whose output size matches that of the target second feature extraction layer is selected from the multiple first feature extraction layers included in the sub-neural network; and, Based on the selected first feature information output by the first feature extraction layer, the second feature information output by the target second feature extraction layer is updated to obtain the updated second feature information for the target second feature extraction layer.

6. The method according to claim 5, characterized in that, The step of updating the second feature information output by the second feature extraction layer of the target based on the first feature information selected from the first feature extraction layer includes: Based on a selected connection layer between the first feature extraction layer and the target second feature extraction layer, the first feature information output by the selected first feature extraction layer is fused with the second feature information output by the target second feature extraction layer to obtain the updated second feature information for the target second feature extraction layer.

7. The method according to claim 6, characterized in that, In the case where a selected first feature extraction layer comprises multiple feature extraction blocks, and each feature extraction block has a connection layer with the target second feature extraction layer; the fusion of the first feature information output by the selected first feature extraction layer and the second feature information output by the target second feature extraction layer includes: The first feature information output by each feature extraction block of the selected first feature extraction layer is fused with the second feature information output by the target second feature extraction layer to obtain the updated second feature information for the target second feature extraction layer.

8. The method according to claim 4, characterized in that, The step of updating the second feature information output by the second feature extraction layer of the target based on the first feature information output by at least one first feature extraction layer of the sub-neural network includes: For a target second feature extraction layer among the multiple second feature extraction layers included in the main neural network, multiple first feature extraction layers corresponding to the target second feature extraction layer are selected from the multiple first feature extraction layers included in the sub-neural network; and, The second feature information output by the second feature extraction layer of the target is updated based on the first feature information selected from multiple first feature extraction layers.

9. The method according to any one of claims 1-3 and 5-8, characterized in that, After obtaining the trained joint neural network, the method further includes: The training data is used to extract features using a distillation neural network to be trained, thereby obtaining the feature information output by the feature extraction layer of the distillation neural network. Based on the feature similarity between the feature information output by the feature extraction layer of the distillation neural network and the feature information output by the feature extraction layer of each target neural network in the joint neural network, the loss function value of the distillation neural network to be trained is determined. Based on the loss function value, at least one round of network training is performed on the distillation neural network to be trained to obtain a trained distillation neural network.

10. The method according to any one of claims 1-3 and 5-8, characterized in that, After obtaining the trained joint neural network, the method further includes: The test data collected under the target task and the pruning weight mask constructed for the target task are obtained, wherein the pruning weight mask is used to characterize whether each computing unit in the joint neural network participates in the network training of the target task. The collected test data is input into the trained joint neural network to determine the loss function value of the joint neural network; Repeat the following steps until the loss function value is less than a preset threshold: The pruning weight mask is adjusted based on the loss function value, and the joint neural network is pruned based on the adjusted pruning weight mask. The collected test data is input into the pruned joint neural network for the next round of network training.

11. The method according to any one of claims 1-3 and 5-8, characterized in that, The target neural network includes a feature extraction layer for feature extraction and a task layer for task processing; the target neural network is trained according to the following steps: Select training data that matches the current task from the acquired training data; The selected training data is input into the feature extraction layer of the target neural network to be trained, and the feature information output by the feature extraction layer is obtained. The feature information output by the feature extraction layer is input into the task layer of the target neural network to be trained, and the task output result of the task layer is obtained. If the task output and the task labeling results for the selected training data do not match, the target neural network to be trained is trained again until the task output and the task labeling results for the selected training data match.

12. A device for training a neural network, characterized in that, The device includes: An acquisition module is used to acquire training data and multiple trained target neural networks; the training data includes multiple image samples; the multiple target neural networks are used to perform different target tasks. The extraction module is used to extract features from the training data using the multiple target neural networks respectively, and to obtain the feature information output by the feature extraction layer of each target neural network. A generation module is used to determine the loss function value of the joint neural network to be trained based on the feature information output by the feature extraction layers of each of the target neural networks, wherein the joint neural network is composed of the plurality of target neural networks and the connection layers between the feature extraction layers of each of the target neural networks; The training module is used to perform at least one round of network training on the joint neural network to be trained based on the loss function value, so as to obtain a trained joint neural network. One of the multiple target neural networks serves as the main neural network of the joint neural network, and the other target neural networks serve as the sub-neural networks of the joint neural network. The sub-neural network includes multiple first feature extraction layers with different output sizes, and the main neural network includes multiple second feature extraction layers with different output sizes. The connection layer realizes the link between the first feature extraction layers and the second feature extraction layers, and is used to fuse the first feature information output by the first feature extraction layer with the second feature information output by the second feature extraction layer.

13. An electronic device, characterized in that, include: The device includes a processor, a memory, and a bus. The memory stores machine-readable instructions executable by the processor. When the electronic device is running, the processor communicates with the memory via the bus. When the machine-readable instructions are executed by the processor, they perform the steps of the neural network training method as described in any one of claims 1 to 11.

14. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, performs the steps of the method for training a neural network as described in any one of claims 1 to 11.

Images