A deep neural network training method, device and computer equipment

Abstract

The embodiment of the present invention provides a deep neural network training method, device and computer equipment, wherein the deep neural network training method includes: for the current network layer in the tree network topology structure, obtaining the task attributes of each node in the current network layer, wherein each node in the tree network topology structure is a neural network corresponding to a different task, and the leaf node in the tree network topology structure is a neural network that has completed training for a specified task; based on the task attributes of each node in the current network layer, clustering analysis is performed on each node in the current network layer, and the common parts of the task attributes of multiple nodes in the same category are extracted as the task attributes of the parent nodes of the multiple nodes; based on the task attributes of each parent node, the network parameters of each parent node are trained; after completing the training of each node in each network layer, determining that the deep neural network training corresponding to the tree network topology structure is completed. This solution can improve the computational efficiency of deep learning.

Description

Technical Field

[0001] The present invention relates to the technical field of machine learning, and in particular to a deep neural network training method, device and computer equipment. Background Art

[0002] Deep learning is a machine learning method developed on the basis of artificial intelligence neural networks. As the main model of deep learning, deep neural networks interpret data by imitating the mechanisms of the human brain. It is an intelligent model that analyzes and learns by building and simulating the human brain. It has been widely used in application fields such as speech recognition, image classification, face recognition, natural language processing, and advertising.

[0003] At present, most deep learning only targets a single task, such as detecting the attributes of an object, estimating the state of an object, etc. For complex scenarios, multiple tasks often need to be implemented. The commonly used method is to use multiple neural networks to perform operations on each task separately, and then merge the results. This process is very time-consuming, and due to the high redundancy in each neural network, the computational efficiency of deep learning is low. Summary of the invention

[0004] The purpose of the embodiments of the present invention is to provide a deep neural network training method, device and computer equipment to improve the computing efficiency of deep learning. The specific technical solution is as follows:

[0005] In a first aspect, an embodiment of the present invention provides a deep neural network training method, the method comprising:

[0006] For a current network layer in a tree-like network topology structure, obtaining a task attribute of each node in the current network layer, wherein each node in the tree-like network topology structure is a neural network corresponding to a different task, and a leaf node in the tree-like network topology structure is a neural network that has completed training for a specified task;

[0007] Based on the task attributes of each node in the current network layer, cluster analysis is performed on each node in the current network layer, and common parts of the task attributes of multiple nodes in the same category are extracted as the task attributes of the parent nodes of the multiple nodes;

[0008] Based on the task attributes of each parent node, train the network parameters of each parent node;

[0009] After completing the training of each node in each network layer, it is determined that the training of the deep neural network corresponding to the tree network topology structure is completed.

[0010] Optionally, the clustering analysis of each node in the current network layer based on the task attributes of each node in the current network layer, extracting common parts of the task attributes of multiple nodes in the same category as the task attributes of the parent nodes of the multiple nodes, includes:

[0011] According to the task attributes of each node in the current network layer, a similarity measurement matrix corresponding to the task attributes of each node is generated by a preset algorithm;

[0012] According to the similarity measurement matrix, multiple nodes whose similarity is greater than a preset threshold are determined to be of the same category;

[0013] The common parts of the task attributes of multiple nodes in the same category are extracted as the task attributes of the parent nodes of the multiple nodes.

[0014] Optionally, the training of network parameters of each parent node based on the task attributes of each parent node includes:

[0015] For any parent node, based on the task attributes of the parent node, the output features of each child node of the parent node are used as the input of the parent node to train the network parameters of the parent node, thereby completing the training of the parent node.

[0016] Optionally, the training of network parameters of each parent node based on the task attributes of each parent node includes:

[0017] For any parent node, based on the task attributes of the parent node, the parent node is generated using a preset structure with a feature signal control mechanism. The output features of each child node of the parent node are used as the input of the parent node, and the feature signals related to the task attributes are selected, and the feature signals irrelevant to the task attributes are eliminated. The weighted combination of the feature signals related to the task attributes is obtained as the network parameters of the neural network corresponding to the parent node, and the training of the parent node is completed.

[0018] Optionally, after completing the training of each node in each network layer, determining that the training of the deep neural network corresponding to the tree network topology structure is completed includes:

[0019] Starting from the network layer where the leaf node is located, the network parameters of the nodes in each network layer are trained in sequence from the bottom layer to the top layer to complete the training of each node;

[0020] After the training of each node at the top level is completed, it is determined that the training of the deep neural network corresponding to the tree network topology structure is completed.

[0021] In a second aspect, an embodiment of the present invention provides a deep neural network training device, the device comprising:

[0022] An acquisition module, for acquiring, for a current network layer in a tree-like network topology, a task attribute of each node in the current network layer, wherein each node in the tree-like network topology is a neural network corresponding to a different task, and a leaf node in the tree-like network topology is a neural network that has completed training for a specified task;

[0023] A clustering module, configured to perform cluster analysis on each node in the current network layer based on the task attributes of each node in the current network layer, and extract the common parts of the task attributes of multiple nodes in the same category as the task attributes of the parent nodes of the multiple nodes;

[0024] The training module is used to train the network parameters of each parent node based on the task attributes of each parent node;

[0025] The determination module is used to determine that the training of the deep neural network corresponding to the tree network topology structure is completed after the training of each node in each network layer is completed.

[0026] Optionally, the clustering module is specifically used to:

[0027] According to the task attributes of each node in the current network layer, a similarity measurement matrix corresponding to the task attributes of each node is generated by a preset algorithm;

[0028] According to the similarity measurement matrix, multiple nodes whose similarity is greater than a preset threshold are determined to be of the same category;

[0029] The common parts of the task attributes of multiple nodes in the same category are extracted as the task attributes of the parent nodes of the multiple nodes.

[0030] Optionally, the training module is specifically used for:

[0031] For any parent node, based on the task attributes of the parent node, the output features of each child node of the parent node are used as the input of the parent node to train the network parameters of the parent node, thereby completing the training of the parent node.

[0032] Optionally, the training module is specifically used for:

[0033] For any parent node, based on the task attributes of the parent node, the parent node is generated using a preset structure with a feature signal control mechanism. The output features of each child node of the parent node are used as the input of the parent node, and the feature signals related to the task attributes are selected, and the feature signals irrelevant to the task attributes are eliminated. The weighted combination of the feature signals related to the task attributes is obtained as the network parameters of the neural network corresponding to the parent node, and the training of the parent node is completed.

[0034] Optionally, the determining module is specifically used to:

[0035] Starting from the network layer where the leaf node is located, the network parameters of the nodes in each network layer are trained in sequence from the bottom layer to the top layer to complete the training of each node;

[0036] After the training of each node at the top level is completed, the deep neural network corresponding to the tree network topology is determined.

[0037] In a third aspect, an embodiment of the present invention provides a computer device, including a processor and a memory, wherein:

[0038] The memory is used to store computer programs;

[0039] The processor is used to implement the method steps described in the first aspect when executing the program stored in the memory.

[0040] The embodiment of the present invention provides a deep neural network training method, device and computer equipment, which constructs a tree network topology structure, obtains the task attributes of each node in the current network layer that has completed training in the tree network topology structure, performs cluster analysis on each node in the current network layer based on the task attributes, and uses the common part of the task attributes of multiple nodes in the same category as the task attributes of the parent node, so that the network parameters of each parent node can be trained according to the task attributes of each parent node. After the training of each node in each network layer is completed, it can be determined that the deep neural network corresponding to the tree network topology structure has been trained, and the trained deep neural network can realize multiple tasks. Since the leaf nodes in the tree network topology structure are neural networks that have been trained for specified tasks, the neural network corresponding to the specified task can be reused by extracting the common part of the task attributes of each node in the network layer. Through the operation of the tree network topology structure layer by layer, a complete deep neural network can be used to realize multiple specified tasks, and the nodes are clustered based on the task attributes to construct the parent nodes of the nodes belonging to the same category. The parent node can realize the common tasks of the child nodes, so the redundancy between neural networks can be effectively reduced, thereby improving the operation efficiency of deep learning. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0042] Figure 1 A schematic diagram of a deep neural network training method according to an embodiment of the present invention;

[0043] Figure 2 Schematic diagram of the structure of a deep neural network training device according to an embodiment of the present invention;

[0044] Figure 3 Schematic diagram of the structure of a computer device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0045] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0046] In order to improve the operational efficiency of deep learning, an embodiment of the present invention provides a deep neural network training method, device and computer equipment. Below, the deep neural network training method provided by the embodiment of the present invention is first introduced. The execution subject of a deep neural network training method provided by an embodiment of the present invention can be a computer device that implements multiple specified tasks, and the execution subject at least includes a core processing chip with data processing capabilities. The method of implementing a deep neural network training method provided by an embodiment of the present invention can be at least one of software, hardware circuits and logic circuits set in the execution subject.

[0047] like Figure 1 As shown, a deep neural network training method provided by an embodiment of the present invention may include the following steps:

[0048] S101, for a current network layer in a tree network topology structure, obtaining a task attribute of each node in the current network layer.

[0049] For multiple specified tasks, using preset training algorithms, such as back propagation algorithm, forward propagation algorithm, gradient descent training algorithm, etc., a neural network for each specified task can be obtained. Each neural network can implement a specific task, such as target attribute detection, target state estimation, etc. There are often certain similarities between these tasks, so there is redundancy in each neural network that implements these tasks, that is, there are some identical network models between the trained neural networks for implementing the same tasks. Based on the above considerations, a complete deep neural network can be designed based on the neural network used to perform different specified tasks to implement each specified task. The deep neural network can be a tree network topology structure, in which each node can be a neural network corresponding to a different task. There are three types of nodes in the tree network topology structure: leaf nodes, root nodes, and intermediate nodes. Since the ultimate goal is to achieve each specified task, the leaf node is a neural network that has completed training for the specified task, and the leaf node can be regarded as an encoder. The intermediate node and the root node are neural networks trained layer by layer based on the nodes of the next layer.

[0050] S102, based on the task attributes of each node in the current network layer, cluster analysis is performed on each node in the current network layer, and common parts of the task attributes of multiple nodes in the same category are extracted as the task attributes of the parent nodes of the multiple nodes.

[0051] Since there are certain similarities between the tasks corresponding to each node in a network layer in the tree network topology, for example, when identifying the gender of a target and identifying the age of a target, since both of these specified tasks require the detection of the human body or face area first, and gender classification and age estimation are performed based on this, the two nodes corresponding to the two specified tasks can be divided into the same category, and the common parts of the two specified tasks, that is, the detection of the human body or face area, can be extracted, and the detection of the human body or face area can be used as the task attribute of the parent node of the above two nodes. In this way, when performing operations through a deep neural network, the task of detecting the human body or face area can be performed first, and after the human body or face area is detected, the tasks of gender recognition and age estimation of the target can be performed.

[0052] Cluster analysis is the process of classifying data into different categories, so objects in the same category have great similarities. There are many ways of cluster analysis, such as system clustering, decomposition, addition, dynamic clustering, etc., which are not limited here. In this embodiment, cluster analysis is to classify nodes with similar task attributes into the same category. Therefore, tasks can be clustered according to the similarity of task attributes, and nodes corresponding to task attributes of the same category can be set to have a common parent node. In order to improve the efficiency of cluster analysis, similarity metrics can be used for clustering. Therefore, optionally, the method of clustering each node in the network layer to obtain a parent node can include the following steps:

[0053] In the first step, according to the task attributes of each node in the current network layer, a similarity measurement matrix corresponding to the task attributes of each node is generated through a preset algorithm.

[0054] In the second step, based on the similarity measurement matrix, multiple nodes whose similarity is greater than a preset threshold are determined to be in the same category.

[0055] The third step is to extract the common parts of the task attributes of multiple nodes in the same category as the task attributes of the parent nodes of the multiple nodes.

[0056] The preset algorithm is the algorithm used for similarity measurement, which may be Euclidean distance, Manhattan distance, Chebyshev distance, Minkowski distance, standardized Euclidean distance, Mahalanobis distance, angle cosine, Hamming distance, etc., without specific limitation here. Through these algorithms, a similarity measurement matrix between task attributes and task attributes can be generated. According to the similarity measurement matrix, if the similarity of multiple task attributes is greater than a preset threshold, it means that the multiple task attributes are relatively similar, then the nodes that implement these task attributes can be determined as the same category, and the common parts of these task attributes can be calculated only once. After the execution of the common parts of the task attributes, the characteristic parts of each task attribute are executed. In this way, the redundancy between the neural networks can be effectively reduced and the operation efficiency can be improved.

[0057] S103: training network parameters of each parent node based on the task attributes of each parent node.

[0058] After determining the task attributes of each parent node, the task that each parent node needs to achieve can be determined. The traditional neural network model that achieves the task can be used. However, the network parameters of each parent node need to be trained before they can be determined, that is, the output of each parent node can meet the task requirements of the neural network as its child node. Therefore, the network parameters of each parent node can be trained through the following steps:

[0059] For any parent node, based on the task attributes of the parent node, the output features of each child node of the parent node are used as the input of the parent node to train the network parameters of the parent node, thereby completing the training of the parent node.

[0060] The training process of network parameters can be achieved by taking the output features of the child nodes of each parent node as the input of the parent node, and then continuously adjusting the network parameters so that the output features of each child node meet the specified task. For the leaf nodes of the tree network topology, since the penultimate layer of the neural network is often an instantaneous function, and the penultimate layer is a feature layer, that is, the output of the penultimate layer is the encoded feature, the output of the penultimate layer is used as the input of the node of the previous level.

[0061] Since different features will influence each other, if the feature weights of the input parent node are the same, the results obtained after the operation of the parent node and the child node may be quite different from the original task results. Therefore, the parent node can be composed of any structure with a signal control mechanism, such as LSTM (Long Short-Term Memory) network mechanism, attention mechanism, GRU (Gated Recurrent Unit) network mechanism, etc., by weighted combination of the input of the parent node, training the network parameters of the parent node, and adjusting the weights during the training process. That is, the network parameters of each parent node can also be trained through the following steps:

[0062] For any parent node, based on the task attributes of the parent node, the parent node is generated using a preset structure with a feature signal control mechanism. The output features of each child node of the parent node are used as the input of the parent node, and the feature signals related to the task attributes are selected, and the feature signals irrelevant to the task attributes are eliminated. The weighted combination of the feature signals related to the task attributes is obtained as the network parameters of the neural network corresponding to the parent node, and the training of the parent node is completed.

[0063] The essence of the parent node being composed of any structure with a signal control mechanism is to automatically select relevant feature signals for specific task attributes and eliminate feature signals that are irrelevant to the task attributes. And through weighted combination, a larger weight can be assigned to the valid part of the child node and a smaller weight can be assigned to the invalid part. In this way, the expected task effect can be highlighted in the task result, while the invalid part can be shielded, thereby improving the performance on the specified task.

[0064] S104, after completing the training of each node in each network layer, determine that the training of the deep neural network corresponding to the tree network topology structure is completed.

[0065] Based on the above process, we can get the nodes of a network layer in the tree network topology structure. By training each node in each network layer, the deep neural network corresponding to the tree network topology structure can be determined after the training is completed.

[0066] Optionally, the step of determining the end of deep neural network training may specifically include:

[0067] Starting from the network layer where the leaf node is located, the network parameters of the nodes in each network layer are trained in sequence from the bottom layer to the top layer to complete the training of each node;

[0068] After the training of each node in the top layer is completed, it is determined that the training of the deep neural network corresponding to the tree network topology is completed.

[0069] The execution subject can periodically detect whether there are new neural networks. If a new neural network corresponding to a specified task is added, the neural network can be used as a leaf node of the tree network topology structure according to the above process, and trained step by step from bottom to top until the root node. In order to ensure the consistency of the update, the training process of each node in the same network layer is independent of each other and will not be interfered by other nodes.

[0070] By using this embodiment, by constructing a tree network topology structure, the task attributes of each node in the current network layer that has completed training in the tree network topology structure are obtained, and based on the task attributes, cluster analysis is performed on each node in the current network layer, and the common part of the task attributes of multiple nodes in the same category is used as the task attribute of the parent node, so that the network parameters of each parent node can be trained according to the task attributes of each parent node. After the training of each node in each network layer is completed, it can be determined that the deep neural network corresponding to the tree network topology structure has been trained, and the trained deep neural network can realize multiple tasks. Since the leaf nodes in the tree network topology structure are neural networks that have been trained for specified tasks, by extracting the common part of the task attributes of each node in the network layer, the neural network corresponding to the specified task can be reused, and through the operation of the tree network topology structure layer by layer, a complete deep neural network can be used to realize multiple specified tasks, and by clustering the nodes based on the task attributes, the parent nodes of the nodes belonging to the same category are constructed, and the parent nodes can realize the common tasks of the child nodes, so the redundancy between neural networks can be effectively reduced, thereby improving the operation efficiency of deep learning.

[0071] Corresponding to the above method embodiment, the embodiment of the present invention provides a deep neural network training device, such as Figure 2 As shown, the deep neural network training device may include:

[0072] An acquisition module 210 is used to acquire, for a current network layer in a tree network topology structure, a task attribute of each node in the current network layer, wherein each node in the tree network topology structure is a neural network corresponding to a different task, and a leaf node in the tree network topology structure is a neural network that has completed training for a specified task;

[0073] A clustering module 220, configured to perform cluster analysis on each node in the current network layer based on the task attributes of each node in the current network layer, and extract common parts of the task attributes of multiple nodes in the same category as the task attributes of the parent nodes of the multiple nodes;

[0074] A training module 230, for training network parameters of each parent node based on the task attributes of each parent node;

[0075] The determination module 240 is used to determine that the training of the deep neural network corresponding to the tree network topology structure is completed after the training of each node in each network layer is completed.

[0076] Optionally, the clustering module 220 may be specifically used for:

[0077] According to the task attributes of each node in the current network layer, a similarity measurement matrix corresponding to the task attributes of each node is generated by a preset algorithm;

[0078] According to the similarity measurement matrix, multiple nodes whose similarity is greater than a preset threshold are determined to be of the same category;

[0079] The common parts of the task attributes of multiple nodes in the same category are extracted as the task attributes of the parent nodes of the multiple nodes.

[0080] Optionally, the training module 230 may be specifically used for:

[0081] For any parent node, based on the task attributes of the parent node, the output features of each child node of the parent node are used as the input of the parent node to train the network parameters of the parent node, thereby completing the training of the parent node.

[0082] Optionally, the training module 230 may be specifically used for:

[0083] For any parent node, based on the task attributes of the parent node, the parent node is generated using a preset structure with a feature signal control mechanism. The output features of each child node of the parent node are used as the input of the parent node, and the feature signals related to the task attributes are selected, and the feature signals irrelevant to the task attributes are eliminated. The weighted combination of the feature signals related to the task attributes is obtained as the network parameters of the neural network corresponding to the parent node, and the training of the parent node is completed.

[0084] Optionally, the determining module 240 may be specifically configured to:

[0085] Starting from the network layer where the leaf node is located, the network parameters of the nodes in each network layer are trained in sequence from the bottom layer to the top layer to complete the training of each node;

[0086] After the training of each node at the top level is completed, it is determined that the training of the deep neural network corresponding to the tree network topology structure is completed.

[0087] By using this embodiment, by constructing a tree network topology structure, the task attributes of each node in the current network layer that has completed training in the tree network topology structure are obtained, and based on the task attributes, cluster analysis is performed on each node in the current network layer, and the common part of the task attributes of multiple nodes in the same category is used as the task attribute of the parent node, so that the network parameters of each parent node can be trained according to the task attributes of each parent node. After the training of each node in each network layer is completed, it can be determined that the deep neural network corresponding to the tree network topology structure has been trained, and the trained deep neural network can realize multiple tasks. Since the leaf nodes in the tree network topology structure are neural networks that have been trained for specified tasks, by extracting the common part of the task attributes of each node in the network layer, the neural network corresponding to the specified task can be reused, and through the operation of the tree network topology structure layer by layer, a complete deep neural network can be used to realize multiple specified tasks, and by clustering the nodes based on the task attributes, the parent nodes of the nodes belonging to the same category are constructed, and the parent nodes can realize the common tasks of the child nodes, so the redundancy between neural networks can be effectively reduced, thereby improving the operation efficiency of deep learning.

[0088] The embodiment of the present invention also provides a computer device, such as Figure 3 As shown, it includes a processor 301 and a memory 302, wherein,

[0089] Memory 302, used for storing computer programs;

[0090] The processor 301 is used to execute the program stored in the memory 302 to implement the following steps:

[0091] For a current network layer in a tree-like network topology structure, obtaining a task attribute of each node in the current network layer, wherein each node in the tree-like network topology structure is a neural network corresponding to a different task, and a leaf node in the tree-like network topology structure is a neural network that has completed training for a specified task;

[0092] Based on the task attributes of each node in the current network layer, cluster analysis is performed on each node in the current network layer, and common parts of the task attributes of multiple first nodes in the same category are extracted as the task attributes of the parent nodes of the multiple first nodes;

[0093] Based on the task attributes of each parent node, train the network parameters of each parent node;

[0094] After completing the training of each node in each network layer, it is determined that the training of the deep neural network corresponding to the tree network topology structure is completed.

[0095] Optionally, in the step of performing cluster analysis on each node in the current network layer based on the task attributes of each node in the current network layer, and extracting common parts of the task attributes of multiple nodes in the same category as the task attributes of the parent nodes of the multiple nodes, the processor 301 may specifically implement:

[0096] According to the task attributes of each node in the current network layer, a similarity measurement matrix corresponding to the task attributes of each node is generated by a preset algorithm;

[0097] According to the similarity measurement matrix, multiple nodes whose similarity is greater than a preset threshold are determined to be of the same category;

[0098] The common parts of the task attributes of multiple nodes in the same category are extracted as the task attributes of the parent nodes of the multiple nodes.

[0099] Optionally, in implementing the step of training the network parameters of each parent node based on the task attributes of each parent node, the processor 301 may specifically implement:

[0100] For any parent node, based on the task attributes of the parent node, the output features of each child node of the parent node are used as the input of the parent node to train the network parameters of the parent node, thereby completing the training of the parent node.

[0101] Optionally, in the step of implementing the step of training the network parameters of each parent node based on the task attributes of each parent node, the processor 301 may further specifically implement:

[0102] For any parent node, based on the task attributes of the parent node, the parent node is generated using a preset structure with a feature signal control mechanism. The output features of each child node of the parent node are used as the input of the parent node, and the feature signals related to the task attributes are selected, and the feature signals irrelevant to the task attributes are eliminated. The weighted combination of the feature signals related to the task attributes is obtained as the network parameters of the neural network corresponding to the parent node, and the training of the parent node is completed.

[0103] Optionally, in the step of determining that the training of the deep neural network corresponding to the tree network topology structure is completed after the training of each node in each network layer is completed, the processor 301 may further specifically implement:

[0104] Starting from the network layer where the leaf node is located, the network parameters of the nodes in each network layer are trained in sequence from the bottom layer to the top layer to complete the training of each node;

[0105] After the training of each node at the top level is completed, it is determined that the training of the deep neural network corresponding to the tree network topology structure is completed.

[0106] The memory may include a RAM (Random Access Memory) or a NVM (Non-Volatile Memory), such as at least one disk memory. Optionally, the memory may also be at least one storage device located away from the processor.

[0107] The above-mentioned processor can be a general-purpose processor, including a CPU (Central Processing Unit), an NP (Network Processor), etc.; it can also be a DSP (Digital Signal Processing), an ASIC (Application Specific Integrated Circuit), an FPGA (Field-Programmable Gate Array) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0108] In this embodiment, the processor of the computer device can achieve the following by reading the computer program stored in the memory and running the computer program: by constructing a tree network topology structure, obtaining the task attributes of each node in the current network layer that has completed training in the tree network topology structure, performing cluster analysis on each node in the current network layer based on the task attributes, and taking the common parts of the task attributes of multiple nodes in the same category as the task attributes of the parent node, so that the network parameters of each parent node can be trained according to the task attributes of each parent node. After completing the training of each node in each network layer, it can be determined that the deep neural network corresponding to the tree network topology structure has been trained, and the trained deep neural network can achieve multiple tasks. Since the leaf nodes in the tree network topology are neural networks that have been trained for specified tasks, the neural networks corresponding to the specified tasks can be reused by extracting the common parts of the task attributes of each node in the network layer. Through the operation of the tree network topology layer by layer, a complete deep neural network can be used to implement multiple specified tasks. By clustering the nodes based on the task attributes, the parent nodes of the nodes belonging to the same category are constructed. The parent nodes can implement the common tasks of the child nodes, thereby effectively reducing the redundancy between neural networks and improving the computational efficiency of deep learning.

[0109] In addition, corresponding to the deep neural network training method provided in the above embodiment, an embodiment of the present invention provides a computer-readable storage medium for storing a computer program. When the computer program is executed by a processor, the steps of the above deep neural network training method are implemented.

[0110] In this embodiment, a computer-readable storage medium stores an application program that executes the deep neural network training method provided by the embodiment of the present invention at runtime, thereby being able to achieve: by constructing a tree network topology structure, obtaining the task attributes of each node in the current network layer that has completed training in the tree network topology structure, performing cluster analysis on each node in the current network layer based on the task attributes, and taking the common parts of the task attributes of multiple nodes in the same category as the task attributes of the parent node, so that the network parameters of each parent node can be trained according to the task attributes of each parent node. After completing the training of each node in each network layer, it can be determined that the deep neural network corresponding to the tree network topology structure has been trained, and the trained deep neural network can achieve multiple tasks. Since the leaf nodes in the tree network topology are neural networks that have been trained for specified tasks, the neural networks corresponding to the specified tasks can be reused by extracting the common parts of the task attributes of each node in the network layer. Through the operation of the tree network topology layer by layer, a complete deep neural network can be used to implement multiple specified tasks. By clustering the nodes based on the task attributes, the parent nodes of the nodes belonging to the same category are constructed. The parent nodes can implement the common tasks of the child nodes, thereby effectively reducing the redundancy between neural networks and improving the computational efficiency of deep learning.

[0111] As for the computer device and computer-readable storage medium embodiments, since the method contents involved are basically similar to those of the aforementioned method embodiments, the description is relatively simple, and the relevant parts may refer to the partial description of the method embodiments.

[0112] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0113] Each embodiment in this specification is described in a related manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device, computer equipment, and computer-readable storage medium embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiments.

[0114] The above description is only a preferred embodiment of the present invention and is not intended to limit the protection scope of the present invention. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention are included in the protection scope of the present invention.

Claims

1. A deep neural network training method, characterized in that: The execution subject of the method includes a core processing chip with data processing capability, and the method includes: For a current network layer in a tree network topology structure, obtaining a task attribute of each node in the current network layer, wherein each node in the tree network topology structure is a neural network corresponding to a different task, and a leaf node in the tree network topology structure is a neural network that has completed training for a specified task, and the specified task includes identifying the gender of a target and identifying the age of a target; Based on the task attributes of each node in the current network layer, cluster analysis is performed on each node in the current network layer, and common parts of the task attributes of multiple nodes in the same category are extracted as the task attributes of the parent node of the multiple nodes, where the task attribute of the parent node is detection of human body or face area; Based on the task attributes of each parent node, train the network parameters of each parent node; After completing the training of each node in each network layer, it is determined that the training of the deep neural network corresponding to the tree network topology structure is completed, and the deep neural network is used to identify the gender and age of the target.

2. The method according to claim 1, characterized in that The clustering analysis of each node in the current network layer based on the task attributes of each node in the current network layer, and extracting the common parts of the task attributes of multiple nodes in the same category as the task attributes of the parent nodes of the multiple nodes, includes: According to the task attributes of each node in the current network layer, a similarity measurement matrix corresponding to the task attributes of each node is generated by a preset algorithm; According to the similarity measurement matrix, multiple nodes whose similarity is greater than a preset threshold are determined to be of the same category; The common parts of the task attributes of multiple nodes in the same category are extracted as the task attributes of the parent nodes of the multiple nodes.

3. The method according to claim 1, characterized in that The training of network parameters of each parent node based on the task attributes of each parent node includes: For any parent node, based on the task attributes of the parent node, the output features of each child node of the parent node are used as the input of the parent node to train the network parameters of the parent node, thereby completing the training of the parent node.

4. The method according to claim 1, characterized in that: The training of network parameters of each parent node based on the task attributes of each parent node includes: For any parent node, based on the task attributes of the parent node, the parent node is generated using a preset structure with a feature signal control mechanism. The output features of each child node of the parent node are used as the input of the parent node, and the feature signals related to the task attributes are selected, and the feature signals irrelevant to the task attributes are eliminated. The weighted combination of the feature signals related to the task attributes is obtained as the network parameters of the neural network corresponding to the parent node, and the training of the parent node is completed.

5. The method according to claim 1, characterized in that After completing the training of each node in each network layer, determining that the deep neural network training corresponding to the tree network topology structure is completed includes: Starting from the network layer where the leaf node is located, the network parameters of the nodes in each network layer are trained in sequence from the bottom layer to the top layer to complete the training of each node; After the training of each node at the top level is completed, it is determined that the training of the deep neural network corresponding to the tree network topology structure is completed.

6. A deep neural network training device, characterized in that: The device is a core processing chip with data processing capability, and the device comprises: An acquisition module, for acquiring, for a current network layer in a tree network topology, a task attribute of each node in the current network layer, wherein each node in the tree network topology is a neural network corresponding to a different task, and a leaf node in the tree network topology is a neural network that has been trained for a specified task, wherein the specified task includes identifying the gender of a target and identifying the age of a target; A clustering module, configured to perform cluster analysis on each node in the current network layer based on the task attributes of each node in the current network layer, extract the common parts of the task attributes of multiple nodes in the same category as the task attributes of the parent node of the multiple nodes, wherein the task attribute of the parent node is the detection of human body or face area; The training module is used to train the network parameters of each parent node based on the task attributes of each parent node; The determination module is used to determine that the training of the deep neural network corresponding to the tree network topology structure is completed after the training of each node in each network layer is completed, and the deep neural network is used to identify the gender and age of the target.

7. The device according to claim 6, characterized in that The clustering module is specifically used for: According to the task attributes of each node in the current network layer, a similarity measurement matrix corresponding to the task attributes of each node is generated by a preset algorithm; According to the similarity measurement matrix, multiple nodes whose similarity is greater than a preset threshold are determined to be of the same category; The common parts of the task attributes of multiple nodes in the same category are extracted as the task attributes of the parent nodes of the multiple nodes.

8. The device according to claim 6, characterized in that The training module is specifically used for: For any parent node, based on the task attributes of the parent node, the output features of each child node of the parent node are used as the input of the parent node to train the network parameters of the parent node, thereby completing the training of the parent node.

9. The device according to claim 6, characterized in that The training module is specifically used for: For any parent node, based on the task attributes of the parent node, the parent node is generated using a preset structure with a feature signal control mechanism. The output features of each child node of the parent node are used as the input of the parent node, and the feature signals related to the task attributes are selected, and the feature signals irrelevant to the task attributes are eliminated. The weighted combination of the feature signals related to the task attributes is obtained as the network parameters of the neural network corresponding to the parent node, and the training of the parent node is completed.

10. The device according to claim 6, characterized in that The determining module is specifically used for: Starting from the network layer where the leaf node is located, the network parameters of the nodes in each network layer are trained in sequence from the bottom layer to the top layer to complete the training of each node; After the training of each node at the top level is completed, it is determined that the training of the deep neural network corresponding to the tree network topology structure is completed.

11. A computer device, characterized in that: The device is a core processing chip with data processing capability, and includes a processor and a memory, wherein: The memory is used to store computer programs; The processor is used to implement the method steps described in any one of claims 1-5 when executing the program stored in the memory.

Images