Progressive supervised deep learning neural network training method, system, medium and device

Abstract

The invention discloses a progressive supervision deep learning neural network training method, system, medium and equipment. The method includes: establishing an initial deep learning neural network; using the classifier layer of the initial deep learning neural network as the current training layer to be supervised ; Use the labeled training input data as the input data of the initial deep learning neural network, and perform top-down supervised training on the current unsupervised training layer; when the supervised training fails, the current unsupervised training layer and its previous hidden layer As the current to-be-supervised training layer, perform top-down supervised training on the current to-be-supervised training layer; when the supervised training is successful, output the current deep learning neural network. The invention corrects unnecessary characteristic details not eliminated during unsupervised training through supervised training, so as to achieve the effect of sufficient fitting but not overfitting, thereby improving the training success rate of sample data and test data.

Description

technical field

[0001] The invention relates to a deep learning neural network training method, in particular to a progressively supervised deep learning neural network training method, system, medium and equipment, belonging to the field of neural network training. Background technique

[0002] Existing deep learning technology can obtain output labels through input data (such as obtaining the ID card number of the person through the avatar, or obtaining the ID card number of the person through voice). Supervised training (such as avatars with ID number labels, and speech with ID number labels).

[0003] However, the top-down supervised training of existing deep learning techniques either only adjusts the network weights between the output layer and the hidden layer, or adjusts the network weights of all layers. When there are more categories of top-level concepts than labels, if only the weights of the classifier network between the output layer and the hidden layer are a...

Examples

Embodiment 1

[0066] The description and establishment process of deep learning is as follows:

[0067] The calculations involved in producing an output from an input can be represented by a flow graph: a flow graph is a graph that can represent calculations, in which each node represents a basic calculation and a calculation The value, the result of the calculation is applied to the value of this node's child nodes. Consider a collection of computations that are allowed at every node and possible graph structure, and define a family of functions. Input nodes have no parents, and output nodes have no children.

[0068] A special property of such flow graphs is depth: the length of the longest path from an input to an output.

[0069] Considering the learning structure as a network, the core idea of ​​deep learning is as follows:

[0070] Step 1: Adopt bottom-up unsupervised training

[0071] 1) Construct a single layer of neurons layer by layer.

[0072] 2) Each layer is tuned using th...

Embodiment 2

[0113] like Figure 9 As shown, the present embodiment provides a progressive supervised deep learning neural network training system, the system includes a building module 901, an acquisition module 902, a first supervised training module 903, a second supervised training module 904 and an output module 905, each The specific functions of the module are as follows:

[0114] The establishment module 901 is used to establish an initial deep learning neural network; wherein, the initial deep learning neural network includes an input layer, a hidden layer, a classifier and an output layer, and the module is as follows Figure 10 shown, including:

[0115] An acquisition unit 9011, configured to acquire unlabeled training input data;

[0116] The unsupervised training unit 9012 is used to use the unlabeled training input data as the input data of the deep neural network to perform bottom-up unsupervised training to obtain the initial deep learning neural network.

[0117] The o...

Embodiment 3

[0130] This embodiment provides a storage medium, the storage medium stores one or more programs, and when the programs are executed by the processor, the progressive supervision deep learning neural network training method of the above-mentioned embodiment 1 is implemented, as follows:

[0131] An initial deep learning neural network is set up; wherein, the initial deep learning neural network includes an input layer, a hidden layer, a classifier and an output layer;

[0132] Use the classifier layer of the initial deep learning neural network as the current training layer to be supervised;

[0133] Use the labeled training input data as the input data of the initial deep learning neural network, and perform top-down supervised training on the current training layer to be supervised;

[0134] When the supervised training fails, the current to-be-supervised training layer and its previous hidden layer are used as the current to-be-supervised training layer, and top-down superv...

technical field

[0001] The invention relates to a deep learning neural network training method, in particular to a progressively supervised deep learning neural network training method, system, medium and equipment, belonging to the field of neural network training. Background technique

[0002] Existing deep learning technology can obtain output labels through input data (such as obtaining the ID card number of the person through the avatar, or obtaining the ID card number of the person through voice). Supervised training (such as avatars with ID number labels, and speech with ID number labels).

[0003] However, the top-down supervised training of existing deep learning techniques either only adjusts the network weights between the output layer and the hidden layer, or adjusts the network weights of all layers. When there are more categories of top-level concepts than labels, if only the weights of the classifier network between the output layer and the hidden layer are a...

Examples

Embodiment 1

[0066] The description and establishment process of deep learning is as follows:

[0067] The calculations involved in producing an output from an input can be represented by a flow graph: a flow graph is a graph that can represent calculations, in which each node represents a basic calculation and a calculation The value, the result of the calculation is applied to the value of this node's child nodes. Consider a collection of computations that are allowed at every node and possible graph structure, and define a family of functions. Input nodes have no parents, and output nodes have no children.

[0068] A special property of such flow graphs is depth: the length of the longest path from an input to an output.

[0069] Considering the learning structure as a network, the core idea of ​​deep learning is as follows:

[0070] Step 1: Adopt bottom-up unsupervised training

[0071] 1) Construct a single layer of neurons layer by layer.

[0072] 2) Each layer is tuned using th...

Embodiment 2

[0113] like Figure 9 As shown, the present embodiment provides a progressive supervised deep learning neural network training system, the system includes a building module 901, an acquisition module 902, a first supervised training module 903, a second supervised training module 904 and an output module 905, each The specific functions of the module are as follows:

[0114] The establishment module 901 is used to establish an initial deep learning neural network; wherein, the initial deep learning neural network includes an input layer, a hidden layer, a classifier and an output layer, and the module is as follows Figure 10 shown, including:

[0115] An acquisition unit 9011, configured to acquire unlabeled training input data;

[0116] The unsupervised training unit 9012 is used to use the unlabeled training input data as the input data of the deep neural network to perform bottom-up unsupervised training to obtain the initial deep learning neural network.

[0117] The o...

Embodiment 3

[0130] This embodiment provides a storage medium, the storage medium stores one or more programs, and when the programs are executed by the processor, the progressive supervision deep learning neural network training method of the above-mentioned embodiment 1 is implemented, as follows:

[0131] An initial deep learning neural network is set up; wherein, the initial deep learning neural network includes an input layer, a hidden layer, a classifier and an output layer;

[0132] Use the classifier layer of the initial deep learning neural network as the current training layer to be supervised;

[0133] Use the labeled training input data as the input data of the initial deep learning neural network, and perform top-down supervised training on the current training layer to be supervised;

[0134] When the supervised training fails, the current to-be-supervised training layer and its previous hidden layer are used as the current to-be-supervised training layer, and top-down superv...

Images