Construction method and device of deep neural network model, medium and electronic equipment

A neural network model and deep neural network technology, applied in the computer field, can solve problems such as difficult acquisition, poor model training effect, generalization of neural network model features, etc., to achieve high-quality results

Active Publication Date: 2019-11-08
JIANGSU MANYUN SOFTWARE TECH CO LTD
View PDF6 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

On the one hand, if unlabeled data is used as training samples, although there are many sample data, the characteristics of the trained neural network model are relatively generalized, and cannot provide high-quality output results for specific problems.
On the other hand, if labeled data is used as training samples, due to the relatively scarce amount of data and the difficulty in obtaining it, it may result in insufficient training of the model, resulting in a poor training effect of the model.

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Construction method and device of deep neural network model, medium and electronic equipment
  • Construction method and device of deep neural network model, medium and electronic equipment
  • Construction method and device of deep neural network model, medium and electronic equipment

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0047] figure 1 It is a flow chart of the method for constructing a deep neural network model provided in Embodiment 1 of the present application. This embodiment is applicable to situations such as model training. The method can be run by the device for constructing a deep neural network model provided in the embodiment of the present application. The device can be implemented by software and / or hardware, and can be integrated into electronic devices with computing functions for model training, such as smart terminals and servers.

[0048] Such as figure 1 Shown, the construction method of described depth neural network model comprises:

[0049] S110. Input the labeled sample data into the first neural network model to obtain the feature representation of the labeled sample data; wherein, the first neural network model trains the parameters of the network structure of the first neural network model according to the unlabeled sample data owned.

[0050] Among them, the labele...

Embodiment 2

[0064] In order to enable those skilled in the art to understand the technical solution disclosed in the application more clearly, the application also provides a preferred implementation manner.

[0065] For the following shortcoming that prior art exists:

[0066] 1. The training of the neural network model requires a large amount of labeled data, which is relatively rare and difficult to obtain, so the corresponding effect is poor;

[0067] 2. Different natural language tasks need to train their own neural network models, which takes a long time;

[0068] 3. The number of hyperparameters of the deep learning model is huge, especially the embedding matrix, which takes up a lot of memory and consumes a lot of resources.

[0069] The present invention gathers unlabeled data related to all natural language processing tasks in reality to train a pre-training model, which solves the problem that all the fine-tuning processes are very short in addition to the long time-consuming ...

Embodiment 3

[0076] image 3 It is a schematic structural diagram of a device for constructing a deep neural network model provided in Embodiment 3 of the present application. Such as image 3 As shown, the construction device of the deep neural network model includes:

[0077] The feature representation acquisition module 310 is used to input the labeled sample data into the first neural network model to obtain the feature representation of the labeled sample data; wherein, the first neural network model is based on the unlabeled sample data to the first neural network model The parameters of the network structure are trained;

[0078] The parameter training module 320 is configured to input the feature representation and label data of the labeled sample data into the second neural network model, so as to train the parameters of the second neural network model.

[0079] In the technical solution provided by the embodiment of the present application, the labeled sample data is input int...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The embodiment of the invention discloses a construction method and a device of a deep neural network model, a medium and electronic equipment. The method comprises the steps of inputting labeled sample data into a first neural network model to obtain feature representation of the labeled sample data; wherein the first neural network model is obtained by training parameters of a network structureof the first neural network model according to the label-free sample data; and inputting the feature representation of the labeled sample data and the labeled data into a second neural network model so as to train parameters of the second neural network model. By operating the construction method provided by the invention, the purposes of ensuring a high-quality model training effect and labelingmore data without consuming a large amount of human resources under the condition of limited labeled training sample resources can be achieved.

Description

technical field [0001] The embodiments of the present application relate to the field of computer technology, and in particular to a method, device, medium, and electronic equipment for constructing a deep neural network model. Background technique [0002] With the rapid development of technology, understanding the real intention of users has become the new standard of intelligence. In the process of processing natural language, it is often necessary to build a neural network model to realize the processing of natural language text implication, intelligent question and answer, semantic similarity judgment, and text classification to obtain the user's true intention. [0003] However, the current neural network model construction process often has two problems. On the one hand, if unlabeled data is used as training samples, although there are many sample data, the characteristics of the trained neural network model are relatively generalized, and cannot provide high-quality...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
IPC IPC(8): G06N3/08G06N3/04G06F16/35G06F16/332G06F16/33
CPCG06N3/08G06N3/088G06F16/355G06F16/353G06F16/3329G06F16/3344G06N3/045
Inventor 陶超沙韬伟邓金秋
Owner JIANGSU MANYUN SOFTWARE TECH CO LTD
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap