A neural network language model training method, device, equipment and storage medium
A language model and neural network technology, applied in speech analysis, speech recognition, instruments, etc., can solve the problem of data sparseness that cannot be effectively solved, and achieve the effects of shortening training time, improving performance, and reducing complexity
Active Publication Date: 2021-10-22
TENCENT TECH (SHENZHEN) CO LTD
View PDF17 Cites 0 Cited by
- Summary
- Abstract
- Description
- Claims
- Application Information
AI Technical Summary
Problems solved by technology
Although the smoothing technology is introduced, the problem of data sparsity cannot be effectively solved
Method used
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View moreImage
Smart Image Click on the blue labels to locate them in the text.
Smart ImageViewing Examples
Examples
Experimental program
Comparison scheme
Effect test
Embodiment Construction
[0043] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiment of the application will be clearly and completely described below in conjunction with the accompanying drawings in the embodiment of the application. Obviously, the described embodiment is only It is a part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by persons of ordinary skill in the art without making creative efforts belong to the scope of protection of this application.
[0044] The terms "first", "second", "third", "fourth", etc. (if any) in the specification and claims of the present application and the above drawings are used to distinguish similar objects, and not necessarily Used to describe a specific sequence or sequence. It is to be understood that the data so used are interchangeable under appropriate circumstances such th...
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More PUM
Login to View More Abstract
The present application discloses a model training method, including: obtaining multiple sets of batch processing data for model training; and training the same neural network in parallel based on different batch processing data through multiple computing nodes running on multiple graphics processors. Network language model; in the current round of iteration, the gradient of each parameter is determined by each of the multiple computing nodes; the gradient of each parameter is summarized to obtain the global gradient of each parameter, and multiple computing nodes are controlled to synchronize parameters based on the global gradient Update; through the first computing node to perform performance evaluation on the neural network language model after the current round of update based on the test data set to obtain the current language model performance index value; broadcast the current language model performance index value through the first computing node; according to the current language model The relationship between the performance index value and the language model performance index value determined in the previous round controls whether multiple computing nodes stop training. The present application also discloses a corresponding system.
Description
technical field [0001] The present application relates to the technical field of machine learning, and in particular to a neural network language model training method, device, equipment and storage medium. Background technique [0002] Language model is a basic problem in the field of natural language processing, and it plays an important role in tasks such as part-of-speech tagging, syntactic analysis, machine translation, and information retrieval. In short, the statistical language model is expressed as: in a word sequence, given a word and all words in the context, the probability of this sequence appearing. [0003] Language models are a fundamental part of many systems that attempt to solve natural language processing tasks such as machine translation and speech recognition. Before the neural network (Neural Network, NN) was successfully applied to language modeling, the mainstream language model was the N-gram model, which used counting statistics to represent the d...
Claims
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More Application Information
Patent Timeline
Login to View More IPC IPC(8): G10L15/06G10L15/16
CPCG10L15/063G10L15/16
Inventor 黄羿衡
Owner TENCENT TECH (SHENZHEN) CO LTD