Unlock instant, AI-driven research and patent intelligence for your innovation.

Word vector training method, system, device and medium based on multi-task learning

A technology of multi-task learning and training methods, which is applied in the fields of instruments, computing, and electrical digital data processing, etc., and can solve problems such as inability to express word similarity, word replacement interference, and high computational complexity

Active Publication Date: 2020-10-27
SOUTH CHINA UNIV OF TECH
View PDF8 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0008] 1) Each word vector has a high dimension and is very sparse, which leads to too much computational complexity and low efficiency in the calculation of subsequent tasks
[0009] 2) The similarity between words cannot be expressed, and the similarity between different words cannot be obtained through the calculation of word vectors. The method based on SVD decomposition has the following disadvantages:
[0012] 3) The complexity of the training process is too high
[0019] 1) Ignoring the impact of multiple external actual tasks on word vector training, word vectors may not be able to achieve better test results in multiple external tasks
[0020] 2) The robustness of the word vector is low, and it is more sensitive to human interference such as word replacement

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Word vector training method, system, device and medium based on multi-task learning
  • Word vector training method, system, device and medium based on multi-task learning
  • Word vector training method, system, device and medium based on multi-task learning

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0071] This embodiment provides a word vector training method based on multi-task learning, which is realized by using a word vector dictionary, a language model module and a named entity recognition module. The specific descriptions of the word vector dictionary, language model module and named entity recognition module are as follows :

[0072] 1) The input of the word vector dictionary is the one-hot vector of the word to be queried, and the output is the word vector representation of the word. The word vector dictionary is actually a dictionary matrix. For the input one-hot vector, the index value is 1 Perform a query to get the word vector representation of the word. The principle of the word vector dictionary is as follows figure 1 shown.

[0073] 2) The language model module is the first external practical task, that is, to establish a language model. The language model refers to the probability of occurrence of a word sequence. For example, the probability of occurren...

Embodiment 2

[0105] Such as Figure 9 As shown, the present embodiment provides a word vector training system based on multi-task learning, the system includes an acquisition unit 901, a construction unit 902 and a training unit 903, and the specific functions of each unit are as follows:

[0106] The acquiring unit 901 is configured to acquire a training set; wherein, the training set includes paired data of text word sequence-named entity tag sequence.

[0107] The building unit 902 is configured to build a language model module and a named entity recognition module, and use the language model module and the named entity recognition module as external modules.

[0108] The training unit 903 is used to alternately train the word vector dictionary and the external module; wherein, the word vector dictionary uses the text word sequence and the output of the external module for training, and the language model module uses the word vector dictionary training output of the word vector The nam...

Embodiment 3

[0111] This embodiment provides a computer device, which may be a server, a computer, etc., such as Figure 10 As shown, it includes a processor 1002 connected through a system bus 1001, a memory, an input device 1003, a display 1004 and a network interface 1005. The processor is used to provide calculation and control capabilities. The memory includes a non-volatile storage medium 1006 and internal Memory 1007, the non-volatile storage medium 1006 stores an operating system, computer programs and databases, the internal memory 1007 provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium, and the processor 1002 executes memory storage During computer program, realize the word vector training method of above-mentioned embodiment 1, as follows:

[0112] Obtain a training set; Wherein, the training set includes paired data of text word sequence-named entity tag sequence;

[0113] Build the language model module a...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The present invention discloses a word vector training method, system, equipment and medium based on multi-task learning. The method includes: obtaining a training set, the training set including paired data of text word sequence-named entity tag sequence; constructing language The model module and the named entity recognition module use the language model module and the named entity recognition module as external modules; the word vector dictionary and the external module are alternately trained, and the word vector dictionary is trained using the text word sequence and the output of the external module, and the language The model module uses the word vector sequence output from word vector dictionary training for training, and the named entity recognition module uses the word vector sequence and named entity tag sequence output from word vector dictionary training for training. The invention can improve the test effect of the word vector in multiple external actual tasks and enhance the robustness of the word vector representation.

Description

technical field [0001] The invention relates to a word vector training method, system, equipment and medium based on multi-task learning, belonging to the field of word vector training. Background technique [0002] The representation of word vectors is one of the most basic tasks in natural language processing. It is the representation of each word processed by a computer. Conceptually, it is a mathematical embedding of each word in a vector space. The representation of word vectors includes techniques such as language modeling and feature learning, and the purpose is to solve how to represent the mapping process of words or phrases from the vocabulary to the vector space. [0003] The representation of word vectors can be mainly divided into three methods: [0004] 1) One-hot vector representation: Assuming that there are a total of n words in the lexicon, each word vector is represented as a 1*n high-dimensional vector, and each word will have an index value of 1, and a...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Patents(China)
IPC IPC(8): G06F40/295G06F40/247
CPCG06F40/247G06F40/295
Inventor 庄浩杰王聪孙庆华
Owner SOUTH CHINA UNIV OF TECH