Multi-information fusion word vector generation method for medical consultation scenario dialogue analysis

Abstract

The application provides a multi-information fusion word vector generation method for medical consultation scene dialogue analysis, comprising the following steps: 1, preprocessing the original data to establish an original corpus; 2, using a new dictionary to perform word segmentation processing on the original corpus to obtain a word part-of-speech sequence and a character part-of-speech sequence; 3, using one-hot coding to encode the word part-of-speech sequence and the character part-of-speech sequence into 50-dimensional part-of-speech vectors to obtain a word part-of-speech vector sequence and a character part-of-speech vector sequence; 4, introducing a position weighting vector; 5, fusing the word part-of-speech vector sequence and the character part-of-speech vector sequence to obtain a word character fusion vector; 6, for a given sentence, predicting a target word fusion vector; 7, using a Huffman tree to perform gradient ascent iteration until the gradient converges, thereby obtaining a multi-information fusion word vector.

Description

TECHNICAL FIELD

[0001] The application belongs to the field of natural language processing, and relates to a multi-information fusion word vector generation method for medical consultation scene dialogue analysis. BACKGROUND

[0002] In recent years, with the continuous deepening of artificial intelligence research, natural language processing technology has developed rapidly. As one of the most basic problems in the field of natural language processing, the digital expression of text actually affects the performance of subsequent machine learning algorithms. Based on the large number of Chinese words, we hope that computers can learn as many digital representations of words as possible from a large amount of unlabeled text data, and the trained word vectors can contain as much information as possible.

[0003] For the medical consultation scene, the dialogue of the doctor consultation contains many special medical professional terms, such as green pyocyanin pneumonia, compound gingko Tongmai oral liquid, etc. These words are rarely contained in daily dialogue texts and have low word frequency, so the current word vectors trained based on common dialogue and article corpus are often difficult to adapt. And with the continuous development of medical technology, similar professional terms are still increasing rapidly, and the simple word-level word vector training method is difficult to meet the analysis of medical consultation scene corpus, so a word vector training method containing multiple dimension information is needed. SUMMARY

[0004] The application aims to provide a multi-information fusion word vector generation method for medical consultation scene dialogue analysis to meet the needs of medical consultation scene corpus analysis.

[0005] To achieve the above-mentioned purpose, the application provides a multi-information fusion word vector generation method for medical consultation scene dialogue analysis, and the specific steps are as follows:

[0006] Step 1, collect and obtain original consultation dialogue text corpus, perform data preprocessing, and establish an original corpus library;

[0007] Step 2, establish a medical special dictionary containing word features to supplement the common dictionary of the word segmentation tool, use the new dictionary to perform word segmentation processing on the original corpus library, and obtain word part-of-speech sequence (W_P) and character part-of-speech sequence (C_P);

[0008] Step 3, encode the word part-of-speech sequence (W_P) and the character part-of-speech sequence (C_P) into 50-dimensional part-of-speech vectors using one-hot encoding to obtain word part-of-speech vector sequence w W_P =(w1,w2,…,w n ) and character part-of-speech vector sequence c C_P =(c1,c2,…,c m );

[0009] Step 4, introducing a position weighted vector where 2k is the window size, is the decreasing amount of weight value with distance;

[0010] Step 5, fusing the obtained sequence w W_P word vector and character vector of each word in the given sentence D to obtain a word and character fused vector;

[0011] Step 6, for a given sentence D = {x1, x2, …, x n}, a loop is performed to make each word as a target word, and then the context is predicted for the target word, and the fused vector of the predicted target word is obtained by combining the position weight given in step 4:

[0012]

[0013] where 1 2k is a full 1 vector with a dimension of 2k, if the target word is close to the edge of the sentence so that the window exceeds the range of the sentence, the missing word vector in the window is assigned as a 0 vector, and 1 2k the value 1 in the corresponding position is also assigned as 0;

[0014] Step 7, using the word frequency as the weight to establish a Huffman tree, and stipulating that the weight of the left subtree is not less than the weight of the right subtree, and randomly initializing all model parameters θ. The gradient iteration step is specified as η.

[0015] Step 8, gradient ascent iteration is performed, and each target word w in the training set is used to update the Huffman tree node parameters and the word vector in the window:

[0016]

[0017]

[0018] where j = 1, 2, … l w -1, l w is the total number of nodes contained from the root node to the leaf node where w is located, represents the parameter of the jth node in the path from the root node to the node where w is located.

[0019] Step 9, repeating step 8 until the gradient converges, to obtain a multi-information fused word vector.

[0020] As a preferred, the pre-processing method in step 1 is to correct the wrong characters in the text information against the medical database, and remove the stop words.

[0021] As a preferred, the specific method of step 2 is: for a given sentence D nWherein n is the number of words in the sentence, the word sequence (W) is obtained by word segmentation, and the word sequence is obtained by integrating the word corresponding to the word into the sequence to obtain the word sequence (W_P) of the word.

[0022] As preferred, the coding method in step 3 is: using the obtained word sequence (W_P) to insert the corresponding word vector into each word vector in the sequence, and finally generating a 150-dimensional word vector sequence w W_P =(w1,w2,…,w n ), wherein n is the number of words in the word; similarly, the corresponding word vector is inserted into each Chinese character vector of the character word sequence (C_P) to obtain the character word vector sequence c C_P =(c1,c2,…,c m ) of each word, wherein m is the number of words in the word.

[0023] As preferred, the vector representation w,c of each word and Chinese character is randomly generated before training, and the dimension is 100 dimensions, and the numerical value of each dimension is limited to 0 to 1.

[0024] As preferred, the expression of the word vector fusion vector in the case of a word composed of multiple characters is as follows:

[0025]

[0026] Wherein β represents the proportion of the word vector in the joint training of the word, and 0≤β≤1.

[0027] As preferred, the expression of the word vector fusion vector in the case of a word composed of a single character is as follows:

[0028] x i =(1-β)w i +βc S

[0029] Wherein β represents the proportion of the word vector in the joint training of the word, and 0≤β≤1.

[0030] The beneficial effects of the present application are:

[0031] The present application uses the common context-based word vector generation method and the context-based word vector generation method, and fuses the position of Chinese characters in the word, the word information and the like, to help obtain a Chinese word vector with multi-dimensional information.

[0032] In addition, the data enhancement technology is used to fuse various word information, so that the model can improve the generalization ability and prevent overfitting, and thus a new word vector generation method is provided for the dialogue analysis in the medical consultation scene. BRIEF DESCRIPTION OF DRAWINGS

[0033] Figure 1 A principle diagram of the medical consultation scene dialogue analysis multi-information fusion word vector generation method of the present application;

[0034] Figure 2 A specific flowchart of the medical consultation scene dialogue analysis multi-information fusion word vector generation method of the present application;

[0035] Figure 3 A gradient iterative training schematic diagram;

[0036] Figure 4 A schematic diagram of the evaluation results of the traditional word vector generation method and the multi-information fusion word vector generation method in the analogy reasoning task. DETAILED DESCRIPTION

[0037] The present application will be further described in detail below with reference to the accompanying drawings:

[0038] The present embodiment provides a medical consultation scene dialogue analysis multi-information fusion word vector generation method, as shown in the accompanying drawings, and the specific implementation method comprises the following steps: Figure 1

[0039] Step 1. Collect and obtain the original medical consultation dialogue text corpus, and perform data preprocessing to establish the original corpus. A special part-of-speech dictionary is established to supplement the common dictionary of the word segmentation tool, and the original corpus is segmented by using the new dictionary. For a given sentence D n where n is the number of words in the sentence, the word sequence (W) is obtained by word segmentation, and the part-of-speech corresponding to the word is integrated into the sequence to obtain the word part-of-speech sequence (W_P). The word part-of-speech sequence is cut into characters, and the part-of-speech is assigned to each character in the word to obtain the character part-of-speech sequence (C_P) of each word. The position of each character in the word is obtained, and is respectively denoted as B, M, and E, indicating that it is at the beginning, middle, and end of the word, and S represents that the character is a whole word.

[0040] Randomly generate the vector representation w,c of each word and Chinese character, with a dimension of 100, and limit the numerical value of each dimension to between 0 and 1.

[0041] Step 2. The part-of-speech is encoded by one-hot to a part-of-speech vector of 50 dimensions. The word part-of-speech sequence (W_P) obtained in step 1 is inserted into the corresponding part-of-speech vector after each word vector in the sequence, and finally a word part-of-speech vector sequence w W_P ​= (w1, w2, …, w n ). Using the same method, the corresponding part-of-speech vector is inserted into each Chinese character vector of the character part-of-speech sequence (C_P) to obtain the character part-of-speech vector sequence c C_P = (c1, c2, …, c m ) of each word, where m is the number of characters in the word.

[0042] Step 3. Introducing a position weighting vector where 2k is the window size, i.e., only the words within a distance of k on the left and right of the current word have a direct impact on the vector generation of the current word, is the decreasing amount of weight with distance, and here we assume that the closer the distance, the greater the impact of the word on the current word.

[0043] Step 4. Fusing the word vector and character vector of each word in the sequence w W_P obtained in step 2 to obtain the word-character fusion vector:

[0044]

[0045] or:

[0046] x i = (1-β)w i +βc S ,

[0047] where β represents the proportion of character vectors in word-character joint training, and 0≤β≤1.

[0048] Step 5. For a given sentence D = {x1, x2, …, x n}, cycle each word as the target word, and use the context to predict the target word, and combine the position weight given in step 3 to obtain the fusion vector of the predicted target word as:

[0049]

[0050] where 1 2k is a full 1 vector with a dimension of 2k, if the target word is close to the edge of the sentence so that the window exceeds the sentence range, then the missing fusion word vector in the window is assigned as a 0 vector, and the value 1 in the corresponding position of 1 2k is also assigned as 0.

[0051] Step 6. As shown in Figure 3 , a Huffman tree is established using the word frequency as the weight, it is specified that the weight of the left subtree is not less than that of the right subtree, and all model parameters θ in the Huffman tree are randomly initialized. Using Hierarchical Softmax optimization calculation, gradient iteration is performed using gradient ascent method, and the specified step size is η.

[0052] It can be understood that each node of the neural network established by using the structure of the Huffman tree is a parameter of the neural network.

[0053] Step 7. Gradient ascent iteration is performed to update the Huffman tree node parameters and the word vector in the window by using each target word w in the training set:

[0054]

[0055]

[0056] wherein j = 1, 2,..., l w -1, l w is the total number of nodes contained in the Huffman tree from the root node to the leaf node where w is located, represents the parameter of the jth node in the path from the root node to the node where w is located.

[0057] Step 8. Step 7 is repeated until the gradient converges, and the multi-information fusion word vector is obtained.

[0058] In order to better illustrate the effectiveness of the method, as Figure 4 , the same analogy reasoning task is performed by using the traditional word vector generation methods Word2Vec and Glove and the multi-information fusion word vector generation method of the present application, and it can be seen from the results that the word vector generated by the method of the present application contains more semantic information.

Claims

1. A multi-information fusion word vector generation method for dialogue analysis in medical consultation scenarios, characterized in that, Includes the following steps: Step 1: Collect and obtain the original consultation dialogue text data, perform data preprocessing, and establish the original corpus; Step 2: Establish a dedicated dictionary containing parts of speech to supplement the commonly used dictionaries of word segmentation tools. Use the new dictionary to perform word segmentation on the original corpus to obtain word part-of-speech sequences and character part-of-speech sequences. Step 3: Encode the word part-of-speech sequence and the character part-of-speech sequence into 50-dimensional part-of-speech vectors using one-hot encoding to obtain the word part-of-speech vector sequence and the character part-of-speech vector sequence; The encoding method is as follows: the obtained word part-of-speech sequence is used to insert the corresponding part-of-speech vector after each word vector in the sequence, and finally a 150-dimensional word part-of-speech vector sequence is generated. Similarly, the corresponding part-of-speech vector is inserted into each Chinese character vector in the character part-of-speech sequence to obtain the character part-of-speech vector sequence of each word. Step 4: Introduce a position-weighted vector with dimensions equal to the set window size; When introducing position-weighted vectors, word vectors related to the target word are fused based on word distance weighting, with words closer to the current word having a larger weight. Step 5: Merge the word part-of-speech vector sequence and the character part-of-speech vector sequence to obtain the word-word fusion vector; The fusion method is as follows: The word vector and character vector of each word in the word part-of-speech vector sequence obtained in step 3 are fused to obtain the word-character fusion vector. Where β represents the proportion of character vectors in joint character and word training, and 0≤β≤1, the position of each character in the word is denoted as B, M, and E respectively; Step 6: For a given sentence, loop through each word to make it a target word, then use the context to predict the target word, and combine it with the position weights given in Step 4 to obtain the fusion vector of the predicted target word; Step 7: Use word frequency as weights to build a Huffman tree, and set the weight of the left subtree in the built Huffman tree to be no less than the weight of the right subtree. Step 8: Iteratively update the Huffman tree node parameters and the fusion vector of words within the window using gradient ascent; Step 9: Repeat step 8 until the gradient converges, and obtain the word vector fused with multiple information.

2. The multi-information fusion word vector generation method for dialogue analysis in medical consultation scenarios according to claim 1, characterized in that, The preprocessing method in step 1 is as follows: correct typos in the text information by comparing it with the medical database and remove stop words.

3. The multi-information fusion word vector generation method for dialogue analysis in medical consultation scenarios according to claim 1, characterized in that, The specific method of step 2 is as follows: for a given sentence, segment the words to obtain a word sequence, and integrate the parts of speech of the words into the sequence to obtain a word part-of-speech sequence; perform character segmentation on the word part-of-speech sequence, and assign the part of speech to each character in the word to obtain the character part-of-speech sequence of each word; obtain the position of each character in the word.

4. The multi-information fusion word vector generation method for dialogue analysis in medical consultation scenarios according to claim 3, characterized in that, Before training, vectors for each word and Chinese character are randomly generated, with a dimension of 100, and the value of each dimension is restricted to between 0 and 1.

Images