Chinese machine reading understanding method and device based on pinyin embedding enhancement and multilayer information fusion
By employing pinyin embedding and multi-layer information fusion, the challenges of word polysemy and contextual understanding in Chinese machine reading comprehension were addressed, thereby improving the model's performance.
Patent Information
- Application Number
- CN202410567560.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-05-09
- Publication Date
- 2024-11-15
- Estimated Expiration
- 2044-05-09
AI Technical Summary
In Chinese machine reading comprehension tasks, due to the complexity and ambiguity of the Chinese language, existing models are prone to errors when understanding Chinese text, and lack datasets, making it difficult to effectively capture word polysemy and contextual information.
We employ a method based on pinyin embedding enhancement and multi-layer information fusion. We obtain pinyin embedding vectors through convolutional networks, combine word embedding and position embedding of the BERT model, and perform multi-layer attention fusion to improve the model's ability to understand Chinese text.
It improves the performance of Chinese machine reading comprehension models, enabling them to capture word polysemy and contextual information more accurately, and enhancing the model's ability to handle complex linguistic phenomena.
Smart Images

Figure CN118964537A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to an extractive machine reading comprehension method in the field of natural language processing, and in particular to a Chinese extractive machine reading comprehension method and device based on pinyin embedding enhancement and multi-layer information fusion. Technical Background
[0002] Extractive machine reading comprehension tasks aim to enable the constructed model to find accurate answers from a given text, and these answers can usually be directly extracted from the original text without generating new content. This task usually involves understanding the question and text, as well as the ability to locate relevant information in a given text.
[0003] The research focus of neural network-based reading comprehension models is to use bidirectional encoding or attention mechanisms to fully interact information between questions and texts, focusing on the parts related to the questions, or using part-of-speech tagging, syntactic analysis, etc. to deeply understand the semantic information of questions and texts. However, these all rely on a large amount of annotated data, especially in specific fields, where large-scale annotated data is required to achieve good performance. In the field of English machine reading comprehension, due to the relatively simple structure of the English language and rich data set resources, many advanced models and technologies have been widely used and achieved good results. In contrast, in the Chinese machine reading comprehension task, due to the complexity and ambiguity of the Chinese language, as well as the lack of Chinese data sets, it is more challenging.
[0004] Chinese has a rich vocabulary and grammatical structure, with a large number of polyphones and ambiguous words, which makes it easier for the model to make mistakes when understanding Chinese text. In addition, the structure of Chinese sentences is flexible and needs to consider more contextual information, which increases the difficulty of the model's understanding. Therefore, for the task of Chinese machine reading comprehension, it is necessary to solve the polysemy of Chinese words and improve the model's ability to accurately understand polysemy and context. Therefore, developing a model that can alleviate the polysemy of Chinese words and capture more contextual information is of great significance for building applications based on machine reading comprehension. Summary of the invention
[0005] The present invention aims to overcome the above-mentioned shortcomings of the prior art and provide a Chinese machine reading comprehension method based on pinyin embedding enhancement and multi-layer information fusion.
[0006] Based on the pinyin information unique to Chinese and the different understandings of words at different layers of the model, the present invention fuses information of word level, low level and high level representation vectors to improve the performance of the machine reading comprehension model.
[0007] The present invention is realized by the following technical scheme: a Chinese machine reading comprehension method based on pinyin embedding enhancement and multi-layer information fusion, comprising the following steps:
[0008] Step 1: Preprocess the dataset into a triple consisting of context C, question Q and answer A. The dataset D can be specifically represented as {D i ∣D i =(C,Q,A),1≤i≤n}, where n represents the size of the data set D, D i Represents the i-th data in the data set, and the answer A is a continuous substring in the context C.
[0009] Step 2: Perform Chinese word segmentation on the data processed in step 1 to obtain the word sequence of the text, and pass the obtained Chinese word sequence through a convolutional network to obtain the pinyin embedding vector PE of the Chinese word;
[0010] Step 3: Based on the data obtained in step 1, the paragraphs and questions are concatenated by marking [CLS] and [SEP], and then input into BERT. Using BERT’s wordpiece Chinese word segmentation, the word embedding vector E of each word is obtained. token ;
[0011] Step 4: Pinyin embedding PE and word embedding E obtained in steps 2 and 3 respectively token , through a fully connected layer, we get a fused word embedding FE that integrates the pinyin embedding vector. Then we combine it with the segment embedding E of the input sequence. seg and position embedding E pos Add them together to get the embedding vector w of the complete input sequence;
[0012] Step 5: Perform BERT encoding based on the sequence embedding vector obtained in step 4, so that the encoding vector representation of the output of different encoding layers can be obtained;
[0013] Step 6: Based on the operation in step 5, perform multi-layer attention fusion calculation on the encoded representation obtained by the first six layers of encoders of the BERT model and the word embedding representation obtained in step 5 to obtain the context representation vector v that integrates the low-level question semantic information Cl , where C represents context and l represents low-level information fusion;
[0014] Step 7: Input the context encoding representation obtained in step 6 and the sixth-layer question encoding representation into the seventh-layer encoder, and obtain the encoding representations of the seventh to eleventh layers of encoders in turn. Then, perform multi-layer attention fusion calculation with the word embedding vector in step 5 and the low-level encoding representation obtained in step 6 to obtain the context representation vector v that integrates high-level question semantic information. Ch, where C represents context and h represents high-level information fusion;
[0015] Step 8: Based on the high-level context representation obtained in step 7 and the question representation vector of the eleventh layer encoder, they are input into the last encoder. Using the self-attention of the encoder, the final context semantic vector that integrates multiple layers of information is obtained. Where C represents the context, Q represents the question, m and n represent the length of the context and question respectively, and L 12 represents the encoded representation obtained by the twelfth layer encoder, and h represents the low-level encoding vector.
[0016] Step 9: Based on the sentence vector representation obtained in step 8, the softmax function is used to predict the probability distribution of each position as the starting point and ending point of the answer. Finally, all possible combinations of starting points and ending points are traversed, and the combination with the largest probability product is selected to determine the span of the answer.
[0017] Furthermore, the pinyin embedding obtained by using the convolutional network model in step 2 specifically refers to: assuming that the number of pinyins of Chinese characters is len, a pd-dimensional vector is randomly initialized for the pinyin of each Chinese character to form a pinyin matrix Convert Chinese text to pinyin sequence x=[x 1 ,x 2 ,…,x lx ], each pinyin sequence is mapped to the corresponding pinyin vector through PV, and the matrix px = [x 1 ,x 2 ,…,x lx ] where x i =PV(x i ),1≤i≤lx, represents pinyin x i The corresponding pinyin vector, lx represents the number of Chinese characters. By performing convolution operations on the pinyin sequence and then performing maximum pooling, the pinyin embedding representation PE is obtained.
[0018] Furthermore, the embedding vector w in step 4 can be expressed as w = FE + E seg +E pos , where FE = concat(E token ,PE)W fc +b fc , where W fc represents the weight of the fully connected layer, W fc represents the bias of the fully connected layer,
[0019] Furthermore, the encoding operation of the input sequence embedding using BERT in step 5 specifically refers to: using the encoder in BERT to model the semantic information of the words in the input sequence to obtain the encoding output of each layer of the input sequence. Given the input sequence embedding w, input it into the BERT model:
[0020]
[0021] Where C represents the context paragraph, Q represents the question, m and n represent the length of the context and question respectively, and L i represents the encoding representation obtained by the i-th layer encoder, 1≤i≤6, l represents the low-level encoding vector. Similarly, the encoding representation of the first six layers of encoders can be obtained.
[0022] Furthermore, the attention fusion calculation described in step 6 specifically refers to: defining two sets of vectors about context and question and Concatenate the word-level and first six-layer encoding vectors to fuse all low-level information in question Q into context C:
[0023]
[0024] Middle W C and W Q represents the learnable weight matrix, v represents the weight vector, and then a softmax is used to get the attention weight: Finally get Next, use BiLSTM to get and The representation vector is encoded to obtain the context representation vector of the low-level fusion information:
[0025] Furthermore, the high-level information fusion described in step 7 specifically refers to: obtaining Among them, h represents the high-level coding operation. Similarly, the coding representations of the seventh to eleventh layers can be obtained, and we get and The high-level semantic information in question Q is integrated into the context to obtain Similarly, BiLSTM is used to fusion vectors at low levels High-level fusion vector and Constructing contextual representations of high-level fused information:
[0026] Furthermore, the context semantic vector of the fusion of multi-layer information described in step 8 specifically refers to: using the last layer encoder of BERT to perform self-attention calculation on the high-level context representation to obtain the context encoding vector of all hierarchical information in the fusion problem:
[0027] Furthermore, the softmax function is used to predict the probability of each position as the starting and ending point as described in step 9. Specifically, if the input context length is M, the model will output P start and P end :
[0028]
[0029] Among them, s i and e i They represent the scores of the i-th position predicted by the model as the starting point and the ending point of the answer. Finally, the starting point and the ending point with the highest joint probability are selected: P joint (i,j)=P start (i)×P end (j) By traversing all possible combinations of starting and ending points, select joint The largest combination (i, j) is the final answer span.
[0030] After the above introduction, the loss function of the model adopts the cross entropy loss function, which is used to measure the difference between the answer predicted by the model and the actual answer. The specific calculation formula is as follows:
[0031]
[0032] Among them, y start and end The true label vectors represent the start and end positions of the actual answer. Finally, the weighted sum of the losses at the start and end positions is used to obtain the complete loss function:
[0033] Loss = λ·Loss start +(1-λ)·Loss end (4)
[0034] Among them, λ is a weight parameter used to balance the loss at the start and end positions.
[0035] The second aspect of the present invention relates to a Chinese machine reading comprehension device based on pinyin embedding enhancement and multi-layer information fusion, comprising a memory and one or more processors, wherein the memory stores executable code, and when the one or more processors execute the executable code, they are used to implement the Chinese machine reading comprehension method based on pinyin embedding enhancement and multi-layer information fusion of the present invention.
[0036] The third aspect of the present invention relates to a computer-readable storage medium having a program stored thereon, which, when executed by a processor, implements the Chinese machine reading comprehension method based on pinyin embedding enhancement and multi-layer information fusion of the present invention.
[0037] The beneficial effects of the present invention are:
[0038] The present invention uses the method of pinyin embedding enhancement and multi-layer information fusion to achieve the Chinese machine reading comprehension task. By using pinyin embedding to enhance the information of word vectors, the model can obtain more dimensional word-level information. In addition, through multi-layer information fusion, the input text is subjected to word-level, low-level vector representation, and high-level vector representation for attention fusion, so that the problem information is fully integrated into the context, so that the model can capture more comprehensive semantic information and improve the ability to handle complex language phenomena. The Chinese machine reading comprehension method based on pinyin embedding and multi-layer information fusion constructed by the present invention aims to achieve better performance by obtaining richer and more comprehensive interactive information. BRIEF DESCRIPTION OF THE DRAWINGS
[0039] Figure 1 It is a flow chart of the method proposed by the present invention.
[0040] Figure 2 It is the architecture of the reading comprehension model designed by the present invention based on pinyin embedding and multi-layer information fusion.
[0041] Figure 3 This is a model loss change trend diagram of the present invention. DETAILED DESCRIPTION
[0042] In order to better explain the present invention, detailed implementation steps are described below based on the drawings and examples.
[0043] Example 1
[0044] The present invention proposes a Chinese machine reading comprehension method based on pinyin embedding enhancement and multi-layer information fusion, and the adopted technical solution includes the following steps:
[0045] Step 1: Obtain a Chinese machine reading comprehension dataset, which is the CMRC2018 reading comprehension dataset. For convenience of description, select a context paragraph and a question-answer pair as an example: Context = "A complex sentence consists of a main clause and a subordinate clause. Subordinate clauses are usually divided into three types: attributive clauses, adverbial clauses, and nominal clauses.", Question = "What is a complex sentence?", Answer = "A complex sentence consists of a main clause and a subordinate clause".
[0046] Step 2: Use the Chinese word segmentation tool jieba to perform word segmentation on the data to obtain the word lists of Context and Question. The word segmentation in this embodiment is: C = ['main-subordinate','complex sentence', 'is', 'composed of', 'one','main clause', 'and', 'one','subordinate clause', 'constitute', 'the', 'complex sentence', '.', 'Subordinate clause', 'usually', 'divided into', 'three types', ':', 'attributive clause', ',', 'adverbial clause', 'and', 'noun', 'clause', '.'] and Q = ['what', 'is','main-subordinate', 'complex sentence', '?']. Then use the open-source word pinyin library pypinyin to process the words obtained by word segmentation to get the correct pinyin. In addition to letters, pinyin also has tones. In the experiments of the present invention, numbers are used to represent tones and are appended after the pinyin sequence, and the length of the pinyin sequence is fixed at 8, and the insufficient part is filled with the special character "-". In this embodiment, for example, "main-subordinate" is converted to [zhu3----cong2---];
[0047] Step 3: We manually define the mapping dictionary of pinyin characters, pinyin_map = {<t 1 , x 1 , <t 2 , x 2 , …, <t 32 , x 32 >}, where t i represents the characters that make up the pinyin, and x i represents the set id number, and 32 is the size of the mapping dictionary of all pinyin characters. Because there are a total of 26 characters that make up the pinyin, plus 4 tones, and the padding character, a total of 32 are designed; Table 1 is the mapping comparison table of pinyin characters:
[0048]
[0049] Convert the input pinyin characters to a pinyin sequence pinyin_ids through the pinyin character mapping dictionary; in this embodiment, a total of 48 pinyins including special identifiers can be obtained, and
[0050]
[0051] Step 4: Get the pinyin embedding matrix PV by random initialization k×pd , where k represents the number of entries in the pinyin dictionary, pd represents the dimension of pinyin embedding, and the pinyin embedding matrix PV is the parameter of model training. Convert the pinyin sequence into a vector representation PE = max_pooling (CNN (PV (pinyin_ids), C, H, W = pd)) lx×C , where pinyin_ids represents the pinyin sequence, which is converted into a pinyin vector through the pinyin embedding matrix PV, C represents the convolution channel, lx represents the number of Chinese characters in the input sequence, and both the functions CNN() and max_pooling() return vectors; in this embodiment, the number of convolution channels C = 300, lx = 48, and the pinyin embedding matrix PE of the context context can be obtained 48×300 :
[0052]
[0053] Step 5: Concatenate the context and question using special identifiers [CLS] and [SEP] to obtain the input sequence of the BERT model. Use BERT's built-in word segmentation tool to segment the input sequence, and then use the BERT pre-trained vocabulary to obtain the word vector representation E of each word. token , through the fully connected layer, PE and word embedding E token Fusion is performed to obtain the vector FE=concat(E token ,PE)W fc +b fc , where W fc is the weight matrix, b fc is the bias; in this embodiment, W fc The dimension of is 1068×768, and the embedding vector of fused pinyin embedding is
[0054]
[0055] Then use the segment embedding E seg and position embedding E pos Added to the enhanced word embedding FE, we get the complete embedding vector Where lx represents the length of the input sequence after the context and question are concatenated, d embed is the embedding dimension of the model, It means to add the vector element by element;
[0056] Step 6: The BERT model encodes the complete embedding vector, performs multi-layer information attention fusion calculation on the six low-level encoding vectors and the embedding vector, and obtains the context vector representation that integrates the semantic information of the word level and the low-level question. in in Indicates concatenation of vectors, l indicates low-level encoding operations, and similarly, WH can be obtained. Q ; Then through BiLSTM The vector is encoded with the sixth-layer context vector to obtain a low-level fusion information representation In this embodiment, the obtained low-level information fusion context representation v Cl
[0057]
[0058] Similarly, according to the obtained vector representation v Cl Input to the seventh layer encoder, get The high-level encoding representations of the seventh to eleventh layers can be obtained, which are consistent with the word-level embedding w C and low-level fusion information Perform multi-layer information attention fusion to obtain the contextual representation of high-level semantic information of question Q in Similarly, we can get WH Q ; Then through BiLSTM The vector is encoded with the context vector of the eleventh layer to obtain a high-level fusion information representation In this embodiment, the vector representation of the high-level information fusion of context C is:
[0059]
[0060] Finally, the last encoder layer of BERT is used to fusion represent the high-level information vector v Ch Encode with the encoding vector of the eleventh layer to obtain the final hidden vector v output ; In this embodiment, the final encoding output of the input sequence is
[0061]
[0062] Step 7: Based on the encoded vector representation of the context and question input sequence, calculate the probability P of each word in the context sequence as the starting position of the predicted answer start (i) and the probability P of being the end position of the answerend (i), finally take the value that makes P joint (i,j)=P start (i)×P end (j) The combination with the largest result.
[0063] In this embodiment, the highest joint probability is finally obtained to be 0.246156, where the calculated P start (0) = 0.562, P end (19) = 0.438. The joint probability of these two combinations is the highest. Therefore, the starting position is 0 and the ending position is 19. The continuous sequence between these two positions is the answer to the question. In addition, the corresponding loss L start is 0.62, L end The weighted sum of the two losses is 0.78, and the complete loss of the predicted answer is obtained.
[0064] At this point, a sample training of the Chinese machine reading comprehension model based on pinyin embedding enhancement and multi-layer information fusion has been completed. Repeat the above steps, calculate the complete loss of each data sample, and finally optimize the fine-tuning model.
[0065] Example 2
[0066] The present embodiment relates to a Chinese machine reading comprehension device based on pinyin embedding enhancement and multi-layer information fusion, including a memory and one or more processors, wherein the memory stores executable code, and when the one or more processors execute the executable code, they are used to implement the Chinese machine reading comprehension method based on pinyin embedding enhancement and multi-layer information fusion of Example 1.
[0067] Example 3
[0068] This embodiment relates to a computer-readable storage medium on which a program is stored. When the program is executed by a processor, the Chinese machine reading comprehension method based on pinyin embedding enhancement and multi-layer information fusion of embodiment 1 is implemented.
[0069] For those skilled in the art, the above content is only used to illustrate the present invention, and is not intended to limit the present invention. As long as it is within the essential spirit of the present invention, any modifications, equivalent substitutions, etc. should be included in the protection scope of the invention.
Claims
1. A Chinese machine reading comprehension method based on pinyin embedding enhancement and multi-layer information fusion, The following steps are involved: Step 1: Preprocess the dataset into a triple consisting of context C, question Q and answer A. The dataset D is represented as {D i |D i =(C, Q, A), 1≤i≤n}, where n represents the size of the data set D, D i Represents the i-th data in the data set, answer A is a continuous substring in context C; Step 2: Perform Chinese word segmentation on the data processed in step 1 to obtain the word sequence of the text, and pass the obtained Chinese word sequence through a convolutional network to obtain the pinyin embedding vector PE of the Chinese word; Step 3: Based on the data obtained in step 1, the paragraphs and questions are concatenated by marking [CLS] and [SEP], and then input into BERT. Using BERT’s wordpiece Chinese word segmentation, the word embedding vector E of each word is obtained. token ; Step 4: Pinyin embedding PE and word embedding E obtained in steps 2 and 3 respectively token , through a fully connected layer, we get a fused word embedding FE that integrates the pinyin embedding vector. Then we combine it with the segment embedding E of the input sequence. seg and position embedding E pos Add them together to get the embedding vector w of the complete input sequence; Step 5: Perform BERT encoding based on the sequence embedding vector obtained in step 4, so that the encoding vector representation of the output of different encoding layers can be obtained; Step 6: Based on the operation in step 5, perform multi-layer attention fusion calculation on the encoded representation obtained by the first six layers of encoders of the BERT model and the word embedding representation obtained in step 5 to obtain the context representation vector v that integrates the low-level question semantic information Cl , where C represents the context and l represents the low-level information fusion; Step 7: Input the context encoding representation obtained in step 6 and the sixth-layer question encoding representation into the seventh-layer encoder, and obtain the encoding representations of the seventh to eleventh layers of encoders in turn. Then, perform multi-layer attention fusion calculation with the word embedding vector in step 5 and the low-level encoding representation obtained in step 6 to obtain the context representation vector v that integrates high-level question semantic information. Ch , where C represents context and h represents high-level information fusion; Step 8: Based on the high-level context representation obtained in step 7 and the question representation vector of the eleventh layer encoder, they are input into the last encoder. Using the self-attention of the encoder, the final context semantic vector that integrates multiple layers of information is obtained. Where C represents the context, Q represents the question, m and n represent the length of the context and question respectively, and L 12 represents the encoded representation obtained by the twelfth layer encoder, and h represents the low-level encoding vector; Step 9: Based on the sentence vector representation obtained in step 8, the softmax function is used to predict the probability distribution of each position as the starting point and ending point of the answer. Finally, all possible combinations of starting points and ending points are traversed, and the combination with the largest probability product is selected to determine the span of the answer.
2. A Chinese machine reading comprehension method based on pinyin embedding enhancement and multi-layer information fusion, It is characterized in that Step 2 uses the convolutional network model to obtain pinyin embedding, specifically: let the number of pinyins of Chinese characters be len, randomly initialize a pd-dimensional vector for the pinyin of each Chinese character, and form a pinyin matrix Convert Chinese text to pinyin sequence x=[x 1 , x 2 , ..., x lx ], each pinyin sequence is mapped to the corresponding pinyin vector through PV, and the matrix px = [x 1 , x 2 , ..., x lx ]; where x i =PV(x i ), 1≤i≤lx, represents pinyin x i The corresponding pinyin vector, lx represents the number of Chinese characters; by performing convolution operation on the pinyin sequence and then performing maximum pooling, the pinyin embedding representation PE is obtained.
3. A Chinese machine reading comprehension method based on pinyin embedding enhancement and multi-layer information fusion, It is characterized in that The embedding vector w in step 4 can be expressed as w = FE + E seg +E pos , where FE = concat(E token ,PE)W fc +b fc , where W fc represents the weight of the fully connected layer, W fc represents the bias of the fully connected layer, 4. A Chinese machine reading comprehension method based on pinyin embedding enhancement and multi-layer information fusion, It is characterized in that The encoding operation of embedding the input sequence using BERT in step 5 specifically refers to: using the encoder in BERT to model the semantic information of the words in the input sequence to obtain the encoding output of each layer of the input sequence; Given an input sequence embedding w, input into the BERT model: Where C represents the context paragraph, Q represents the question, m and n represent the length of the context and question respectively, and L i represents the encoded representation obtained by the i-th layer encoder, 1≤i≤6, l represents the low-level encoding vector; similarly, the encoded representation of the first six layers of encoders can be obtained.
5. A Chinese machine reading comprehension method based on pinyin embedding enhancement and multi-layer information fusion, It is characterized in that The attention fusion calculation described in step 6 specifically refers to: defining two sets of vectors about context and question and Concatenate the word-level and first six-layer encoding vectors to fuse all low-level information in question Q into context C: Middle W C and W Q represents the learnable weight matrix, v represents the weight vector, and then a softmax is used to get the attention weight: Finally get Next, use BiLSTM to get and The representation vector is encoded to obtain the context representation vector of the low-level fusion information:
6. A Chinese machine reading comprehension method based on pinyin embedding enhancement and multi-layer information fusion, It is characterized in that The high-level information fusion described in step 7 specifically refers to: obtaining Among them, h represents the high-level coding operation; the coding of the seventh to eleventh layers is expressed as and The high-level semantic information in question Q is integrated into the context to obtain Constructing contextual representations of high-level fused information:
7. A Chinese machine reading comprehension method based on pinyin embedding enhancement and multi-layer information fusion, It is characterized in that The context semantic vector of the fusion of multi-layer information described in step 8 specifically refers to: using the last layer encoder of BERT to perform self-attention calculation on the high-level context representation to obtain the context encoding vector of all hierarchical information in the fusion problem:
8. A Chinese machine reading comprehension method based on pinyin embedding enhancement and multi-layer information fusion, It is characterized in that For the softmax function used in step 9 to predict the probability of each position as the starting and ending point, specifically: let the input context length be M, the model will output P start and P end : Among them, s i and e i They represent the scores of the i-th position predicted by the model as the starting point and the ending point of the answer respectively; finally, the starting point and the ending point with the highest joint probability are selected: P joint (i, j) = P start (i)×P end (j) By traversing all possible combinations of starting and ending points, select joint The largest combination (i, j) is the final answer span; The loss function of the model uses the cross entropy loss function, which is used to measure the difference between the answer predicted by the model and the actual answer. The specific calculation formula is as follows: Among them, y start and end The true label vectors represent the start and end positions of the actual answer respectively. Finally, the weighted sum of the losses at the start and end positions is performed to obtain the complete loss function: Loss=λ·Loss start +(1-λ)·Loss end (4) Among them, λ is a weight parameter used to balance the loss at the start and end positions.
9. A Chinese machine reading comprehension device based on pinyin embedding enhancement and multi-layer information fusion, It is characterized in that It comprises a memory and one or more processors, wherein the memory stores executable code, and when the one or more processors execute the executable code, it is used to implement the Chinese machine reading comprehension method based on pinyin embedding enhancement and multi-layer information fusion as described in any one of claims 1-8.
10. A computer-readable storage medium, It is characterized in that A program is stored thereon, and when the program is executed by a processor, the Chinese machine reading comprehension method based on pinyin embedding enhancement and multi-layer information fusion as described in any one of claims 1-8 is implemented.
Citation Information
Patent Citations
Method for generating Chinese word vector with multi-submodule information
CN111160020A
Chemical emergency news classification method based on ChineseBERT model and attention mechanism
CN114510569A
Chinese short text similarity calculation method based on CNN and pinyin features
CN115525735A
Text processing method and device based on attention enhancement, equipment and medium
CN117291187A
