A keyword generation method based on stem information fusion
By integrating stemming information, the problem of inaccurate keyword generation caused by word form transformation was solved, achieving higher prediction accuracy and semantic consistency, and improving the performance of the keyword generation algorithm.
Patent Information
- Application Number
- CN202210422977.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Priority Date
- 2022-04-18
- Filing Date
- 2022-04-21
- Publication Date
- 2024-11-12
- Estimated Expiration
- 2042-04-21
AI Technical Summary
Existing keyword generation algorithms lack effective solutions to the problem of inaccurate predictions caused by word transformation, especially the inconsistency between the initial text and the generated target semantics. Existing methods mainly improve automatic measurement through post-processing, but have failed to effectively solve this problem.
By incorporating stemming information into the embedding layer from a morphological perspective, a reference text is constructed. A double soft-switch copying mechanism is used to correct the copying mechanism, and multi-task joint training is performed at the decoder level to generate keyword probability distributions. Finally, keywords are generated through the BeamSearch method.
It improved the accuracy and consistency of keyword generation, enhanced the correlation with words in the same stem, alleviated the network performance deficiencies caused by training bias, and improved text understanding capabilities.
Smart Images

Figure CN114781367B_ABST
Abstract
Description
Technical Field
[0001] The invention relates to a keyword generation method based on stem information fusion, belonging to the technical field of Internet and artificial intelligence. Background Art
[0002] With the continuous advancement of the Internet wave, the era of big data has also arrived. Among them, various data are dazzling, and text data is showing an exponential growth trend. It is crucial to study how people can quickly extract useful and demand-oriented information from a large amount of text information. Keywords, as a highly condensed and refined expression of the text theme, can quickly help people understand the key information of the initial text and help the system find relevant information that is closer to people's needs. The technology of automatically extracting keywords by building an end-to-end neural network is the basic implementation method of keyword automatic generation technology. Different from keyword automatic extraction technology, the former can well solve the problem of predicting absent keywords that do not appear in the text. Naturally, how to improve the ability to predict absent keywords is also a key issue in measuring keyword automatic generation technology.
[0003] The general idea of word form transformation is that the same word stem can produce words with multiple semantics after being transformed into different forms. Currently, most cutting-edge keyword generation algorithms use a "post-processing" method to deal with the problem of absent prediction caused by word form transformation, in order to improve the measurement. However, this method only bypasses the problem from the perspective of improving automatic measurement, and has no practical significance for the problem of inconsistency between the initial text and the generated target semantics caused by actual word form transformation, and has defects. Summary of the invention
[0004] In view of the problems existing in the prior art, the present invention proposes a keyword generation method based on stem information fusion. From the perspective of word morphology, the extracted stem information is integrated into the Embedding layer to enhance the strong correlation between words with the same stem. A reference text is constructed based on a combination of variants with the same stem in a vocabulary search text. The word semantic information in the reference text is extracted through the encoding layer module. The initial text and the reference text are weighted and expanded to generate a probability distribution. The dual soft switch replication mechanism is used to correct the replication mechanism, and a stem generation task is constructed at the decoder level. The final keyword generation probability distribution is obtained through multi-task joint training of words and stems. Finally, keywords are generated based on the BeamSearch method.
[0005] In order to achieve the above object, the present invention provides the following technical solutions:
[0006] A keyword generation method based on stem information fusion includes the following steps:
[0007] Step 1: Extract stem information and integrate it into the Embedding layer
[0008] First input text X, and output stem sequence X through stem extraction algorithm s , the stem sequence is counted to build a stem table, and then the input text and stem sequence are represented and accumulated as high-dimensional vectors in the Embedding layer, and the data is fused using the set LayerNorm method;
[0009] Step 2: Search for the same stem variant combination in the text to construct a reference text, and extract the word semantic information in the reference text through the encoding layer module
[0010] By counting the different words with the same stem of each word in the input text X, a set is obtained, and the set is arranged in disorder to obtain the reference text A. The reference text representation vector e is obtained in the same way as in step 1 A , the reference text representation vector is used as the model input, the long-term dependency between different words is learned through the attention mechanism, and the multi-layer attention mechanism is jointly accumulated to obtain the semantic information of different levels and directions. The residual network and LayerNorm are used to obtain the semantic vector h output by the encoder module A ;
[0011] Step 3, based on the weighted expansion of the initial text and the reference text, a probability distribution is generated, and a double soft-switch replication mechanism is used to correct the replication mechanism.
[0012] The unidirectional gated recurrent unit GRU is used as the decoding unit and the decoder state vector s is iteratively calculated. t , and then calculate the current generation probability s j Generate context semantic vector based on the relevance of reference text A Then based on the initial text context semantic vector c and the decoder state vector s t Get the initial probability of keyword generation p t , combine the context semantic vector of reference text A and the context semantic vector of initial text to generate weights, superimpose attention distribution, modify the copy mechanism, and generate the final generation probability
[0013] Step 4: Construct a stem generation task at the decoder level and optimize the final keyword generation probability distribution through multi-task joint training of words and stems
[0014] The stem generation task uses the encoder and decoder output states in steps 2 and 3, which are the same as the keyword generation task, to concatenate the main encoder semantic vector and the decoder output state to construct the stem generation probability distribution. The cross entropy loss function is used to train the model by combining the keyword generation task and the stem production task for synchronous training to obtain the final keyword generation probability distribution.
[0015] Step 5: Based on the final generation probability The final keyword sequence is generated based on the BeamSearch search method.
[0016] Preferably, the step 1 specifically includes the following process:
[0017] Define a given text X = [x1, x2, ..., x M ], generate a series of Present keywords and Absent keyword Where M and N p 、N a They are text length, number of Present keywords, and number of Absent keywords. Among them l y As the length of a single keyword, the length of each keyword is not fixed. The formula is p = f θ (γ|X), where θ is the parameter of model f and p refers to the probability distribution of predicted keyword generation.
[0018] Using input text X = [x1, x2, ..., x M ] as the input of the stemming algorithm, which then outputs a sequence of stems Then count the stem sequence X s Constructing a stem table in Φ s , where Φ s The length of is a hyperparameter, which is adjusted according to the difficulty of the task, the size of the dataset and the model effect.
[0019] High-dimensional vector representation of the input text X and high-dimensional vector representation of stem sequences The formula is as follows:
[0020] e l =LayerNorm(e o +e s )
[0021] where e=[e1,e2,…,e M ] is the fused text representation matrix, α and β represent the parameters that the model needs to learn.
[0022] Preferably, the step 2 specifically includes the following process:
[0023] Construct the reference text A by counting each word x in the input X i The same stem but different words are obtained as a set, and the set is arranged in random order to obtain the reference text A = [a1, a2, ..., a P ], where P is the length of the reference text. The reference text A can be obtained by using the same method as in step 1 to obtain the text representation matrix
[0024] Represent the reference text as a vector As the model input, the long-term dependencies between different words are learned through the attention mechanism, and the multi-layer attention mechanism jointly accumulates to obtain semantic information at different levels and in different directions.
[0025] In order to control the complexity of the model, after learning the associated semantics through the multi-head attention mechanism, the residual network part and LayerNorm part of the Transformer model are retained. The formula is as follows:
[0026] h A =LayerNorm(e A +MultiHead(e A ))
[0027] h A =LayerNorm(h A )
[0028] where Represents the semantic vector finally output by the sub-encoder module.
[0029] Preferably, step 3 includes the following 4 sub-steps:
[0030] Sub-step 3-1, define the process of generating keywords as follows:
[0031]
[0032] where y <j = <y1,y2,…,y j-1 >, the above formula means that when the input semantic vectors h, h A Under the conditions of and model parameters θ, the model gives the probability distribution of generating the currently specified keyword.
[0033] Sub-step 3-2, use the unidirectional gated recurrent unit GRU as the decoding unit and input an initial text semantic vector h M Iterate as a preliminary internal memory vector, the formula is as follows:
[0034] st =f GRU (h M ;s t-1 )
[0035] where t=1,2,…,T x The length of t is determined by the specific generation formula. x is the decoder time stop step for the actual iteration.
[0036] Sub-step 3-3, using the Bahdanau correlation calculation method to calculate the correlation coefficient e tj , the formula is as follows:
[0037]
[0038] where t=1,,2,…,T x , where W α ,b α It is the learnable parameter matrix and parameter vector in the process of calculating the score. The result of the Attention score is a correlation measure between the decoder hidden vector and the encoder sequence vector. In order to realize the measurement of the probability distribution, the Attention score and the sequence vector are multiplied by the following formula:
[0039]
[0040]
[0041] Sub-step 3-4, based on the initial text context semantic vector c and the decoder state vector s t Generate, the formula is as follows:
[0042] p t =softmax(W t [s t ;c t ]+b t )
[0043] Where W t is the learnable parameter matrix, b t is a learnable parameter vector. Then the soft switch λ is calculated t , And calculate the probability of keyword generation Among them, λ t ,
[0044] Preferably, step 4 specifically includes the following process:
[0045] The semantic vector of the main encoder and the output state of the decoder are concatenated to construct the probability distribution of stem generation. The specific formula is as follows:
[0046]
[0047] Where W1, W2, b1, and b2 represent the learnable parameters of the model during the training process.
[0048] Both the keyword generation sequence task and the stem sequence generation task use cross entropy as the loss function, and use the MLE training method to train the model by minimizing the loss value between the predicted value and the true value in all training samples.
[0049] In order to solve the problem that the cross entropy loss function only focuses on the probability of correct targets and ignores the probability of incorrect targets, the model training is implemented by synchronously training the keyword generation task and the stem production task. The joint loss function is defined as follows:
[0050]
[0051] in The value of depends on the prediction results of the stem sequence and keyword sequence. Only when the stem prediction is correct and the keyword prediction is wrong, otherwise
[0052] Preferably, the step 5 specifically includes the following process:
[0053] Under the guidance of the neural network, multiple groups of keywords are selected from the vocabulary; the BeamSearch algorithm is used to retain multiple rounds of results for each prediction sequence and gradually select the best one; the details are as follows:
[0054] First use ' <bos>'As the initial state of the search, get the probability distribution of a single word of the model; then select the largest first k probability distributions and store the specified word in an array; then traverse all the words in the array, use the word as the model input, execute the above steps, and execute T x After that, all the keywords are output as a sequence.
[0055] Compared with the prior art, the present invention has the following advantages and beneficial effects:
[0056] From the perspective of word morphology, the present invention integrates the stem information of the text into the network through embedding fusion to enhance the strong correlation between words with the same stem, adopts a dual soft switch replication mechanism to correct the generation deviation caused by the replication mechanism, and combines the stem sequence generation task to make a multi-task learning model to alleviate the problem that the network performance cannot be fully utilized due to training deviation. Compared with other methods, the method of the present invention has strong generation ability and high prediction accuracy, and has obvious advantages in text understanding. BRIEF DESCRIPTION OF THE DRAWINGS
[0057] Figure 1 A flow chart of a keyword generation method based on stem information fusion provided by the present invention;
[0058] Figure 2 A model architecture diagram for implementing the keyword generation method based on stem information fusion provided by the present invention;
[0059] Figure 3 Schematic diagram of the steps of stem information fusion;
[0060] Figure 4 This is the encoder structure diagram;
[0061] Figure 5 This is the structure diagram of the decoder. DETAILED DESCRIPTION
[0062] The technical solution provided by the present invention will be described in detail below in conjunction with specific embodiments. It should be understood that the following specific implementation methods are only used to illustrate the present invention and are not used to limit the scope of the present invention.
[0063] This method provides a keyword generation method based on stem information fusion, and its process is as follows: Figure 1 As shown, the model architecture is as Figure 2 As shown, the specific implementation steps are as follows:
[0064] Step 1: Extract stem information and integrate it into the Embedding layer.
[0065] Define a given text X = [x1, x2, ..., x M ], generate a series of Present keywords and Absent keyword Where M and N p 、N a The length of the text and the number of Present keywords and Absent keywords respectively. Where l is the length of a single keyword, and the length of each keyword is not fixed. The formula is p = f θ (γ|X), where θ is the parameter of model f and p refers to the probability distribution of predicted keyword generation.
[0066] Using input text X = [x1, x2, ..., x M ] as the input of the stemming algorithm, which then outputs a sequence of stems Then count the stem sequence X s Constructing a stem table in Φ s , where Φ s The length of is a hyperparameter, which is adjusted according to the difficulty of the task, the size of the dataset and the model effect.
[0067] like Figure 3 As shown, the high-dimensional vector representation of the input text X and high-dimensional vector representation of stem sequences The formula is as follows:
[0068] e l =(e o +e s ) (1)
[0069]
[0070] where e=[e1,e2,…,e M ] is the fused text representation matrix, α and β represent the parameters that the model needs to learn.
[0071] Step 2: Search for the same stem variant combination in the text to construct a reference text, and extract the semantic information of the words in the reference text through the encoding layer module, such as Figure 4 shown.
[0072] Construct a reference text A by counting the different words with the same stem of each word in the input text X to obtain a set, and then arrange the set in random order to obtain a reference text A = [a1, a2, ..., a P ], where P is the length of the reference text. The reference text representation vector is obtained in the same way as in step 1
[0073] Represent the reference text as a vector As model input, generate Q i , K i 、V i vector, and obtain the vector head by learning the long-term dependencies between different words through the attention mechanism i , the multi-layer attention mechanism jointly accumulates the semantic information of different levels and directions, and finally obtains the comprehensive semantic vector MultiHead(e A ). The formula is as follows:
[0074]
[0075]
[0076] MultiHead(e A )=Concat(head1, head2,..., head n ) (5)
[0077] Where n is the number of heads in the multi-head attention mechanism, are the learnable parameter matrices of Query, Key and Value, respectively, k As a scaling factor for scaling self-attention.
[0078] In order to control the complexity of the model, after learning the associated semantics through the multi-head attention mechanism, the residual network part and LayerNorm part of the Transformer model are retained. The formula is as follows:
[0079] h A =e A +MultiHead(e A ) (6)
[0080] h A =LayerNorm(h A ) (7)
[0081] in Represents the semantic vector finally output by the sub-encoder module.
[0082] Step 3, based on the weighted expansion of the initial text and the reference text, a probability distribution is generated, and a double soft switch replication mechanism is used to correct the replication mechanism, such as Figure 5 shown.
[0083] The unidirectional gated recurrent unit GRU is used as the decoding unit and the decoder state vector s is iteratively calculated. t , and then calculate the current generation probability s j Generate context semantic vector based on the relevance of reference text A Then based on the initial text context semantic vector c and the decoder state vector s t Get keyword generation probability The implementation process of this step is divided into 4 sub-steps:
[0084] Sub-step 3-1, define the process of generating keywords as follows:
[0085]
[0086] p j =Pr(y i |y <j ,θ,h,h A ) (9)
[0087] where y <j = <y1,y2,...,y j-1 >, the above formula means that when the input semantic vectors h, h A Under the conditions of and model parameters θ, the model gives the probability distribution of generating the currently specified keyword.
[0088] Sub-step 3-2, use the unidirectional gated recurrent unit GRU as the decoding unit and input an initial text semantic vector h M Iterate as a preliminary internal memory vector to generate the decoder state vector s t , the formula is as follows:
[0089] s t =f GRU (h M ;s t-1 ) (10)
[0090] where t=1,2,...,T x The length of t is determined by the specific generation formula. x is the actual iterative decoder time stop step. In addition, f GRU Represents the function mapping relationship implemented by the GRU decoding unit.
[0091] Sub-step 3-3, using the Bahdanau correlation calculation method to calculate the correlation coefficient e tj , the formula is as follows:
[0092]
[0093] where t = 1,, 2, ..., T x , where W α , b α It is the learnable parameter matrix and parameter vector in the process of calculating the score. Tanh() represents the nonlinear activation function. Represents the value vector corresponding to the attention formula, which is the semantic vector h based on the initial text described in the above statement. The result of the attention score is a measure of the correlation between the decoder hidden vector and the encoder sequence vector. In order to realize the measurement of probability distribution, the attention score and sequence vector Perform dot multiplication calculation, the formula is as follows:
[0094]
[0095]
[0096] Sub-steps 3-4, based on the initial text context semantic vector c t and the decoder state vector s t Generate preliminary keyword generation probability p t , the formula is as follows:
[0097] p t =softmax(W t [s t ;c t ]+b t ) (14)
[0098] Where W t is the learnable parameter matrix, b t is a learnable parameter vector. Then the soft switch λ is calculated t , And calculate the final probability of keyword generation
[0099] λ t =sigmoid(W λ [s t ;c t ;e t ]+b λ ) (15)
[0100]
[0101]
[0102] Among them, λ t , Sigmoid() is a non-linear activation function.
[0103] Step 4: Construct a stem generation task at the decoder level and optimize the final keyword generation probability distribution through multi-task joint training of words and stems
[0104] The semantic vector of the main encoder and the output state of the decoder are concatenated to construct the probability distribution of stem generation. The specific formula is as follows:
[0105]
[0106] Where W1, W2, b1, and b2 represent the learnable parameters of the model during the training process.
[0107] The keyword sequence generation task and the stem sequence generation task both use cross entropy as the loss function. and The model is trained by minimizing the loss between the predicted value and the true value in all training samples using the MLE training method. The specific formula is defined as follows:
[0108]
[0109]
[0110] In order to solve the problem that the cross entropy loss function only focuses on the probability of correct targets and ignores the probability of incorrect targets, the model training is implemented by synchronously training the keyword generation task and the stem production task. The joint loss function is defined as follows:
[0111]
[0112] in The value of depends on the prediction results of the stem sequence and keyword sequence. Only when the stem prediction is correct and the keyword prediction is wrong, otherwise
[0113] Step 5, based on the final generation probability The final keyword sequence is generated based on the BeamSearch search method.
[0114] After the model training converges, the next step is to select multiple groups of keywords from the vocabulary under the guidance of the neural network. One of the most direct ways is to select the specified words from the vocabulary according to the probability distribution of keyword generation. This search method has an important flaw: because it only considers the optimal solution of each step, and does not start from a global perspective, the final result is too far away from the actual optimal solution. The effect is not ideal. Another way is to search the global optimal solution by brute force, calculate each possible result, and give the result with the highest probability, but an unavoidable problem is that the search complexity reaches O(n*n*T), and the actual operation space is very low. The BeamSearch algorithm between the two takes multiple rounds of results for each prediction sequence and gradually takes the best.
[0115] The idea of the BeamSearch algorithm is to first use' <bos>'As the initial state of the search, get the probability distribution of a single word of the model; then select the largest first k probability distributions and store the specified word in an array; then traverse all the words in the array, use the word as the model input, execute the above steps, and execute T x After that, all the keywords are output as a sequence.
[0116] The technical means disclosed in the scheme of the present invention are not limited to the technical means disclosed in the above-mentioned implementation mode, but also include technical schemes composed of any combination of the above-mentioned technical features. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention, and these improvements and modifications are also regarded as the protection scope of the present invention.< / bos> < / bos>
Claims
1. A keyword generation method based on stem information fusion, characterized in that: The steps include: Step 1: Extract stem information and integrate it into the Embedding layer First input text X, and output stem sequence X through stem extraction algorithm s , the stem sequence is counted to build a stem table, and then the input text and stem sequence are represented and accumulated as high-dimensional vectors in the Embedding layer, and the data is fused using the set LayerNorm method; Step 2: Search for the same stem variant combination in the text to construct a reference text, and extract the word semantic information in the reference text through the encoding layer module By counting the different words with the same stem of each word in the input text X, a set is obtained, and the set is arranged in disorder to obtain the reference text A. The reference text representation vector e is obtained in the same way as in step 1 A , the reference text representation vector is used as the model input, the long-term dependency between different words is learned through the attention mechanism, and the multi-layer attention mechanism is jointly accumulated to obtain the semantic information of different levels and directions. The residual network and LayerNorm are used to obtain the semantic vector h output by the encoder module A ; Step 3: Generate probability distribution based on weighted expansion of initial text and reference text, and use double soft switch replication mechanism to correct replication mechanism The unidirectional gated recurrent unit GRU is used as the decoding unit and the decoder state vector s is iteratively calculated. t , and then calculate the current generation probability s j Generate context semantic vector based on the relevance of reference text A Then based on the initial text context semantic vector and the decoder state vector s t Get the initial probability of keyword generation p t , combine the context semantic vector of reference text A and the context semantic vector of initial text to generate weights, superimpose attention distribution, modify the copy mechanism, and generate the final generation probability Step 4: Construct a stem generation task at the decoder level and optimize the final keyword generation probability distribution through multi-task joint training of words and stems The stem generation task uses the encoder and decoder output states in steps 2 and 3, which are the same as the keyword generation task, to concatenate the main encoder semantic vector and the decoder output state to construct the stem generation probability distribution. The cross entropy loss function is used to train the model by combining the keyword generation task and the stem production task for synchronous training to obtain the final keyword generation probability distribution. Step 5: Based on the final generation probability The final keyword sequence is generated based on the BeamSearch search method.
2. The keyword generation method based on stem information fusion according to claim 1, characterized in that: The step 1 specifically includes the following process: Define a given text X = [x1, x2, ..., x M ], generate a series of Present keywords and Absent keyword Where M and N p 、N a They are text length, number of Present keywords and number of Absent keywords respectively; Among them l y As the length of a single keyword, the length of each keyword is uncertain; the formula is p = f θ (γ|X), where θ is the parameter of model f, and p refers to the probability distribution of predicted keyword generation; Using input text X = [x1, x2, ..., x M ] as the input of the stemming algorithm, which then outputs a sequence of stems Then count the stem sequence X s Constructing a stem table in Φ s , where Φ s The length of is a hyperparameter; High-dimensional vector representation of the input text X and high-dimensional vector representation of stem sequences The formula is as follows: And l =(and o +e s ) where e=[e1,e2,…,e M ] is the fused text representation matrix, α and β represent the parameters that the model needs to learn.
3. The keyword generation method based on stem information fusion according to claim 1, characterized in that: The step 2 specifically includes the following process: Construct the reference text A by counting each word x in the input X i The same stem but different words are obtained as a set, and the set is arranged in random order to obtain the reference text A = [a1, a2, ..., a P ], where P is the length of the reference text; the reference text A can be obtained by using the same method as in step 1 to obtain the text representation matrix Represent the reference text as a vector As the model input, the long-term dependencies between different words are learned through the attention mechanism. The multi-layer attention mechanism jointly accumulates and obtains semantic information at different levels and in different directions. After learning the associated semantics through the multi-head attention mechanism, choose to retain the residual network part and LayerNorm part of the Transformer model. The formula is as follows: h A =LayerNorm(e A +MultiHead(e A )) h A =LayerNorm(h A ) in Represents the semantic vector finally output by the sub-encoder module.
4. The keyword generation method based on stem information fusion according to claim 1, characterized in that: The step 3 includes the following 4 sub-steps: Sub-step 3-1, define the process of generating keywords as follows: p j =Pr(y i |y <j ,θ,h,h A , where y <j = <y1,y2,…,y j-1 >, the above formula means that when the input semantic vectors h, h A Under the conditions of model parameters θ, the model gives the probability distribution of generating the currently specified keyword; Sub-step 3-2, use the unidirectional gated recurrent unit GRU as the decoding unit and input an initial text semantic vector h M As the initial internal memory vector for iteration, the formula is as follows: s t =f GRU (h M ;s t-1 ) where t=1,2,…,T x , the length of t is determined by the specific generation formula, T x is the decoder time stop step of the actual iteration; Sub-step 3-3, using the Bahdanau correlation calculation method to calculate the correlation coefficient e tj , the formula is as follows: where t=1,2,…,T x , where W α ,b α are the learnable parameter matrices and parameter vectors used in calculating the scores; The result of the attention score is a measure of the correlation between the decoder hidden vector and the encoder sequence vector. In order to realize the representation of the probability distribution of the metric, the attention score and the sequence vector are multiplied by the following formula: Sub-steps 3-4, based on the initial text context semantic vector c t and the decoder state vector s t Generate preliminary keyword generation probability p t , the formula is as follows: p t =softmax(W t [s t ;c t ]+b t ) Where W t is the learnable parameter matrix, b t is a learnable parameter vector; then calculate the soft switch λ t , And calculate the probability of keyword generation Among them, λ t , 5. The keyword generation method based on stem information fusion according to claim 1, characterized in that: The step 4 specifically includes the following process: The semantic vector of the main encoder and the output state of the decoder are concatenated to construct the probability distribution of stem generation. The specific formula is as follows: Where W1, W2, b1, and b2 represent the learnable parameters of the model during training; The keyword sequence generation task and the stem sequence generation task both use cross entropy as the loss function, and use the MLE training method to train the model by minimizing the loss between the predicted value and the true value in all training samples. The model training is realized by combining the keyword generation task and the stem production task for synchronous training. The joint loss function is defined as follows: in, The value of depends on the prediction results of the stem sequence and keyword sequence; only when the stem prediction is correct and the keyword prediction is wrong, otherwise 6. The keyword generation method based on stem information fusion according to claim 1, characterized in that: The step 5 specifically includes the following process: Under the guidance of the neural network, multiple groups of keywords are selected from the vocabulary; the BeamSearch algorithm is used to retain multiple rounds of results for each prediction sequence and gradually select the best one; the details are as follows: First use ' <bos>'As the initial state of the search, get the probability distribution of a single word of the model; then select the largest first k probability distributions and store the specified word in an array; then traverse all the words in the array, use the word as the model input, execute the above steps, and execute T x After that, all the keywords are output as a sequence.< / bos>
