Question retrieval method and device, equipment, medium and product
By introducing an attention layer to the LSTM model and extracting local features in combination with the FCN model, the problem of traditional search methods insufficient understanding of semantic fuzzy and context association is solved, and the accuracy and efficiency of problem retrieval is improved.
Patent Information
- Application Number
- CN202411760232.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-12-03
- Publication Date
- 2025-03-28
AI Technical Summary
The search method based on traditional vector space model has weak understanding of semantic fuzzy and contextual associations, while deep learning models rely on good training data when obtaining accurate feedback, have strong dependence, and insufficient recognition ability of new terms, resulting in reduced relevance and accuracy of search results.
Introducing attention layer to the LSTM model allows it to assign different weights to different word vectors, highlighting the role of important words in semantic understanding. Combined with the FCN model, the feature sequence output by the LSTM model is further processed, and multiple local features are extracted, so that the model can fully obtain important feature information in the problem text.
It improves the sensitivity of the model to identify key information, improves the accuracy of similarity calculation, enhances the recognition of problems by the model, thereby improving the accuracy and efficiency of problem retrieval.
Smart Images
Figure CN119719280A_ABST
Abstract
Description
Technical Field
[0001] The embodiments of the present application relate to the field of natural language processing technology, and in particular, to a method, apparatus, device, medium and product for question retrieval. Background Art
[0002] At present, the rapid development of question retrieval technology provides users with a more efficient way to obtain the required information. Existing question retrieval technology can be mainly divided into two categories: question retrieval technology based on vector space model (VSM) and question retrieval technology based on machine learning.
[0003] Question retrieval technology based on the vector space model first preprocesses the user's question text, then simplifies the processed content into vector operations in the vector space, measures the similarity between texts by calculating the similarity between vectors, assigns corresponding weights, and finally outputs texts with higher weights. In this way, the cosine distance between the user's query text and the text vector stored in the database is usually calculated as the similarity measurement method.
[0004] The question retrieval method based on machine learning mainly uses an artificial neural network model to continuously perform semantic and lexical analysis on the pre-processed user input information sequence, thereby realizing the learning of question data and rules, and automatically retrieving questions with high similarity to the user's question text based on these characteristics.
[0005] The above scheme has the following main disadvantages: in the question retrieval technology based on the vector space model, the basis for judging similar texts is the word frequency information appearing in the text, and this method assumes that each word in the question is independent of each other, so its ability to distinguish semantically ambiguous words in natural language is poor, and it is difficult to handle some complex problems by relying on this model. Once the problem is more complex and there is a strong correlation between the words and the context, the retrieval matching degree of this technology will be greatly reduced. The main disadvantage of the question retrieval technology based on deep learning is that if you want to obtain more accurate retrieval feedback, you need a good training model. The retrieval system trained by the single CNN (Convolutional Neural Network) model or RNN (Recurrent Neural Network) model used in the current technical model cannot judge the semantics of the user input question according to the context of the question, resulting in the feedback to the user. There are grammatical errors or poor relevance to the question. In addition, when the model searches for some specific questions, the term describing the question generates new words, and if the model has never encountered these words during training, it cannot directly identify and retrieve, resulting in the new terms being ignored, reducing the accuracy of the retrieval. Summary of the invention
[0006] The embodiments of the present application provide a method, apparatus, device, medium and product for question retrieval to solve the technical problem that the retrieval method based on the traditional vector space model has a weak understanding of semantic ambiguity and contextual association, and the deep learning model relies heavily on good training data when obtaining accurate feedback, and has insufficient recognition ability for new terms, resulting in reduced relevance and accuracy of the retrieval results.
[0007] In order to solve the above technical problems, this application is implemented as follows:
[0008] In a first aspect, an embodiment of the present application provides a method for question retrieval, the method comprising:
[0009] Obtaining a question text to be searched input by a user and at least one reference question text in a reference data set;
[0010] The question text to be retrieved and the reference question text are respectively used as target question texts, and a target operation is performed, wherein the target operation includes: mapping each word in the target question text to a word vector, combining the word vectors according to the order of the corresponding words in the target question text, and obtaining a word vector sequence corresponding to the target question text; inputting the word vector sequence into an LSTM model to obtain a feature sequence of the word vectors, wherein the attention layer in the LSTM model is used to assign a weight value to each word vector in the word vector sequence, and the weight value is used to indicate the degree of influence of the word vector on the semantics of the word vector sequence; inputting the feature sequence into an FCN model to obtain a plurality of local features corresponding to the feature sequence; and fitting the plurality of local features to obtain a fitting result;
[0011] Inputting the fitting result corresponding to the to-be-searched question text and the fitting result corresponding to the reference question text into a similarity function to obtain a similarity comparison result;
[0012] If the similarity comparison result is greater than or equal to a first preset threshold, the reference question text corresponding to the similarity comparison result is output.
[0013] Optionally, after inputting the fitting result corresponding to the to-be-searched question text and the fitting result corresponding to the reference question text into a similarity function to obtain a similarity comparison result, the method further includes:
[0014] If the similarity comparison result is less than the first preset threshold, the question text to be retrieved is added to the reference data set.
[0015] Optionally, before mapping each word in the target question text to a word vector, the method further includes:
[0016] Deleting low-frequency words in the pre-training corpus; wherein the low-frequency words are words whose occurrence frequency in the pre-training corpus is lower than a second preset threshold;
[0017] Train the CBOW model based on the pre-trained corpus after removing low-frequency words;
[0018] Mapping each word in the target question text into a word vector includes:
[0019] Based on the CBOW model, each word in the target question text is mapped into a word vector.
[0020] Optionally, based on the pre-trained corpus after deleting low-frequency words, training the CBOW model includes:
[0021] Comparing the first vector space of the pre-training corpus with the second vector space of the test corpus; determining feature vectors that exist in the second vector space but are not included in the first vector space, and adding the feature vectors to the first vector space to obtain a new pre-training corpus;
[0022] The CBOW model is trained based on the new pre-training corpus.
[0023] Optionally, the attention layer in the LSTM model is used to determine the weight value of each word vector in the word vector sequence based on a preset context vector value, and based on the weight value, calculate the weighted sum of each word vector in the word vector sequence, wherein the weighted sum is used as one of the inputs of the LSTM model to consider the degree of influence of each word vector on the semantics of the word vector sequence during the operation of the LSTM model.
[0024] Optionally, the similarity function is constructed based on Manhattan distance.
[0025] Optionally, after obtaining the question text to be searched input by the user and at least one reference question text in the reference data set, the method further includes:
[0026] The question text to be retrieved and the at least one reference question text are preprocessed respectively to obtain the preprocessed question text to be retrieved and the at least one reference question text, wherein the preprocessing includes at least one of the following: data cleaning, word segmentation processing, and text format standardization conversion.
[0027] In a second aspect, an embodiment of the present application provides a device for question retrieval, the device comprising:
[0028] An acquisition module, used to acquire a question text to be searched input by a user and at least one reference question text in a reference data set;
[0029] An execution module is used to use the to-be-retrieved question text and the reference question text as target question texts respectively, and perform a target operation, wherein the target operation includes: mapping each word in the target question text to a word vector, combining the word vectors according to the order of the corresponding words in the target question text, and obtaining a word vector sequence corresponding to the target question text; inputting the word vector sequence into an LSTM model to obtain a feature sequence of the word vectors, wherein the attention layer in the LSTM model is used to assign a weight value to each word vector in the word vector sequence, and the weight value is used to indicate the degree of influence of the word vector on the semantics of the word vector sequence; inputting the feature sequence into an FCN model to obtain a plurality of local features corresponding to the feature sequence; and fitting the plurality of local features to obtain a fitting result;
[0030] Inputting the fitting result corresponding to the to-be-searched question text and the fitting result corresponding to the reference question text into a similarity function to obtain a similarity comparison result;
[0031] If the similarity comparison result is greater than or equal to a first preset threshold, the reference question text corresponding to the similarity comparison result is output.
[0032] Optionally, the execution module is also used to input the fitting result corresponding to the question text to be retrieved and the fitting result corresponding to the reference question text into a similarity function to obtain a similarity comparison result. If the similarity comparison result is less than the first preset threshold, the question text to be retrieved is added to the reference data set.
[0033] Optionally, the execution module is further used to delete low-frequency words in the pre-training corpus before mapping each word in the target question text into a word vector; wherein the low-frequency words are words whose occurrence frequency in the pre-training corpus is lower than a second preset threshold;
[0034] Based on the pre-trained corpus after removing low-frequency words, train the continuous bag-of-words model CBOW model;
[0035] Mapping each word in the target question text into a word vector includes:
[0036] Based on the CBOW model, each word in the target question text is mapped into a word vector.
[0037] Optionally, the execution module is further used to compare the first vector space of the pre-training corpus with the second vector space of the test corpus; to determine the feature vectors that exist in the second vector space but are not included in the first vector space, and add the feature vectors to the first vector space to obtain a new pre-training corpus;
[0038] The CBOW model is trained based on the new pre-training corpus.
[0039] Optionally, the attention layer in the LSTM model is used to determine the weight value of each word vector in the word vector sequence based on a preset context vector value, and based on the weight value, calculate the weighted sum of each word vector in the word vector sequence, wherein the weighted sum is used as one of the inputs of the LSTM model to consider the degree of influence of each word vector on the semantics of the word vector sequence during the operation of the LSTM model.
[0040] Optionally, the similarity function is constructed based on Manhattan distance.
[0041] Optionally, the execution module is also used to preprocess the question text to be retrieved and the at least one reference question text respectively after obtaining the question text to be retrieved input by the user and at least one reference question text in the reference data set, so as to obtain the preprocessed question text to be retrieved and the at least one reference question text after preprocessing, wherein the preprocessing includes at least one of the following: data cleaning, word segmentation processing, and text format standardization conversion.
[0042] In a third aspect, an embodiment of the present application provides an electronic device, comprising: a processor, a memory, and a program stored in the memory and executable on the processor, wherein the program, when executed by the processor, implements the steps of the method for question retrieval as described in the first aspect.
[0043] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the method for question retrieval as described in the first aspect are implemented.
[0044] In a fifth aspect, an embodiment of the present application provides a computer program product, comprising computer instructions, which, when executed by a processor, implement the steps of the method for question retrieval as described in the first aspect.
[0045] In the embodiment of the present application, by introducing an attention layer into the LSTM (Long Short-Term Memory) model, the LSTM model can assign different weights to different word vectors to highlight the role of important vocabulary in semantic understanding, improve the model's recognition sensitivity to key information, and thus improve the accuracy of similarity calculation. And by further processing the feature sequence output by the LSTM model in combination with the FCN (Fully Convolutional Network) model, multiple local features can be extracted, so that the model can fully obtain important feature information in the question text, enhance the model's recognition of the question, and thus improve the accuracy and efficiency of question retrieval in practical applications. BRIEF DESCRIPTION OF THE DRAWINGS
[0046] Various other advantages and benefits will become apparent to those of ordinary skill in the art by reading the detailed description of the preferred embodiments below. The accompanying drawings are only for the purpose of illustrating the preferred embodiments and are not to be considered as limiting the present application. Also, the same reference symbols are used throughout the accompanying drawings to represent the same components. In the accompanying drawings:
[0047] Figure 1 A flowchart of a method for question retrieval provided in an embodiment of the present application;
[0048] Figure 2 A structural block diagram of a twin LSTM-FCN model based on an attention mechanism provided in an embodiment of the present application;
[0049] Figure 3 An overall framework diagram of a method for question retrieval provided in an embodiment of the present application;
[0050] Figure 4 A structural block diagram of a question retrieval device provided in an embodiment of the present application;
[0051] Figure 5 A structural block diagram of a network device provided in an embodiment of the present application. DETAILED DESCRIPTION
[0052] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.
[0053] Figure 1 A method for question retrieval provided according to an embodiment of the present application is shown, such as Figure 1As shown, the method includes:
[0054] Step S101, obtaining a question text to be searched input by a user and at least one reference question text in a reference data set;
[0055] Step S102: The question text to be retrieved and the reference question text are respectively used as target question texts, and target operations are performed, which include: mapping each word in the target question text to a word vector, combining the word vectors according to the order of the corresponding words in the target question text, and obtaining a word vector sequence corresponding to the target question text; inputting the word vector sequence into a long short-term memory network LSTM model to obtain a feature sequence of the word vectors, wherein the attention layer in the LSTM model is used to assign a weight value to each word vector in the word vector sequence, and the weight value is used to indicate the degree of influence of the word vector on the semantics of the word vector sequence; inputting the feature sequence into a fully convolutional network FCN model to obtain a plurality of local features corresponding to the feature sequence; and fitting the plurality of local features to obtain a fitting result;
[0056] Step 103: input the fitting result corresponding to the question text to be retrieved and the fitting result corresponding to the reference question text into a similarity function to obtain a similarity comparison result;
[0057] Step S104: If the similarity comparison result is greater than or equal to the first preset threshold, the reference question text corresponding to the similarity comparison result is output.
[0058] In the embodiment of the present application, by introducing an attention layer into the LSTM (Long Short-Term Memory) model, the LSTM model can assign different weights to different word vectors to highlight the role of important vocabulary in semantic understanding, improve the model's recognition sensitivity to key information, and thus improve the accuracy of similarity calculation. And by further processing the feature sequence output by the LSTM model in combination with the FCN (Fully Convolutional Network) model, multiple local features can be extracted, so that the model can fully obtain important feature information in the question text, enhance the model's recognition of the question, and thus improve the accuracy and efficiency of question retrieval in practical applications.
[0059] After step S101, obtaining the question text to be retrieved input by the user and at least one reference question text in the reference data set, the method further includes: preprocessing the question text to be retrieved and at least one reference question text respectively to obtain the preprocessed question text to be retrieved and at least one preprocessed reference question text, wherein the preprocessing includes at least one of the following: data cleaning, word segmentation processing, and text format standardization conversion.
[0060] It should be noted that the purpose of preprocessing is to eliminate noise in the text, standardize the format, and make the text easier to process and analyze. Preprocessing can include at least one of the following: data cleaning, such as removing noise: deleting irrelevant characters, punctuation marks, special symbols, etc.; removing stop words: removing some commonly used words that contribute less semantically, such as Chinese stop words like "的", "是", "在", etc.; unifying case: converting all text to lowercase or uppercase to reduce the impact of different forms of the same word on analysis; word segmentation: dividing continuous text into individual words or phrases. In Chinese, since there are no spaces separating words, word segmentation is an important step. Commonly used word segmentation tools include jieba, HanLP, etc. After word segmentation, the text will be converted into a list of words for subsequent processing and analysis; unifying format: converting the text into a standard format, such as converting numbers to a unified representation form, standardizing the date format, etc.; stemming or lemmatization: reducing words to their basic forms, for example, converting both "跑" and "跑步" to "跑". Through these preprocessing steps, the accuracy and efficiency of subsequent analysis can be improved. The preprocessed text data can better reflect the similarities and relationships between problems, thus providing a better basis for tasks such as information retrieval and question-answer matching.
[0061] In step S102, the attention layer in the LSTM model is used to determine the weight value of each word vector in the word vector sequence based on a preset context vector value, and calculate the weighted sum of each word vector in the word vector sequence based on the weight value. Among them, the weighted sum is used as one of the inputs of the LSTM model to consider the influence degree of each word vector on the semantics of the word vector sequence during the operation of the LSTM model.
[0062] In a possible implementation, before mapping each word in the target question text to a word vector, the method further includes: deleting low-frequency words in the pre-training corpus; wherein the low-frequency words are words whose frequency of appearance in the pre-training corpus is lower than a second preset threshold; training a CBOW (continuous bag of words) model based on the pre-training corpus after deleting the low-frequency words; correspondingly, mapping each word in the target question text to a word vector includes: mapping each word in the target question text to a word vector based on the CBOW model. Based on the pre-training corpus after deleting the low-frequency words, training the CBOW model includes: comparing the first vector space of the pre-training corpus with the second vector space of the test corpus; to determine the feature vectors that exist in the second vector space but are not included in the first vector space, adding the feature vectors to the first vector space to obtain a new pre-training corpus; and training the CBOW model based on the new pre-training corpus. Therefore, deleting low-frequency words can reduce noise and ensure that the vocabulary contained in the training data is relatively common and important to improve the training effect of the model. The CBOW model is retrained using the expanded pre-training corpus. By introducing new feature vectors, the model can better adapt to new data and improve the quality and coverage of word vectors.
[0063] In a possible implementation, after inputting the fitting result corresponding to the question text to be retrieved and the fitting result corresponding to the reference question text into the similarity function to obtain the similarity comparison result in step S103, the method further includes: if the similarity comparison result is less than a first preset threshold, adding the question text to be retrieved to the reference data set. And the similarity function is constructed based on the Manhattan distance.
[0064] Now Figure 1 The question retrieval method shown in the figure is specifically described. The embodiment of the present application proposes a twin LSTM-FCN model based on the attention mechanism (AM) (such as Figure 2 The overall framework is as follows: Figure 3As shown. First, the question text entered by the user and each question word token after preprocessing in the comparison question data set are mapped into a fixed-size vector, which is then used as the input of the two input layers of the twin LSTM model based on the attention mechanism to extract context information features and semantic characteristics, and the attention mechanism is used in the model to determine which words should be given more weight in the question. Then, the FCN model is used to capture the feature vector of the local position of the question text, and the text similarity is measured by using Manhattan distance combined with TF-IDF (Term Frequency-Inverse Document Frequency) vectorization (TF-IDF is a commonly used text vectorization method, widely used in information retrieval and text mining). Finally, the output result of the similarity function is compared with the set threshold (the threshold can be set automatically when the model is executed according to different retrieval scenarios). If the calculated value is greater than the threshold, it is sorted in order by the score and output in layers, with the top layer being the question text with the highest retrieval similarity. If the calculated value is less than the threshold, it is fed back to the question data set and updated in the next round of retrieval.
[0065] The specific implementation steps of the technical solution are:
[0066] Step 1: Preprocess the user input question text and the question text in the dataset, which mainly includes the following processes: text acquisition, data cleaning, word segmentation and text representation. In this process, punctuation, non-letters, diacritics and special characters in the text will be deleted, English letters will be lowercase, dates will be standardized to marked dates, and numbers will be standardized to symbolic numbers.
[0067] Step 2: Perform word embedding on the text representation preprocessed in step 1. This process is divided into two stages:
[0068] (1) Use the training corpus to train the CBOW model. In this process, in order to overcome the defect that the CBOW model may not be able to generate accurate word vectors for low-frequency words, set the corpus word frequency threshold Pmin (this value can be set according to the model training accuracy (for example, it can be set to 6). When the corpus is used to build a dictionary, if the number of times a word appears in the corpus is less than Pmin, it will be deleted from the dictionary.
[0069] (2) To pre-train the feature vector x that does not exist in the corpus and appears for the first time w , use the feature space Ftr associated with the test corpus to compare with the vector space Fte associated with the pre-training corpus. The formula is as follows:
[0070]
[0071] Then x w into the relevant vector space of the training corpus.
[0072] (3) Use the CBOW model to embed the user question text and the question text of the dataset preprocessed in step 1. This process can predict a keyword based on the context of the target word vector, and map each word into a vector of a fixed length. The specific execution process is as follows:
[0073] First, the context {u1,u2,...uc} where the input word vector is located is used as the input of the model input layer, and the word window length is C. Then, the input layer is connected to the hidden layer through the N*V-dimensional weight matrix W, and the output h of the hidden layer is calculated according to the following formula:
[0074]
[0075] Finally, the input vi of each node in the model output layer is calculated based on the hidden layer output h, and then the model's predicted word vector xi is obtained through the N*V-dimensional weight matrix W′. The calculation formula of this process is as follows:
[0076]
[0077] In the above formula represents the i-th column of the weight matrix W′.
[0078] Step 3: Input the word vector sequence output by the word embedding model in step 2 into the LSTM model, and use the twin LSTM-FCN model based on the attention mechanism (such as Figure 2 As shown in Figure 1, two sets of questions (one set is user input and the other set is a dataset) are compared to evaluate their semantic similarity. The specific execution process is as follows:
[0079] (1) Input the word vector sequence output by the word embedding model in step 2 into the LSTM model. The LSTM model will update its hidden state at the index of each sequence. For a given input vector xt, hidden state ht and memory state ct, the LSTM update is performed as follows:
[0080] i t =sigmoid(W i x t +U i h t-1 +b i )
[0081] f t =sigmoid(W f x t +U f ht-1 +b f )
[0082]
[0083] o t =sigmoid(W o x t +U 0 h t-1 +b 0 )
[0084] h t =o t ⊙tanh(c t )
[0085] Where it, ft, ot are the input gate, forget gate, and output gate at time t, respectively. Wk, Uk are the LSTM digitized weight matrices, bk represents the bias vector for each k in {i, f, c, o}, and ⊙ represents the element-by-element product of the matrix.
[0086] (2) In actual question retrieval, it is assumed that each word in the question contributes to the meaning of the entire question, but the influence information possessed by these words is not equal. Therefore, it is necessary to assign a probability to each word to determine the degree of its influence on the entire question. However, the traditional twin LSTM model only uses the last hidden state of the word vector sequence pair and cannot obtain the weight of each question word involved in the retrieval process, which may make some key word information unable to be captured by the model. In order to solve this problem, an attention layer is introduced into the original twin LSTM model. All hidden states H = {h1, h2, ...., hT} are used in this layer, and the context vector uh is introduced in the attention layer (this vector is randomly initialized using the word vector contained in the text to evaluate the influence of the current word on the context), and then the weight value α of each word vector is calculated. i , Finally, the question text is represented by the weighted sum S of the weights of all word vectors, and S is fed into the twin LSTM model.
[0087] (3) The output vector of the LSTM model is passed through the FCN model to obtain the local features of the question text, and multiple convolution kernels of different sizes are used to extract deeper semantic key features in the question text to ensure information features such as long-distance information relevance and position invariance. The convolution operation is performed by a filter of h*n dimensions, where h represents the convolution kernel size, n represents the word vector dimension, H is the input matrix, and S is the convolution kernel weight matrix. The calculation formula is as follows:
[0088] c i =f(S·H i:i+h-1 +b)
[0089] Where f is the Relu nonlinear activation function, b is the bias term, and Hi:i+h-1 is the output of the extracted feature ci after the convolution operation. According to the above calculation formula, each window is calculated.
[0090] c=(c 1 ,c 2 ,...,c k-h+1 )
[0091] The maximum pooling process is then used to filter out the unimportant question text features derived from the convolutional layer and retain
[0092] The most critical text feature information is used to obtain the output vector:
[0093] P = max(c i )
[0094] (4) The multiple feature vectors obtained by the pooling layer of the FCN model are refitted as input vectors, and the similarity between the question text raised by the user and the question text in the retrieval dataset is calculated using a similarity calculation function based on Manhattan distance. The formula is as follows:
[0095]
[0096] Finally, the output of the similarity function is compared with the set threshold. If the calculated value is greater than the threshold, the question text in the data set is output. If the calculated value is less than the threshold, the question text raised by the user is fed back to the question data set.
[0097] The method shown in the embodiment of this application has the following technical effects:
[0098] 1. The method shown in the embodiment of the present application no longer calculates the cosine distance between texts as a similarity measurement method, but instead uses Manhattan distance combined with TF-IDF vectorization to measure text similarity to replace cosine distance, and cooperates with word embedding based on the CBOW model to improve the model's capture of semantic information of problem words, and sets a word frequency threshold in the CBOW model training stage to reduce the impact of low-frequency words on text vector representation and enhance the accuracy of model training.
[0099] 2. During the training and deployment phase of the CBOW model, the relevant vector space of the test corpus is compared with the relevant vector space of the training corpus to detect the first-appearing feature vector and integrate it into the relevant vector space of the training corpus, thereby increasing the matching and recognition of the question text retrieval input by the user.
[0100] 3. The attention mechanism is introduced into the twin LSTM model, so that the model gives different weights to different words, further improving the retrieval accuracy of the model.
[0101] 4. Introducing FCN into the twin LSTM model to calculate and capture the feature vectors that remain unchanged in local positions, enables the model to fully obtain important feature information in the question text and enhances the model's recognition of questions.
[0102] 5. The retrieval method shown in the embodiment of the present application has a feeding mechanism. In the similarity comparison of the question text of the user question text data set, the question text of the data set will be automatically expanded to continuously enrich the question text of the data set according to the actual situation. The enriched data set can help the model better learn the similarities between questions, so that it can match and classify more accurately when facing unseen questions. In summary, the method shown in the embodiment of the present application can improve the intelligence of the product question and answer system, the accuracy of the question and answer feedback and the matching degree of the question and answer scenario, and can be implemented in a wide range of artificial intelligence products in the market (such as: question and answer system, intelligent customer service, knowledge base retrieval), thereby improving the product experience.
[0103] Figure 4 A question retrieval device provided according to an embodiment of the present application is shown, such as Figure 4 As shown, the device 40 includes:
[0104] An acquisition module 401 is used to acquire a question text to be searched input by a user and at least one reference question text in a reference data set;
[0105] The execution module 402 is used to use the question text to be retrieved and the reference question text as the target question text respectively, and perform the target operation, which includes: mapping each word in the target question text to a word vector, combining the word vectors according to the order of the corresponding words in the target question text, and obtaining a word vector sequence corresponding to the target question text; inputting the word vector sequence into the LSTM model to obtain a feature sequence of the word vectors, wherein the attention layer in the LSTM model is used to assign a weight value to each word vector in the word vector sequence, and the weight value is used to indicate the degree of influence of the word vector on the semantics of the word vector sequence; inputting the feature sequence into the FCN model to obtain a plurality of local features corresponding to the feature sequence; and fitting the plurality of local features to obtain a fitting result;
[0106] Input the fitting result corresponding to the question text to be retrieved and the fitting result corresponding to the reference question text into the similarity function to obtain a similarity comparison result;
[0107] If the similarity comparison result is greater than or equal to the first preset threshold, the reference question text corresponding to the similarity comparison result is output.
[0108] In one possible implementation, the execution module 402 is also used to input the fitting result corresponding to the question text to be retrieved and the fitting result corresponding to the reference question text into the similarity function to obtain the similarity comparison result. If the similarity comparison result is less than a first preset threshold, the question text to be retrieved is added to the reference data set.
[0109] In a possible implementation, the execution module 402 is further configured to delete low-frequency words in the pre-training corpus before mapping each word in the target question text into a word vector; wherein the low-frequency words are words whose occurrence frequency in the pre-training corpus is lower than a second preset threshold;
[0110] Based on the pre-trained corpus after removing low-frequency words, train the continuous bag-of-words model CBOW model;
[0111] Mapping each word in the target question text into a word vector includes:
[0112] Based on the CBOW model, each word in the target question text is mapped to a word vector.
[0113] In a possible implementation, the execution module 402 is further configured to compare the first vector space of the pre-training corpus with the second vector space of the test corpus; to determine a feature vector that exists in the second vector space but is not included in the first vector space, and to add the feature vector to the first vector space to obtain a new pre-training corpus;
[0114] Train the CBOW model based on the new pre-training corpus.
[0115] In one possible implementation, the attention layer in the LSTM model is used to determine the weight value of each word vector in the word vector sequence based on a preset context vector value, and based on the weight value, calculate the weighted sum of each word vector in the word vector sequence, wherein the weighted sum is used as one of the inputs of the LSTM model to consider the degree of influence of each word vector on the semantics of the word vector sequence during the operation of the LSTM model.
[0116] In one possible implementation, the similarity function is constructed based on the Manhattan distance.
[0117] In one possible implementation, the execution module 402 is also used to preprocess the question text to be retrieved and the at least one reference question text in the reference data set after obtaining the question text to be retrieved input by the user and at least one reference question text in the reference data set, respectively, to obtain the preprocessed question text to be retrieved and the at least one preprocessed reference question text, wherein the preprocessing includes at least one of the following: data cleaning, word segmentation processing, and text format standardization conversion.
[0118] In the embodiment of the present application, by introducing an attention layer into the LSTM (Long Short-Term Memory) model, the LSTM model can assign different weights to different word vectors to highlight the role of important vocabulary in semantic understanding, improve the model's recognition sensitivity to key information, and thus improve the accuracy of similarity calculation. And by further processing the feature sequence output by the LSTM model in combination with the FCN (Fully Convolutional Network) model, multiple local features can be extracted, so that the model can fully obtain important feature information in the question text, enhance the model's recognition of the question, and thus improve the accuracy and efficiency of question retrieval in practical applications.
[0119] The present application embodiment provides an electronic device 50, such as Figure 5 As shown, the network device 50 includes: a processor 501, a memory 502, and a program stored in the memory 502 and executable on the processor 501. When the program is executed by the processor 501, the steps of the method for question retrieval shown in the above embodiment are implemented.
[0120] The embodiment of the present application also provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the method for question retrieval shown in the above embodiment are implemented, and the same technical effect can be achieved. To avoid repetition, it is not repeated here. The computer-readable storage medium is, for example, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.
[0121] An embodiment of the present application also provides a computer program product, including computer instructions, which, when executed by a processor, implement the steps of the method for question retrieval shown in the above embodiment and can achieve the same technical effect. To avoid repetition, they are not described here.
[0122] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element.
[0123] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, a magnetic disk, or an optical disk), and includes a number of instructions for a terminal (which can be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) to execute the methods described in each embodiment of the present application.
[0124] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present application, ordinary technicians in this field can also make many forms without departing from the purpose of the present application and the scope of protection of the claims, all of which are within the protection of the present application.
Claims
1. A method for question retrieval, characterized in that: The method comprises: Obtaining a question text to be searched input by a user and at least one reference question text in a reference data set; The question text to be retrieved and the reference question text are respectively used as target question texts, and a target operation is performed, wherein the target operation includes: mapping each word in the target question text to a word vector, combining the word vectors according to the order of the corresponding words in the target question text, and obtaining a word vector sequence corresponding to the target question text; inputting the word vector sequence into a long short-term memory network LSTM model to obtain a feature sequence of the word vectors, wherein the attention layer in the LSTM model is used to assign a weight value to each word vector in the word vector sequence, and the weight value is used to indicate the degree of influence of the word vector on the semantics of the word vector sequence; inputting the feature sequence into a fully convolutional neural network FCN model to obtain a plurality of local features corresponding to the feature sequence; and fitting the plurality of local features to obtain a fitting result; Inputting the fitting result corresponding to the to-be-searched question text and the fitting result corresponding to the reference question text into a similarity function to obtain a similarity comparison result; If the similarity comparison result is greater than or equal to a first preset threshold, the reference question text corresponding to the similarity comparison result is output.
2. The method according to claim 1, characterized in that After inputting the fitting result corresponding to the to-be-searched question text and the fitting result corresponding to the reference question text into a similarity function to obtain a similarity comparison result, the method further includes: If the similarity comparison result is less than the first preset threshold, the question text to be retrieved is added to the reference data set.
3. The method according to claim 1, characterized in that: Before mapping each word in the target question text into a word vector, the method further includes: Deleting low-frequency words in the pre-training corpus; wherein the low-frequency words are words whose occurrence frequency in the pre-training corpus is lower than a second preset threshold; Based on the pre-trained corpus after removing low-frequency words, train the continuous bag-of-words model CBOW model; Mapping each word in the target question text into a word vector includes: Based on the CBOW model, each word in the target question text is mapped into a word vector.
4. The method according to claim 3, characterized in that Based on the pre-trained corpus after removing low-frequency words, training the CBOW model includes: Comparing the first vector space of the pre-training corpus with the second vector space of the test corpus; determining feature vectors that exist in the second vector space but are not included in the first vector space, and adding the feature vectors to the first vector space to obtain a new pre-training corpus; The CBOW model is trained based on the new pre-training corpus.
5. The method according to claim 1, characterized in that The attention layer in the LSTM model is used to determine the weight value of each word vector in the word vector sequence based on a preset context vector value, and based on the weight value, calculate the weighted sum of each word vector in the word vector sequence, wherein the weighted sum is used as one of the inputs of the LSTM model to consider the degree of influence of each word vector on the semantics of the word vector sequence during the operation of the LSTM model.
6. The method according to claim 1, characterized in that The similarity function is constructed based on Manhattan distance.
7. The method according to any one of claims 1 to 6, characterized in that After obtaining the question text to be searched input by the user and at least one reference question text in the reference data set, the method further includes: The question text to be retrieved and the at least one reference question text are preprocessed respectively to obtain the preprocessed question text to be retrieved and the at least one reference question text, wherein the preprocessing includes at least one of the following: data cleaning, word segmentation processing, and text format standardization conversion.
8. A question retrieval device, characterized in that: The device comprises: An acquisition module, used to acquire a question text to be searched input by a user and at least one reference question text in a reference data set; An execution module is used to use the to-be-retrieved question text and the reference question text as target question texts respectively, and perform a target operation, wherein the target operation includes: mapping each word in the target question text to a word vector, combining the word vectors according to the order of the corresponding words in the target question text, and obtaining a word vector sequence corresponding to the target question text; inputting the word vector sequence into an LSTM model to obtain a feature sequence of the word vectors, wherein the attention layer in the LSTM model is used to assign a weight value to each word vector in the word vector sequence, and the weight value is used to indicate the degree of influence of the word vector on the semantics of the word vector sequence; inputting the feature sequence into an FCN model to obtain a plurality of local features corresponding to the feature sequence; and fitting the plurality of local features to obtain a fitting result; Inputting the fitting result corresponding to the to-be-searched question text and the fitting result corresponding to the reference question text into a similarity function to obtain a similarity comparison result; If the similarity comparison result is greater than or equal to a first preset threshold, the reference question text corresponding to the similarity comparison result is output.
9. An electronic device, characterized in that: include: A processor, a memory, and a program stored in the memory and executable on the processor, wherein when the program is executed by the processor, the steps of the method for question retrieval as claimed in any one of claims 1 to 7 are implemented.
10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the question retrieval method according to any one of claims 1 to 7 are implemented.
11. A computer program product, characterized in that The method comprises computer instructions, which, when executed by a processor, implement the steps of the method for question retrieval as claimed in any one of claims 1 to 7.