A method for recommending words based on information retrieval and sorting
By combining information retrieval and sorting technology, using word2vec word vectors and catboost models, a dialogue database was constructed, which solved the problem of insufficient accuracy and context combination in multiple rounds of dialogue, and achieved high-quality dialogue data recommendation and effectiveness improvement.
Patent Information
- Application Number
- CN202110912199.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-08-10
- Publication Date
- 2024-11-22
- Estimated Expiration
- 2041-08-10
AI Technical Summary
The existing multi-round dialogue technology has shortcomings in combining accuracy and dialogue context. The retrieval model cannot break out of database limitations. The statements generated by the generative model are of low quality, making it difficult to provide high-quality dialogue data services.
Using a combination of information retrieval and sorting method, the database to be retrieved is constructed through word2vec word vectors, cosine similarity and catboost sorting models, and the database to be retrieved and sorted is comprehensively calculated. The most appropriate words are recommended in real time.
The quality and effectiveness of speech recommendations are improved, and the accuracy and efficiency of result output are improved by screening high-quality dialogue data and reducing the scope of information retrieval.
Smart Images

Figure CN113656542B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of computer technology, and in particular to a speech recommendation method based on information retrieval and sorting. Background Art
[0002] Multi-round dialogue is considered one of the most difficult problems in the field of artificial intelligence. It involves language understanding, reasoning, and the use of common sense knowledge. Although with the development of the Internet, data information can be better shared, and multi-round dialogue can be better studied and developed, but due to the characteristics of language itself, there is still a problem of low accuracy. One of the main reasons is the lack of a large amount of high-quality real dialogue data.
[0003] The current solutions are mainly divided into two types of models: search-based and generative. The search-based model is developed from the search-based dialogue model supported by traditional TF-IDF, BM25 and other ideas. The search-based model is mainly to find the most similar dialogue context in the database and extract the content as the answer. The generative model is mainly based on the seq2seq model. It encodes the above information and cyclically decodes it to generate the answer content. Compared with the statistical model, seq2seq is data-driven and can handle more complex situations.
[0004] Although the above two solutions can solve the problem of multi-round dialogue to a certain extent, they still have shortcomings.
[0005] Retrieval model: It is difficult to combine with the conversation context. The retrieval content is to extract existing corpus. It cannot jump out of the limitations of the retrieval database and the accuracy is not high.
[0006] Generative models: The quality of the sentences generated is often low, and they perform poorly in terms of fluency and contextual relevance.
[0007] How to accumulate these conversation data and help users better serve the public has become an urgent problem that needs to be solved. Summary of the invention
[0008] The present invention provides a speech recommendation method based on information retrieval and sorting, which integrates the two methods of information retrieval and sorting, introduces text classification, word2vec word vector, cosine similarity, catboost sorting model and other technologies, uses information retrieval to recall conversations with similar topics and similar scenes, uses a sorting model to fine-rank the recalled conversations, and comprehensively calculates indicators such as topic similarity, scene similarity, and fine-ranking order to recommend the most appropriate speech in real time.
[0009] The present invention provides a method for recommending words based on information retrieval and sorting, the method comprising:
[0010] S1: Obtain conversation data in the required field;
[0011] S2: Process the obtained dialogue data, obtain sentence vectors of different dialogue types, and build a library to be searched;
[0012] S3: Build and train a ranking model;
[0013] S4: Input the sentence vector corresponding to the question to be searched, and perform search and matching on the data in the database to be searched;
[0014] S5: Calculate the matching degree through the trained ranking model to obtain the matching score;
[0015] S6: Perform comprehensive calculation and sorting based on the obtained matching scores, and output recommended words according to the preset output sorting position.
[0016] Furthermore, the dialogue data includes questions, dialogues and responses, and the dialogue data is obtained by obtaining the dialogue data in the corresponding question and answer domain database based on the question and answer effectiveness, and the effectiveness includes business transaction rate and satisfaction.
[0017] Furthermore, processing the conversation data includes the following steps:
[0018] First, remove stop words in sentence segmentation in the conversation data;
[0019] Then, we obtain the word vector of each word and calculate the average value. We obtain the sentence vectors of the question, dialogue, and response respectively and store them to build a library to be retrieved.
[0020] Furthermore, the word vector is obtained by mapping the word to a fixed dimension through a trained word2vec model.
[0021] Further, step S3 includes the following steps:
[0022] S301: Sample negative sampling, the dialogue sentence vector and the original response sentence vector are combined into a question-answer pair positive sample, and a number of samples are randomly selected from the same expertise responses with the same dialogue round or a round difference less than a preset value to form a negative sample with the dialogue sentence vector;
[0023] S302: catboost model is used for training, the obtained positive and negative samples are constructed into triplets, a target loss function is constructed, and a sorting model is trained. The target loss function is as follows:
[0024]
[0025] Among them, R(W) is the regularization term, is the loss of triples.
[0026] Furthermore, in step S4, the questions raised by the users are divided into expertise through a BERT-based text classification model;
[0027] The retrieval is to obtain the sentence vector corresponding to the input question, and calculate its similarity with the questions of the same expertise in the to-be-searched library. The similarity adopts cosine similarity, as shown below:
[0028]
[0029] Among them, A and B represent the input question sentence vector and the question sentence vector in the to-be-retrieved library respectively, and the dialogues whose question similarity is greater than the first threshold and whose turn difference is the same or less than the preset value are recalled.
[0030] Furthermore, in step S4, the matching process is as follows:
[0031] Get the sentence vector of the input dialogue and calculate the cosine similarity between it and the sentence vector corresponding to the retrieved dialogue;
[0032] The question similarity calculated in the retrieval is added to the dialogue similarity as a weight, and responses corresponding to dialogues whose product of the question similarity and the dialogue similarity is greater than a second threshold are recalled.
[0033] Furthermore, the second threshold is smaller than the first threshold.
[0034] Furthermore, in step S5, the sentence vector of the input dialogue is concatenated with the sentence vector corresponding to the recalled response and input into the sorting model trained in step S3 to obtain a matching score between the sentence vector of the input dialogue and the sentence vector of the recalled response.
[0035] Furthermore, in step S6, the matching score includes the question similarity and dialogue similarity obtained in step S4, and the dialogue and response matching obtained in step S5.
[0036] The beneficial effects of the present invention are as follows:
[0037] 1. The present invention recalls conversations of the same field, similar subjects, and similar scenarios from the to-be-searched database through information retrieval, and then accurately sorts the recalled conversations through a trained sorting model. Finally, a comprehensive calculation is performed based on the similarity and matching degree obtained by the retrieval and matching, and the most appropriate conversation result is output. The recommended output of conversations is realized based on the two models of information retrieval and sorting, which improves the quality of the output conversation results and ensures the effectiveness of the conversations in the application process.
[0038] 2. Basic data: According to the corresponding fields, high-quality conversation data is screened, and the various types of data in the conversation data are mapped through the trained word2vec model to obtain vectors that can represent the meaning of words, and the library to be searched is obtained. The quality of the final output results is improved from the construction of the conversation database. At the same time, based on the constructed library to be searched, text classification can be performed, the scope of information retrieval can be narrowed, the amount of data processing can be reduced, and the efficiency of result output can be improved. BRIEF DESCRIPTION OF THE DRAWINGS
[0039] Figure 1 It is a schematic diagram of the overall process of the present invention;
[0040] Figure 2 It is a schematic diagram of the offline calculation process of the present invention;
[0041] Figure 3 It is a schematic diagram of the main process of online calculation of the present invention. DETAILED DESCRIPTION
[0042] The following description clearly and completely describes the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0043] Example 1
[0044] The embodiment of the present invention provides a method for recommending words based on information retrieval and sorting. Figure 1 , combined with Figure 2 and Figure 3 As shown, in this embodiment, the lawyer business field is taken as an example for explanation, and the method includes:
[0045] S1: Acquire conversation data in lawyer services from online or offline business platforms, the conversation data including questions, conversations and responses. The conversation data is acquired based on the effectiveness of questions and answers to obtain conversation data in the corresponding question and answer domain database, the effectiveness including business transaction rate and satisfaction;
[0046] In this embodiment, since the service capabilities of different lawyers are different and their experience becomes richer over time, in this embodiment, for different specialties, the customer transaction rate of each lawyer in a certain period of time is statistically obtained, and lawyers with higher transaction rate rankings are screened, and the conversation data of each lawyer during the period of stable transaction rate is obtained.
[0047] S2: Process the obtained dialogue data, obtain sentence vectors of different dialogue types, and build a library to be searched;
[0048] The specific process is as follows:
[0049] First, remove stop words in sentence segmentation in the conversation data;
[0050] The processed data is mapped to a fixed dimension through the trained word2vec model to obtain word vectors;
[0051] Then, the average value of the word vector of each word is calculated to obtain the sentence vectors of the question, dialogue and response, and the sentence vectors are stored to build a library to be searched; the question, dialogue and response are recorded as topic, query and response respectively.
[0052] The sentence vector may also be generated through one-hot, tf-idf and other methods.
[0053] S3: Build and train a ranking model;
[0054] The specific process is as follows:
[0055] S301: Sample negative sampling, the dialogue sentence vector and the original response sentence vector are combined into a question-answer pair positive sample, and a number of samples are randomly selected from the same expertise responses with the same dialogue turn or a turn difference less than a preset value to form a negative sample together with the dialogue sentence vector; the dialogue turn is recorded as turn.
[0056] S302: catboost model is used for training, the obtained positive and negative samples are constructed into triplets, a target loss function is constructed, and a sorting model is trained. The target loss function is as follows:
[0057]
[0058] Among them, R(W) is the regularization term, is the loss of triples.
[0059] S4: Input the sentence vector corresponding to the question to be searched, and perform search and matching on the data in the database to be searched;
[0060] The questions raised by users are divided into expertise through the BERT-based text classification model. In this embodiment, the questions raised can be classified into categories such as marriage and family, criminal defense, and traffic accidents, which narrows the scope of subsequent information retrieval, reduces the pressure of retrieval, and shortens the retrieval time;
[0061] The retrieval matching process is as follows:
[0062] First, obtain the question sentence vector to be input, match the question in the to-be-searched database, and calculate the similarity between the question and the question of the same expertise in the to-be-searched database. In this embodiment, the similarity adopts cosine similarity, as shown below:
[0063]
[0064] Among them, A and B represent the input question sentence vector and the question sentence vector in the to-be-retrieved library respectively, and recall conversations whose question similarity is greater than a first threshold and whose round difference is the same as or less than a preset value. In this embodiment, the first threshold is set to 0.8 and the round difference is set to 5.
[0065] After that, the sentence vector of the input dialogue is obtained, and the cosine similarity between the sentence vector corresponding to the dialogue retrieved and recalled is calculated;
[0066] adding the question similarity calculated in the retrieval as a weight to the dialogue similarity, and recalling responses corresponding to dialogues whose product of the question similarity and the dialogue similarity is greater than a second threshold;
[0067] The second threshold is smaller than the first threshold. In this embodiment, the second threshold is set to 0.7.
[0068] S5: Calculate the matching degree through the trained ranking model to obtain the matching score;
[0069] The input conversation sentence vector is concatenated with the sentence vector corresponding to the recalled response, and input into the ranking model trained in step S3 to obtain the matching score between the sentence vector of the input conversation and the sentence vector of the recalled response.
[0070] S6: Perform comprehensive calculation and sorting based on the obtained matching scores according to preset rules, and output recommended words according to the preset output ranking.
[0071] The matching score includes the question similarity and dialogue similarity obtained in step S4, and the dialogue and response matching obtained in step S5, and is sorted according to the branch results after calculation;
[0072] The topN response results are obtained as the recommended words output. In this embodiment, N is set to 5, that is, the top five result outputs are obtained.
[0073] The present invention is not limited to the above-mentioned specific embodiments, but extends to any new features or any new combination disclosed in this specification, as well as any new method or process steps or any new combination disclosed.
Claims
1. A method for recommending words based on information retrieval and sorting, characterized in that: The method comprises: S1: Obtain conversation data in the required field; S2: Process the obtained dialogue data, obtain sentence vectors of different dialogue types, and build a library to be searched; S3: Build and train the sorting model. The steps are as follows: S301: Sample negative sampling, the dialogue sentence vector and the original response sentence vector are combined into a question-answer pair positive sample, and a number of samples are randomly selected from the same expertise responses with the same dialogue rounds or a round difference less than a preset value to form a negative sample with the dialogue sentence vector; S302: catboost model is used for training, the obtained positive and negative samples are constructed into triplets, a target loss function is constructed, and a sorting model is trained. The target loss function is as follows: Among them, R(W) is the regularization term, is the loss of triples; S4: Input the sentence vector corresponding to the question to be searched, and perform search and matching on the data in the database to be searched; The questions raised by users are divided into expertise through the BERT-based text classification model; The retrieval is to obtain a sentence vector corresponding to the input question and calculate its similarity with the questions of the same expertise in the database to be searched; The matching process is as follows: Get the sentence vector of the input dialogue and calculate the cosine similarity between it and the sentence vector corresponding to the retrieved dialogue; adding the question similarity calculated in the retrieval as a weight to the dialogue similarity, and recalling responses corresponding to dialogues whose product of the question similarity and the dialogue similarity is greater than a second threshold; S5: Calculate the matching degree through the trained ranking model to obtain the matching score; The input conversation sentence vector is concatenated with the sentence vector corresponding to the recalled response, and the concatenation is input into the ranking model trained in step S3 to obtain the matching score between the sentence vector of the input conversation and the sentence vector of the recalled response; S6: Perform comprehensive calculation and sorting based on the obtained matching scores, and output recommended words according to the preset output sorting position.
2. The method for recommending words according to claim 1, characterized in that: The dialogue data includes questions, dialogues and responses. The dialogue data is obtained by obtaining the dialogue data in the corresponding question and answer domain database based on the question and answer effectiveness. The effectiveness includes business transaction rate and satisfaction.
3. The method for recommending words according to claim 1, characterized in that: Processing the conversation data includes the following steps: First, remove stop words in sentence segmentation in the conversation data; Then, we obtain the word vector of each word and calculate the average value. We obtain the sentence vectors of the question, dialogue, and response respectively and store them to build a library to be retrieved.
4. The method for recommending words according to claim 3, characterized in that: The word vector is obtained by mapping the word to a fixed dimension through the trained word2vec model.
5. The method for recommending words according to claim 1, characterized in that: The similarity adopts cosine similarity, as shown below: Among them, A and B represent the input question sentence vector and the question sentence vector in the to-be-retrieved library respectively, and the dialogues whose question similarity is greater than the first threshold and whose turn difference is the same or less than the preset value are recalled.
6. The method for recommending words according to claim 5, characterized in that: The second threshold is smaller than the first threshold.
7. The method for recommending speech techniques according to claim 6, characterized in that: In step S6, the matching score includes the question similarity and dialogue similarity obtained in step S4, and the dialogue and response matching obtained in step S5.
Citation Information
Patent Citations
Input prompting method and device for conversation robot
CN111078848A
Text processing method and related equipment
CN111125328A
Target object intelligent recommendation method
CN111859124A
Information recommendation method and device, computer equipment and storage medium
CN112307164A
