Key word combination generation model training method and device

Abstract

The application provides a keyword combination generation model training method and device, wherein the keyword combination generation model training method comprises the following steps: acquiring a sample question and a sample answer corresponding to the sample question; extracting at least one initial keyword combination of the sample question, and determining a corresponding associated document set of each initial keyword combination; determining a seed keyword combination according to the sample answer and each associated document set; determining a sample keyword combination of the sample question according to the seed keyword combination; and training a keyword combination generation model according to the sample question and the sample keyword combination until a training stop condition is reached. Not only can the speed of labeling the keyword combination of the sample question be improved, but also the accuracy of the sample keyword combination can be improved, and the problem that the keyword combination in the database built in some vertical field does not match the prior knowledge of people can be effectively avoided.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of artificial intelligence in computer technology, in particular to a keyword combination generation model training method and device, a keyword combination generation method and device, a computing device and a computer readable storage medium. BACKGROUND

[0002] With the development of artificial intelligence in the field of computer technology, the field of natural language processing has also developed rapidly. Information retrieval based on text is an important branch of the field of natural language processing. Artificial intelligence (AI) refers to the ability of an engineered (i.e., designed and manufactured) system to perceive the environment, as well as the ability to acquire, process, apply and represent knowledge. The development status of key technologies in the field of artificial intelligence includes machine learning, knowledge graph, natural language processing, computer vision, human-computer interaction, biometric identification, virtual reality / augmented reality, etc. Natural language processing (NLP) is an important research direction in the field of computer science, which studies various theories and methods that can realize effective communication between people and computers using natural language. Specific manifestations of natural language processing include machine translation, text summarization, text classification, text proofreading, information extraction, speech synthesis, speech recognition, etc. With the development of natural language processing technology, users of various application programs have increasingly high demands on the ability of application programs to accurately recognize and understand questions issued by users.

[0003] For question processing tasks, the prior art usually labels the keyword combination in different questions through supervised manual labeling when retrieving in the cold start stage of a vertical field, and then trains a machine learning model. However, based on manual labeling, it depends on human prior knowledge, and labeling without answer supervision will lead to inaccurate labeling of keyword combinations in the question to be processed, and thus the keyword combination in the database built in some vertical fields does not match the human prior knowledge. Therefore, there is an urgent need for an effective solution to solve the above problems. SUMMARY

[0004] Therefore, the embodiments of the present application provide a keyword combination generation model training method and device, a keyword combination generation method and device, a computing device and a computer readable storage medium to solve the technical defects of inaccurate keyword combination labeling and low efficiency in the prior art.

[0005] According to a first aspect of the embodiments of the present application, a keyword combination generation model training method is provided, comprising:

[0006] obtaining a sample question and a sample answer corresponding to the sample question;

[0007] extract at least one initial keyword combination of the sample question, and determine a corresponding associated document set of each initial keyword combination;

[0008] determine a seed keyword combination according to the sample answer and each associated document set;

[0009] determine a sample keyword combination of the sample question according to the seed keyword combination;

[0010] train a keyword combination generation model according to the sample question and the sample keyword combination until a training stop condition is reached.

[0011] According to a second aspect of the embodiments of the present application, a keyword combination generation method is provided, comprising:

[0012] obtaining a to-be-processed question;

[0013] inputting the to-be-processed question into a pre-trained keyword combination generation model, wherein the keyword combination generation model is trained by the keyword combination generation model training method described above;

[0014] the keyword combination generation model generates a keyword combination in response to the to-be-processed question as input.

[0015] According to a third aspect of the embodiments of the present application, a keyword combination generation model training apparatus is provided, comprising:

[0016] a first obtaining module configured to obtain a sample question and a sample answer corresponding to the sample question;

[0017] an extracting module configured to extract at least one initial keyword combination of the sample question, and determine a corresponding associated document set of each initial keyword combination;

[0018] a seed keyword combination determining module configured to determine a seed keyword combination according to the sample answer and each associated document set;

[0019] a sample keyword combination determining module configured to determine a sample keyword combination of the sample question according to the seed keyword combination;

[0020] a training module configured to train a keyword combination generation model according to the sample question and the sample keyword combination until a training stop condition is reached.

[0021] According to a fourth aspect of the embodiments of the present application, a keyword combination generation apparatus is provided, comprising:

[0022] a second obtaining module configured to obtain a to-be-processed question;

[0023] an input module configured to input the to-be-processed question to a pre-trained keyword combination generation model, wherein the keyword combination generation model is trained by the keyword combination generation model training method described above;

[0024] a generation module configured to generate a keyword combination by the keyword combination generation model in response to the to-be-processed question as input.

[0025] According to a fifth aspect of the embodiments of the present application, a computing device is provided, comprising a memory, a processor, and computer instructions stored in the memory and executable on the processor, wherein the processor executes the computer instructions to implement the steps of the keyword combination generation model training method or the keyword combination generation method.

[0026] According to a sixth aspect of the embodiments of the present application, a computer readable storage medium is provided, which stores computer instructions executable by a processor to implement the steps of the keyword combination generation model training method or the keyword combination generation method.

[0027] According to a seventh aspect of the embodiments of the present application, a chip is provided, which stores computer instructions executable by the chip to implement the steps of the keyword combination generation model training method or the keyword combination generation method.

[0028] The keyword combination generation model training method provided by the embodiments of the present application, by obtaining a sample question and a sample answer corresponding to the sample question, extracting an initial keyword combination for the sample question, and determining a set of associated documents for each initial keyword combination, further determining a seed keyword combination according to the sample answer and the set of associated documents, and then determining a sample keyword combination, and on this basis, training the keyword combination generation model according to the sample question and the sample keyword combination corresponding to the sample question until a training stop condition is reached. In this way, not only can the speed of annotating keyword combinations for sample questions be improved, but also the accuracy of sample keyword combinations can be improved, effectively avoiding the problem that keyword combinations in a database built in some vertical fields do not match prior knowledge of people. Further, by extracting initial keyword combinations in the sample question through word segmentation and filtering stop words, computing resources are saved, and the retrieval efficiency of subsequent retrieval according to keyword combinations is improved; by matching and determining a sample set of associated documents in the management document set according to the sample answer, the matching degree of the sample answer and the sample set of associated documents is ensured, and the problem that the retrieval answer is not accurate in some vertical fields due to manual annotation of keywords in the prior art is solved. BRIEF DESCRIPTION OF DRAWINGS

[0029] Figure 1 is a structural block diagram of a computing device provided by an embodiment of the present application;

[0030] Figure 2 is a flowchart of a keyword combination generation model training method provided by an embodiment of the present application;

[0031] Figure 3 is a schematic diagram of a keyword combination generation method provided by an embodiment of the present application;

[0032] Figure 4 is a flowchart applied to a specific application scenario provided by an embodiment of the present application;

[0033] Figure 5 is a structural schematic diagram of a keyword combination generation model training apparatus provided by an embodiment of the present application;

[0034] Figure 6 is a structural schematic diagram of a keyword combination generation apparatus provided by an embodiment of the present application. DETAILED DESCRIPTION

[0035] In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present application. However, the present application can be practiced without the specific details. In other instances, well-known methods, procedures, components, and circuits have not been described in detail so as not to obscure the present application.

[0036] The terminology used in this disclosure is for the purpose of describing particular embodiments only and is not intended to be limiting of one or more embodiments of the present application. As used in one or more embodiments of the present application and the accompanying claims, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms "comprises" and / or "comprising," when used in one or more embodiments of the present application, specify the presence of stated features, integers, steps, operations, elements, and / or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0037] It will be understood that, although the terms first, second, etc. can be used herein to describe various information, these terms are not intended to denote a temporal or chronological order. Rather, these terms are used solely to distinguish one from another only. For example, without departing from the scope of one or more embodiments of the present application, first can be termed second, and similarly, second can be termed first. The term "if' as used herein can be interpreted as meaning "in response to a determination” depending on context.

[0038] First, the noun terms related to one or more embodiments of the present application are explained.

[0039] Machine Learning is an interdisciplinary subject that involves statistics, system identification, approximation theory, neural networks, optimization theory, computer science, brain science, and many other fields. It studies how computers can simulate or implement human learning behavior to acquire new knowledge or skills, reorganize existing knowledge structures, and continuously improve their performance. Machine Learning is the core of artificial intelligence technology. Data-based Machine Learning is one of the important methods in modern intelligent technology, which studies the rules from observed data (samples) and uses these rules to predict future data or unobservable data. According to the learning mode, learning method, and algorithm, there are different classification methods of Machine Learning. According to the learning mode, Machine Learning is classified into supervised learning, unsupervised learning, and reinforcement learning.

[0040] Cold Start: refers to the starting stage of a product without user data.

[0041] Vertical field: refers to a professional field, such as military, government, automotive, etc.

[0042] Supervised: also known as supervised learning, is a method in machine learning that can learn or establish a pattern (function) from training data and predict new instances based on the pattern. Supervised learning uses a labeled limited training data set to establish a model through a certain learning strategy / method, and realizes the labeling (classification) / mapping of new data / instances. The most typical supervised learning algorithms include regression and classification. Supervised learning requires the classification labels of training samples to be known. The higher the accuracy of classification labels, the more representative the samples, and the higher the accuracy of the learning model. Supervised learning has been widely applied in natural language processing, information retrieval, text mining, handwriting recognition, spam detection, etc.

[0043] Unsupervised learning is to describe the structure / rule hidden in unmarked data using unmarked limited data. The most typical unsupervised learning algorithms include single-class density estimation, single-class data dimensionality reduction, clustering, etc. Unsupervised learning does not require training samples and human-labeled data, which can compress data storage, reduce computational complexity, improve algorithm speed, and avoid classification errors caused by positive and negative sample bias. It is mainly used in economic forecasting, anomaly detection, data mining, image processing, pattern recognition, etc., such as organizing large computer clusters, social network analysis, market segmentation, astronomical data analysis, etc.

[0044] TF-IDF: Term Frequency-Inverse Document Frequency, a commonly used weighting technique for information retrieval and data mining. TF stands for Term Frequency, and IDF stands for Inverse Document Frequency Index.

[0045] BM25 algorithm: an algorithm used to evaluate the relevance between search terms and documents, mainly to calculate the relevance of all words and documents in the problem.

[0046] A search engine refers to a system for providing search services to users by collecting information on the Internet according to certain strategies and using specific computer programs, displaying the processed information to users after organizing and processing the information, that is, users input keywords in the search engine for searching, and the search engine finds documents matching the keywords from an index database.

[0047] Keyword combination calculation in the search field is generally divided into two stages, a cold start stage and a user stage. For the cold start stage, only keywords in different questions can be annotated to train a machine learning model. There are mainly two ways to annotate keyword combinations, one is supervised manual annotation, that is, manually selecting important words in different question words for searching, ignoring unimportant words in the question such as mood words and interrogative words, and completely relying on prior knowledge of people; the second is to annotate keyword combinations from supervised data such as idf, ngram-idf, etc. by searching database corpus statistics.

[0048] In the present application, a keyword combination generation model training method and device, a keyword combination generation method and device, a computing device and a computer readable storage medium are provided, which are described in detail one by one in the following embodiments.

[0049] Figure 1 The structure block diagram of the computing device 100 provided by an embodiment of the present application is shown. The components of the computing device 100 include but are not limited to the memory 110 and the processor 120. The processor 120 is connected with the memory 110 through the bus 130, and the database 150 is used to save data.

[0050] The computing device 100 also includes an access device 140, which enables the computing device 100 to communicate via one or more networks 160. Examples of these networks include a public switched telephone network (PSTN), a local area network (LAN), a wide area network (WAN), a personal area network (PAN), or a combination of communication networks such as the Internet. The access device 140 can include one or more of any type of network interface (e.g., network interface card (NIC)), such as an IEEE 802.11 wireless local area network (WLAN) wireless interface, a worldwide interoperability for microwave access (Wi-MAX) interface, an Ethernet interface, a universal serial bus (USB) interface, a cellular network interface, a Bluetooth interface, a near-field communication (NFC) interface, and the like.

[0051] In an embodiment of the present application, the above-mentioned components of the computing device 100 and other components not shown in the Figure 1 The above-mentioned components of the computing device 100 and other components not shown in theFigure 1 The illustrated computing device structural block diagram is merely for the purpose of example, and is not a limitation on the scope of the present application. Those skilled in the art can add or replace other components as needed.

[0052] The computing device 100 can be any type of stationary or mobile computing device, including a mobile computer or mobile computing device (e.g., a tablet computer, a personal digital assistant, a laptop computer, a notebook computer, a netbook, etc.), a mobile phone (e.g., a smartphone), a wearable computing device (e.g., a smart watch, smart glasses, etc.), or other type of mobile device, or a stationary computing device such as a desktop computer or PC. The computing device 100 can also be a mobile or stationary server.

[0053] The processor 120 can perform the steps in the keyword combination generation model training method or the keyword combination generation method described below. Figure 2 A flowchart of a keyword combination generation model training method according to an embodiment of the present application is shown, including steps 202 to 210.

[0054] Step 202: Obtain a sample question and a sample answer corresponding to the sample question.

[0055] In the research and development stage of the product without user data, without user click behavior and user data, in order to avoid the difference caused by the priori knowledge of people and TF-IDF (Term Frequency-Inverse Document Frequency) in the retrieval database, the present application provides a keyword combination generation model training method. The keyword combination generation model training method provided by the present application is used to obtain a trained keyword combination generation model, which effectively avoids the problem that the keyword combination in the database built in some vertical field does not match the priori knowledge of people.

[0056] The keyword combination generation model training method provided by the present application uses the proposed sample question and the feedback document to form a training sample, and calculates the sample keyword combination of the sample question according to the proposed sample question and the feedback document. In the training process of the keyword combination generation model, the sample question is the sample data, and the sample answer is the correct answer that can answer the sample question.

[0057] In the present application, the sample question and the sample answer can belong to any field, for example, the sample question is "What are the types of cell division?" and the sample answer is "Mitosis and amitosis", at this time, the sample question and the sample answer belong to the field of biology, that is, the keyword combination generation model trained at this time is a keyword combination generation model for the field of biology, and the sample question and the sample answer in the field of biology can be used to train the keyword combination generation model; for example, the sample question is "What is the influencing factor of gravitational acceleration?" and the sample answer is "latitude", at this time, the sample question and the sample answer belong to the field of physics, that is, the keyword combination generation model trained at this time is a keyword combination generation model for the field of physics, and the sample question and the sample answer in the field of physics can be used to train the keyword combination generation model. That is, when training the keyword combination generation model, the field to which the keyword combination generation model belongs needs to be determined, and then the sample question and the sample answer in the field are used to train the keyword combination generation model. In addition, for the keyword combination generation model of encyclopedia, that is, the keyword combination generation model applicable to different fields, sample questions and sample answers need to be obtained from various fields, and the keyword combination generation model needs to be trained from multiple fields and multiple dimensions.

[0058] It should be noted that in actual application, if a question to be answered is provided for the keyword combination generation model, the question to be answered can be used as a sample question, and the correct answer corresponding to the sample question is obtained as a sample answer. The sample answer usually appears in multiple articles or multiple paragraphs in an article, or for the same sample question, there are multiple articles or paragraphs that can answer the sample question in the actual retrieval process.

[0059] In one specific embodiment provided in the present application, the obtained sample question is "Is 1994 a chicken year or a dog year?", and the sample answer corresponding to the sample question is "1994 is a dog year".

[0060] The sample question and the sample answer corresponding to the sample question are obtained in the cold start stage, which ensures that the article conforming to the sample question can be accurately obtained according to the sample answer, and further ensures that the keyword combination generation model can accurately obtain the keyword combination in the sample question in the training process.

[0061] Step 204: Extract at least one initial keyword combination of the sample question, and determine the associated document set corresponding to each initial keyword combination.

[0062] On the basis of obtaining the sample question and the sample answer corresponding to the sample question, further, initial keyword combinations of the sample question are extracted, and a set of associated documents corresponding to each initial keyword combination is determined.

[0063] Specifically, the initial keyword combination refers to a word group obtained by combining keywords in the sample question. The initial keyword combination can contain one keyword or multiple keywords, that is, the initial keyword combination contains at least one keyword. The set of associated documents refers to a set of recall documents obtained by searching the initial keyword combination, wherein the recall document refers to all documents obtained by searching the initial keyword combination, that is, the initial keyword combination is input into a search engine for retrieval, and the search engine finds all documents matching the initial keyword combination from an index database. Each initial keyword combination corresponds to a set of associated documents, that is, the initial keyword combination and the set of associated documents correspond one by one.

[0064] In actual application, since the sample question includes one or more keywords, at least one keyword of the sample question is extracted; at least one initial keyword combination is determined according to the at least one keyword. That is, after extracting the keywords in the sample question, at least one keyword in the sample question is obtained, and the at least one keyword is combined to obtain at least one initial keyword group of the sample question. On this basis, each initial keyword combination is searched in an existing database to obtain a set of associated documents corresponding to each initial keyword combination, wherein the existing database refers to an index database corresponding to a search engine.

[0065] For example, the sample question is “How big is the territory of China?” which includes the keywords “China”, “territory area”, and “how big”. After the keywords are extracted, the obtained keywords are combined to obtain at least one initial keyword combination. The keywords are combined individually, in pairs, and in threes to obtain seven initial keyword combinations: “China”, “territory area”, “how big”, “China, territory area”, “China, how big”, “territory area, how big”, and “China, territory area, how big”. Further, the seven initial keyword combinations are searched respectively, and the documents recalled according to “China” are taken as the set of associated documents of the initial keyword combination “China”, the documents recalled according to “territory area” are taken as the set of associated documents of the initial keyword combination “territory area”, and so on.

[0066] By acquiring the keywords in the sample question and generating an initial keyword combination, the associated document set is searched and determined according to the initial keyword combination, so that the associated document containing the initial keyword combination can be accurately acquired, and the accuracy of the subsequent sample keywords corresponding to the sample question is further improved.

[0067] In order to improve the extraction speed of the keywords and improve the accuracy of the keywords, when extracting at least one keyword of the sample question, a word segmentation processing can be used, that is, the sample question is subjected to word segmentation processing to obtain at least one keyword of the sample question.

[0068] Specifically, the word segmentation processing of the string matching in the sample question can be a forward maximum matching method, a reverse maximum matching method, a shortest path word segmentation method or a bidirectional maximum matching method. Among them, the forward maximum matching method refers to cutting out the word with the maximum length at the current position in a greedy manner from left to right (from the first word unit to the last word unit) for the input sample question; the reverse maximum matching method refers to cutting out the word with the maximum length at the current position in a greedy manner from right to left (from the last word unit to the first word unit) for the input sample question; the shortest path word segmentation method refers to regarding the word units in the sample question as nodes, first finding all the words in the sample question, regarding the two ends of a word, that is, the word tail unit and the word before the previous word, as having a connection, finding the shortest path between the head and tail units in the sample question, and realizing word segmentation; the bidirectional maximum matching method refers to comparing the word segmentation results obtained by the forward maximum matching method and the reverse maximum matching method, and determining the word segmentation result.

[0069] Optionally, when the sample question is subjected to word segmentation processing, in order to improve the word segmentation processing speed and improve the training efficiency, the forward maximum matching method can be used for word segmentation processing of the sample question, that is, the sample question is subjected to word segmentation processing to obtain at least one keyword of the sample question, and the specific implementation process can be as follows:

[0070] Acquire a preset word segmentation library and determine the number N of word units contained in the longest word segmentation in the preset word segmentation library, N being a positive integer;

[0071] The (i+1)th to (N+i)th word units of the sample question are determined as matching word segmentation, and word segmentation matching is performed in the preset word segmentation library according to the matching word segmentation, i=0;

[0072] If the matching is successful, the matching word segmentation is determined as a keyword;

[0073] If the matching fails, the last character unit of the matching segmented word is deleted, and the step of performing the segmentation matching in the preset segmentation library according to the matching segmented word is returned to be executed.

[0074] In a case where it is determined that there is no character unit in the keyword or the matching segmented word, i is increased by 1, and the step of determining the (i+1)th to (N+i)th character units of the sample question as the matching segmented word is returned to be executed, and the like is repeated until the sample question segmentation is completed.

[0075] Specifically, the preset segmentation library refers to a set containing a plurality of words which is set in advance.

[0076] For example, the sample question is "what is there in the sky", the preset segmentation library is {day, day, have, what}, and the longest segmented word is "day" or "what", that is, N is 2. The 1st to 2th character units "day" of the sample question are selected, "day" is matched with the preset segmentation library, the matching is successful, and "day" is taken as the keyword. At this time, the keyword is determined, the 2nd to 3rd character units "day" of the sample question are selected, "day" is matched with the preset segmentation library, the matching is failed, "day" in "day" is deleted, "day" is obtained, "day" is matched with the preset segmentation library, the matching is failed, "day" in "day" is deleted, and at this time, there is no character unit in the matching segmented word. The 3rd to 4th character units "day" of the sample question are selected, "day" is matched with the preset segmentation library, the matching is failed, "day" in "day" is deleted, "day" is obtained, "day" is matched with the preset segmentation library, the matching is successful, and "day" is taken as the keyword. At this time, the keyword is determined, the 4th to 5th character units "day" of the sample question are selected, "day" is matched with the preset segmentation library, the matching is successful, and "day" is taken as the keyword. At this time, the sample question segmentation is completed, and three keywords "day", "day" and "day" are obtained.

[0077] Optionally, in the segmentation processing of the sample question, in order to improve the segmentation processing speed and training efficiency, a reverse maximum matching method can also be used for the segmentation processing of the sample question, that is, at least one keyword of the sample question is obtained through the segmentation processing of the sample question, and the specific implementation process can be as follows:

[0078] A preset segmentation library is obtained, and the number M of character units contained in the longest segmented word in the preset segmentation library is determined, M being a positive integer;

[0079] The (M+j)th to (j+1)th character units from the end of the sample question are determined as a matching segmented word, segmentation matching in the preset segmentation library is performed according to the matching segmented word, and j=0;

[0080] If the matching is successful, the matching segmented word is determined as a keyword.

[0081] If the matching fails, the first word unit of the matching segmented word is deleted, and the step of performing the segmentation matching in the preset segmentation library according to the matching segmented word is returned to be executed;

[0082] In a case where it is determined that there is no word unit in the keyword or the matching segmented word, j is increased by 1, and the step of determining the (M+j)th to (j+1)th word unit from the end of the sample question as the matching segmented word is returned to be executed until the segmentation of the sample question is completed.

[0083] Specifically, the preset segmentation library refers to a set containing a plurality of words which is set in advance.

[0084] For example, the sample question is "What is life?", the preset segmentation library is {life, life, life, life, what}, and the longest segmented word is "life" or "what", that is, N is 2. The (2nd to 1st) word unit from the end of the sample question "what" is selected, "what" is matched with the preset segmentation library, the matching is successful, and "what" is taken as the keyword; At this time, the keyword is determined, the (3rd to 2nd) word unit from the end of the sample question "is what" is selected, "is what" is matched with the preset segmentation library, the matching is failed, "is" in "is what" is deleted to obtain "what", "what" is matched with the preset segmentation library, the matching is failed, "what" in "what" is deleted, at this time, there is no word unit in the matching segmented word, the (4th to 3rd) word unit from the end of the sample question "life is" is selected; "life is" is matched with the preset segmentation library, the matching is failed, "life" in "life is" is deleted to obtain "is", "is" is matched with the preset segmentation library, the matching is successful, and "is" is taken as the keyword; At this time, the keyword is determined, the (5th to 4th) word unit from the end of the sample question "life" is selected, "life" is matched with the preset segmentation library, the matching is successful, and "life" is taken as the keyword, at this time, the segmentation of the sample question is completed, and three keywords "what", "is" and "life" are obtained.

[0085] Optionally, in the segmentation processing of the sample question, in order to improve the segmentation processing speed and improve the accuracy of segmentation, a bidirectional maximum matching method can be used for the segmentation processing of the sample question, that is, the sample question is segmented to obtain at least one keyword of the sample question, and the specific implementation process can be as follows:

[0086] The sample question is segmented according to the forward maximum matching method to obtain at least one forward keyword of the sample question;

[0087] The sample question is segmented according to the reverse maximum matching method to obtain at least one reverse keyword of the sample question;

[0088] compare the number of the at least one forward keyword with the number of the at least one reverse keyword;

[0089] in a case where the number of the at least one forward keyword is less than the number of the at least one reverse keyword, determine the at least one forward keyword as at least one keyword;

[0090] in a case where the number of the at least one reverse keyword is less than the number of the at least one reverse keyword, determine the at least one reverse keyword as at least one keyword;

[0091] in a case where the number of the at least one forward keyword is equal to the number of the at least one reverse keyword, determine the at least one forward keyword or the at least one reverse keyword as at least one keyword.

[0092] Specifically, the forward keyword refers to a keyword obtained by a forward maximum matching method; and the reverse keyword refers to a keyword obtained by a reverse maximum matching method.

[0093] For example, the sample question "happy happy every day" is processed by the forward maximum matching method and the reverse maximum matching method respectively: if the obtained forward keyword is "happy happy", "every day", the obtained reverse keyword is "happy", "happy", "every day", the forward keyword "happy happy", "every day" is determined as the final keyword; if the obtained forward keyword is "happy", "happy", "every day", the obtained reverse keyword is "happy", "happy", "every day", the forward keyword "happy", "happy", "every day" or the reverse keyword "happy", "happy", "every day" is determined as the final keyword.

[0094] In addition, in order to further improve the accuracy of the word segmentation, after the comparison of the number of the at least one forward keyword with the number of the at least one reverse keyword, the method further comprises:

[0095] in a case where the number of the at least one forward keyword is equal to the number of the at least one reverse keyword, determine a first number of forward keywords composed of single word units in the at least one forward keyword, and / or a second number of reverse keywords composed of single word units in the at least one reverse keyword;

[0096] if the first number is less than the second number, determine the at least one forward keyword as at least one keyword;

[0097] if the first number is greater than or equal to the second number, determine the at least one reverse keyword as at least one keyword.

[0098] For example, by using the forward maximum matching method to process the sample question "study the origin of life", the forward keywords obtained are "study", "life", "origin", and the first quantity is 2; by using the reverse maximum matching method to process the sample question "study the origin of life", the reverse keywords obtained are "study", "life", "origin", and the second quantity is 1. Since 2 is greater than 1, the reverse keywords "study", "life", "origin" are determined as the final keywords.

[0099] In practical applications, after obtaining the sample question, the sample question is processed by word segmentation to obtain the keywords, for example, the sample question "what are the common cylinders?" is processed by word segmentation, and three keywords "common", "cylinder" and "what are" are obtained.

[0100] Since the keywords obtained by processing the sample question by word segmentation have many keywords without actual meaning, these keywords without actual meaning not only occupy storage space, but also consume a part of data processing energy when processing the keywords. Therefore, before generating the initial keyword combination according to the keywords, these keywords without actual meaning can be deleted or filtered, and the specific implementation process can be as follows:

[0101] Processing the sample question by word segmentation to obtain an initial keyword set;

[0102] Filtering the stop words in the initial keyword set to obtain at least one keyword of the sample question.

[0103] Specifically, the stop word refers to in information retrieval, in order to save storage space and improve search efficiency, some characters or words will be automatically filtered before or after processing natural language text, and these characters or words are called stop words. These stop words are generally input by artificial and non-automatically generated, and the generated stop words form a stop word table.

[0104] In practical applications, all the keywords obtained by processing the sample question by word segmentation are regarded as a whole, that is, an initial keyword set, and the keywords without actual meaning in the initial keyword set, that is, the stop words, are deleted or filtered, and the remaining keywords in the initial keyword set, that is, the keywords corresponding to the sample question.

[0105] For example, after the word segmentation processing is performed on the sample question "How many people participate in the national college entrance examination in 2021?", the initial keyword set {“2021 year”, “nationwide”, “participate”, “college entrance examination”, “of”, “number”, “how many”} is obtained, wherein the stop words include “of” and “how many”, and the remaining keywords in the initial keyword set are the keywords of the sample, that is, the keywords of “How many people participate in the national college entrance examination in 2021?” include “2021 year”, “nationwide”, “participate”, “college entrance examination” and “number”.

[0106] By performing the stop word removal processing on the initial keyword set, the redundant data in the initial keyword combination is removed, the computing resources are saved, the accuracy of the keywords extracted from the sample question is further improved, the accuracy of determining the sample keyword combination is ensured, and the retrieval efficiency of subsequent retrieval according to the keyword combination is improved.

[0107] When searching according to the initial keyword combination, hundreds or thousands of associated documents can be searched, and if all the associated documents are processed, the data processing amount will be greatly increased and the processing speed will be reduced. In addition, generally, the lower the ranking or the later the sorting of the associated document obtained by searching for an initial keyword combination, the lower the relevance of the associated document to the initial keyword combination. Therefore, the associated documents obtained by searching can be screened to obtain a set of key documents, and the specific implementation process can be as follows:

[0108] searching for initial associated documents corresponding to each initial keyword combination;

[0109] screening the initial associated documents corresponding to each initial keyword combination to generate a set of associated documents corresponding to each initial keyword combination.

[0110] Specifically, the initial associated document refers to a document obtained by searching according to a certain initial keyword combination, and the set of associated documents is a set of remaining initial associated documents after screening all initial associated documents.

[0111] In actual application, when searching for an initial keyword combination, hundreds or thousands of initial associated documents can be searched, and if all the initial associated documents are processed, the data processing amount will be greatly increased and the processing speed will be reduced. In order to save computing resources and improve data processing speed, after obtaining the initial associated documents, the obtained initial associated documents need to be screened, and a screening condition can be set, for example, the top 40 initial associated documents are selected, that is, the top 40 initial associated document combinations in the initial associated documents are selected to form a set of associated documents.

[0112] For example, the initial keyword combination is "English, composition", a search is performed according to "English, composition", 1000 initial associated documents are obtained, and the top 50 initial associated documents are selected as the associated document set of "English, composition" according to the screening condition.

[0113] It should be noted that the display order of the initial associated documents, i.e., the sorting of the initial associated documents, is determined when the initial associated documents are searched according to the initial keyword combination. For example, a search is performed according to "flowers, cultivation", and the sorting of the initial associated documents is first.

[0114] By obtaining the initial keyword combination in the sample question and performing a search according to the initial keyword combination, the associated document set is obtained, which further ensures that the sample keyword combination can be accurately obtained and the problem that the keyword combination in the database built in some vertical field does not match the prior knowledge of a person is effectively avoided.

[0115] Step 206: determining a seed keyword combination according to the sample answer and each associated document set.

[0116] After the initial keyword combination of the sample question is extracted and the associated document set corresponding to each initial keyword combination is determined, the sample answer and each document set need to be matched to determine the seed keyword combination in the initial keyword combination.

[0117] Specifically, the seed keyword combination refers to a keyword combination in the initial keyword combination that can be an alternative sample keyword combination.

[0118] In actual application, in order to ensure the relevance of the finally determined sample keyword combination and the sample question, the sample answer needs to be matched with the associated document set of each initial keyword combination, the initial keyword combination corresponding to the associated document set that can be successfully matched is taken as an alternative sample keyword combination, that is, at least one seed keyword combination is determined. The associated document set of the seed keyword combination contains an associated document that matches the sample answer.

[0119] For example, there are three initial keyword combinations: KC1, KC2, and KC3. The associated document set corresponding to KC1 is C1, the associated document set corresponding to KC2 is C2, and the associated document set corresponding to KC3 is C3. The associated document set C2 and the associated document set C3 both match the sample answer, so the initial keyword combination KC2 corresponding to the associated document set C2 and the initial keyword combination KC3 corresponding to the associated document set C3 are determined as the seed keyword combination.

[0120] The sample answer and the associated document set are matched to determine the seed keyword combination, which further narrows the range of the keyword combination, and prepares for more accurate determination of the sample keyword combination corresponding to the sample answer, thereby improving the accuracy of the determination of the sample keyword combination.

[0121] In the process of determining the seed keyword combination according to the sample answer and each associated document set, the sample answer and the associated document set are matched to determine the sample associated document set, and then the seed keyword combination is determined. This can improve the accuracy of the seed keyword combination and improve the efficiency of determining the seed keyword. The specific implementation process can be as follows:

[0122] Determine the sample associated document set according to the sample answer and each associated document set.

[0123] Determine the initial keyword combination corresponding to the sample associated document set as the seed keyword combination.

[0124] Specifically, the sample associated document set refers to the sample associated document set in the associated document set that matches the sample answer.

[0125] In practical applications, the sample answer is matched with each associated document set, and at least one associated document set that matches the sample answer is marked as a sample associated document set, i.e., at least one sample associated document set is determined. On this basis, the initial keyword combination corresponding to the sample associated document set is determined as the seed keyword combination.

[0126] For example, there are three initial keyword combinations: A, B, and C. Among them, the initial keyword combination A corresponds to the associated document set a; the initial keyword combination B corresponds to the associated document set b; and the initial keyword combination C corresponds to the associated document set c. The associated document sets a, b, and c are matched with the sample answer, wherein the associated document set a matches the sample answer, the associated document set a is determined as the sample associated document set, and the initial keyword combination A corresponding to the associated document set a is determined as the seed keyword combination.

[0127] In the process of determining the sample associated document set according to the sample answer and each associated document set, the sample answer is matched with each associated document in each associated document set to determine the sample associated document set. The specific implementation process can be as follows:

[0128] Match the sample answer with the associated document in each associated document set;

[0129] Determine the associated document that matches the sample answer as the sample associated document, and determine the associated document set corresponding to the sample associated document as the sample associated document set.

[0130] Specifically, the matching can be an accurate matching, a fuzzy matching or other matching, which is not limited in the present application; the associated document is a document constituting an associated document set; the sample associated document refers to an associated document with a high matching degree with the sample answer.

[0131] In actual application, the sample answer can be matched with each associated document in each associated document set, and when the matching degree of the sample answer with an associated document is higher than a certain value, the associated document is determined as a sample associated document, for example, the matching threshold is set as 75, and when the matching degree of the sample answer with an associated document is 80, the associated document is determined as a sample associated document. After the sample associated document is determined, the associated document set corresponding to the sample associated document is determined as a sample associated document set.

[0132] For example, there are three associated document sets: T1, T2 and T3, and the matching threshold is 80. Among them, the associated document set T1 includes five associated documents: t1, t2, t3, t4 and t5; the associated document set T2 includes five associated documents: t6, t7, t8, t9 and t10; and the associated document set T3 includes five associated documents: t11, t12, t13, t14 and t15. Among them, the matching degree of the associated document t8 and the associated document t12 with the sample answer is greater than 80, and then the associated document t8 and the associated document t12 are determined as sample associated documents, and the associated document set T2 corresponding to the associated document t8 and the associated document set T3 corresponding to the associated document t12 are determined as sample associated document sets.

[0133] The sample associated document set is determined through the sample answer and the associated document set, and then the seed keyword combination is determined, so as to ensure the accuracy of determining the sample keyword combination corresponding to the sample question.

[0134] Step 208: determining the sample keyword combination of the sample question according to the seed keyword combination.

[0135] After the seed keyword combination is determined according to the sample answer and each associated document set, the sample keyword combination in at least one seed keyword combination is further determined.

[0136] Specifically, the sample keyword combination refers to the finally determined keyword combination of the sample question, and each sample question corresponds to a sample keyword combination.

[0137] In actual application, one seed keyword combination is selected as the sample keyword combination from the at least one seed keyword combination according to certain conditions. The sample keyword combination can be selected according to the number of keywords contained in the seed keyword combination, and can also be selected according to the matching degree of the sample answer with the associated document set corresponding to the seed keyword combination. There are various ways to determine the sample keyword combination, which are not limited in the present application.

[0138] In order to improve the speed of determining the sample keyword combination, the seed keyword combination containing the most keywords in the seed keyword combination is determined as the sample keyword combination. For example, there are three seed keyword combinations, the first seed keyword combination contains 3 keywords, the second seed keyword combination contains 5 keywords, and the third seed keyword combination contains 2 keywords. The second seed keyword combination is determined as the sample keyword combination.

[0139] It should be noted that when the number of keywords contained in the multiple seed keyword combinations is the same and the highest, the number of associated documents contained in the associated document set corresponding to the seed keyword combination can be compared, and the seed keyword combination corresponding to the associated document set containing the most associated documents is selected as the sample keyword combination. If the number of associated documents contained in the associated document set of multiple seed keyword combinations is the same and the highest at this time, one of the seed keyword combinations can be randomly selected as the sample keyword combination.

[0140] In order to improve the credibility of determining the sample keyword combination, when the sample keyword combination of the sample question is determined according to the seed keyword combination, the following process can also be used:

[0141] Obtaining the sample associated document in the associated document set corresponding to each seed keyword combination that matches the sample answer;

[0142] Obtaining the ranking of the sample associated document in the associated document set corresponding to each seed keyword combination;

[0143] Determining the sample keyword combination of the sample question in the seed keyword combination according to the ranking of the sample associated document.

[0144] In actual application, the sample associated document in the associated document set corresponding to each seed keyword combination that matches the sample answer is obtained first, and then the ranking of the sample associated document in the corresponding associated document set, i.e. the serial number, is further determined. For example, the sample associated document m is the third associated document in the corresponding associated document set M, and the ranking of the sample associated document m is 3. After the ranking of each sample associated document is determined, one is selected from the at least one seed keyword combination as the sample keyword combination of the sample question according to the ranking of each sample associated document.

[0145] Preferably, in order to improve the efficiency of determining the sample keyword combination, when determining the sample keyword combination of the sample problem in the seed keyword combination according to the ranking of the sample associated document, the seed keyword combination corresponding to the sample associated document with the ranking within a preset ranking range is determined as the sample keyword combination of the sample problem; and / or the seed keyword combination corresponding to the sample associated document with the highest ranking is determined as the sample keyword combination of the sample problem.

[0146] Specifically, after determining the ranking of each sample associated document, a ranking range is preset, and the seed keyword combination corresponding to the sample associated document with the ranking within the ranking range is determined as the sample keyword combination of the sample problem; when there are multiple sample associated documents with the ranking within the ranking range, the rankings of each sample associated document are compared, and the seed keyword combination corresponding to the sample associated document with the highest ranking is determined as the sample keyword combination of the sample problem. Alternatively, the rankings of each sample associated document can be directly compared, and the seed keyword combination corresponding to the sample associated document with the highest ranking is determined as the sample keyword combination of the sample problem.

[0147] For example, there are five seed keyword combinations: the ranking of the sample associated document in the associated document set corresponding to the first seed keyword combination is 5; the ranking of the sample associated document in the associated document set corresponding to the second seed keyword combination is 3; the ranking of the sample associated document in the associated document set corresponding to the third seed keyword combination is 6; the ranking of the sample associated document in the associated document set corresponding to the fourth seed keyword combination is 10; and the ranking of the sample associated document in the associated document set corresponding to the fifth seed keyword combination is 2. When the ranking range is the top 4, the second seed keyword combination and the fifth seed keyword combination meet the condition, and since 2 is higher than 3, the fifth seed keyword combination is determined as the sample keyword combination. Alternatively, the rankings of each sample associated document are directly compared, and since 2>3>5>6>10 in the ranking, the fifth seed keyword combination is determined as the sample keyword combination.

[0148] It should be noted that when the rankings of the associated documents corresponding to multiple seed keyword combinations are the same and the highest, the associated document sets corresponding to these seed keyword combinations can be compared in terms of the number of associated documents contained, and the seed keyword combination corresponding to the associated document set containing the most associated documents is selected as the sample keyword combination. If at this time there are still multiple seed keyword combinations with the number of associated documents contained being the same and the highest, the seed keyword combination containing the least keywords is selected as the sample keyword combination. If there are still multiple seed keyword combinations containing the same number of keywords and the least, one of these seed keyword combinations can be randomly selected as the sample keyword combination.

[0149] Referring to Table 1, first, the sample associated document has the top-ranked and second-ranked seed keyword combination 1, seed keyword combination 2, seed keyword combination 4, and seed keyword combination 5; at this time, the number of associated documents in the associated document set corresponding to the seed keyword combination 1, seed keyword combination 2, seed keyword combination 4, and seed keyword combination 5 is compared, at this time, the seed keyword combination 1, seed keyword combination 4, and seed keyword combination 5 meet the condition; further, the number of keywords is screened, the number of keywords contained in the seed keyword combination 1 and the seed keyword combination 5 is the least and the same, at this time, one of the seed keyword combination 1 and the seed keyword combination 5 can be randomly selected as the sample keyword combination, that is, the seed keyword combination 1 can be selected as the sample keyword combination, or the seed keyword combination 5 can be selected as the sample keyword combination.

[0150] Table 1: Parameters of seed keyword combinations

[0151]

[0152] By analyzing at least one seed keyword combination, the sample keyword combination of the sample problem is determined from the at least one seed keyword, which ensures that the keyword combination generation model can accurately obtain the sample keyword combination corresponding to the sample problem, and improves the accuracy of the keyword combination generation model.

[0153] Step 210: training the keyword combination generation model according to the sample problem and the sample keyword combination until a training stop condition is reached.

[0154] In the case of obtaining the sample keyword combination corresponding to the sample problem, further, the sample problem is used as sample data, and the sample keyword combination corresponding to the sample problem is used as a sample label to train the keyword combination generation model until a training stop condition is reached, and the training of the keyword combination generation model is stopped. The training stop condition can be that the loss value is less than a preset target value, that is, when the loss value is less than the target value, the training of the keyword combination generation model is stopped; the loss value can also stop decreasing, that is, when the loss value stops decreasing, the training of the keyword combination generation model is stopped; or the number of iterations reaches a target number of iterations, that is, when the number of iterations of the keyword combination generation model reaches the target number of iterations, the training of the keyword combination generation model is stopped.

[0155] For example, the sample question is "What are the ethnic minorities in our country?", and the corresponding sample keyword combination is "our country, ethnic minorities". The sample data is "What are the ethnic minorities in our country?", and the sample label pair keyword combination is "our country, ethnic minorities". The keyword combination generation model is trained, and when the iteration number of the keyword combination generation model reaches the target iteration number, the training of the keyword combination generation model is stopped.

[0156] In an optional implementation of the present embodiment, the specific implementation process of training the keyword combination generation model according to the sample question and the sample keyword combination is as follows:

[0157] The sample question is input into the keyword combination generation model.

[0158] The keyword combination generation model generates a predicted keyword combination in response to the sample question as input.

[0159] A loss value is calculated according to the predicted keyword combination and the sample keyword combination.

[0160] The parameters of the keyword combination generation model are adjusted according to the loss value.

[0161] Specifically, the predicted keyword combination is the keyword combination generated by the keyword combination generation model according to the sample question. There are many loss functions for calculating the loss value, such as cross-entropy loss function, L1 norm loss function, maximum loss function, mean square error loss function, and logarithmic loss function. In the present application, the selection of the loss function for calculating the loss value is not limited.

[0162] In actual application, after obtaining the sample question and the sample keyword combination corresponding to the sample question, the sample question is input into the keyword combination generation model. The keyword combination generation model analyzes the sample question after receiving the sample question, and generates a predicted keyword combination corresponding to the sample question. On this basis, the loss value of the predicted keyword combination and the sample keyword combination is calculated by a loss function, and further, the parameters of the keyword combination generation model are adjusted according to the loss value for continuous training.

[0163] In the above example, after inputting the sample question "What are the ethnic minorities in our country?" into the keyword combination generation model, the keyword combination generation model generates the predicted keyword combination "our country, ethnic minorities". The loss value of the predicted keyword combination "our country, ethnic minorities" and the sample keyword combination "our country, ethnic minorities" is calculated by a loss function, and the model parameters in the keyword combination generation model are adjusted based on the loss value.

[0164] Preferably, when calculating the loss value between the predicted keyword combination and the sample keyword combination, an L1 norm loss function can be used, that is, the loss value is calculated according to the predicted keyword combination and the sample keyword combination, comprising:

[0165] performing feature extraction on the sample keyword combination to obtain a first feature vector;

[0166] performing feature extraction on the predicted keyword combination to obtain a second feature vector;

[0167] determining the absolute value of the difference between the second feature vector and the first feature vector;

[0168] calculating the loss value according to the absolute value and the weight of the preset sample keyword combination.

[0169] Specifically, the weight of the preset sample keyword combination refers to the weight of the sample keyword combination set in advance in the L1 norm loss function.

[0170] In actual application, the sample keyword combination and the predicted keyword combination can be converted to obtain the first feature vector and the second feature vector, and then the first feature vector and the second feature vector are input into the formula as shown in formula 1 to determine the absolute value of the difference between the first feature vector and the second feature vector, and then the absolute value is multiplied by the weight of the preset sample keyword combination to obtain the loss value.

[0171] L(X,Y)=W(θ)|Y-X| (Formula 1)

[0172] In formula 1, X represents the first feature vector, Y represents the second feature vector, W(θ) represents the weight of the preset sample keyword combination, and L(X,Y) represents the loss value.

[0173] The L1 norm loss function is used to find the difference between the second feature vector corresponding to the predicted keyword combination and the first feature vector corresponding to the sample keyword combination and take the absolute value, which is insensitive to the output deviating from the preset sample keyword combination, that is, the predicted keyword combination. Therefore, when the loss value exists in the process of training the keyword combination generation model, it is beneficial to maintain the stability of the keyword combination generation model. In addition, the L1 norm loss function is used to calculate the loss value, which is simple and fast, and is beneficial to improve the speed of calculating the loss value and further improve the training speed of the keyword combination generation model.

[0174] Alternatively, when calculating the loss value between the predicted keyword combination and the sample keyword combination, a mean square error loss function can also be used, that is, the loss value is calculated according to the predicted keyword combination and the sample keyword combination, comprising:

[0175] perform feature extraction on the sample keyword combination to obtain a first feature vector;

[0176] perform feature extraction on the predicted keyword combination to obtain a second feature vector;

[0177] determine a square of a distance between the second feature vector and the first feature vector to obtain a distance square value;

[0178] calculate a mathematical expectation of the distance square value to obtain the loss value.

[0179] Specifically, the distance square value refers to a value obtained by squaring the distance between the first feature vector and the second feature vector; the mathematical expectation, i.e., the mean, refers to the mean of a plurality of distance square values.

[0180] In actual application, the sample keyword combination and the predicted keyword combination can be converted to obtain the first feature vector and the second feature vector, and then the first feature vector and the second feature vector are input into a formula as shown in Formula 2 to first determine the distance between the first feature vector and the second feature vector, then solve the square of the distance to obtain the distance square value, and then calculate the mean or mathematical expectation of the distance square value to obtain the loss value.

[0181]

[0182] In Formula 1, X represents the first feature vector, Y represents the second feature vector, E(Y-X) 2 represents the mean or mathematical expectation of the distance square value, and MSE(X, Y) represents the loss value; n represents the number of the second feature vectors or the first feature vectors or the sample keyword combinations; x i represents the i-th first feature vector; y i represents the i-th second feature vector.

[0183] By using the mean square error loss function, the square of the distance between the second feature vector corresponding to the predicted keyword combination and the first feature vector corresponding to the sample keyword combination is obtained to obtain the distance square value, and then the mathematical expectation of the distance square value is determined to obtain the loss value. With continuous training, the loss value quickly converges to the minimum value, which is conducive to improving the efficiency of training the keyword combination generation model.

[0184] The keyword combination generation model training method provided in the embodiments of the present application can improve the speed of labeling the keyword combination of the sample question and the accuracy of the sample keyword combination, effectively avoiding the problem that the keyword combination in the database built in some vertical field does not match the prior knowledge of people. Further, the initial keyword combination is extracted from the sample question by word segmentation and stop word filtering, which saves computing resources and improves the retrieval efficiency of subsequent retrieval based on the keyword combination; the sample associated document set is matched and determined in the management document set according to the sample answer, which ensures the matching degree of the sample answer and the sample associated document set, and solves the problem that the answer is not accurate in some vertical fields due to manual labeling of keywords in the prior art.

[0185] Figure 3 The keyword combination generation method provided in the embodiments of the present application is shown, which includes steps 302 to 306.

[0186] Step 302: obtaining a to-be-processed question.

[0187] Specifically, the to-be-processed question is a question that needs to determine a keyword combination, that is, a question raised by a user, which can be a question manually input by a user or a question input by a user through a voice function, and the present application does not limit this.

[0188] In actual application, after the keyword combination generation model training is completed, the question raised by a user can be received, and the question raised by the user is taken as a to-be-processed question. For example, the user raises a question "What movies were released in March 2021?", and the question raised by the user "What movies were released in March 2021?" is taken as a to-be-processed question.

[0189] In the present application, by obtaining a to-be-processed question, it is ensured that the keyword combination generation model can receive valid input content, avoiding errors caused by invalid input content and improving user experience.

[0190] Step 304: inputting the to-be-processed question into a pre-trained keyword combination generation model, wherein the keyword combination generation model is trained by the keyword combination generation model training method.

[0191] On the basis of obtaining the to-be-processed question, further, the to-be-processed question can be input to the pre-trained keyword combination generation model, so as to generate a keyword combination based on the to-be-processed question.

[0192] As used in the above example, the question "What movies were released in March 2021?" raised by the user is input to the pre-trained keyword combination generation model, that is, the to-be-processed question is input to the pre-trained keyword combination generation model.

[0193] It should be noted that the keyword combination generation model is pre-trained, that is, the model is trained by the above-mentioned keyword combination generation model training method.

[0194] Step 306: The keyword combination generation model generates a keyword combination in response to the to-be-processed question as input.

[0195] After inputting the to-be-processed question to the pre-trained keyword combination generation model, further, the keyword combination generation model will generate a keyword combination corresponding to the to-be-processed question.

[0196] In practical application, after the keyword combination generation model obtains the input to-be-processed question, it finally generates a keyword combination corresponding to the to-be-processed question. As used in the above example, after inputting the to-be-processed question "What movies were released in March 2021?" to the trained keyword combination generation model, the keyword combination generation model will generate a keyword combination corresponding to "What movies were released in March 2021?": {"March 2021", "release", "movie"}.

[0197] The keyword combination generation method provided in the present application inputs the to-be-processed question to the pre-trained keyword combination generation model, and then determines the keyword combination of the to-be-processed question, which ensures that the keyword combination of the to-be-processed question can be quickly and accurately obtained, and ensures that the problem of inaccurate keyword combination caused by manually labeling the keyword combination in the to-be-processed question can be avoided, and the problem of mismatch between the keyword combination in the database built in some vertical field and the prior knowledge of people can be effectively avoided.

[0198] As Figure 4 shown, Figure 4 a flowchart applied in a specific application scenario is shown, which includes steps 402 to 420.

[0199] Step 402: Obtain a sample question and a sample answer corresponding to the sample question.

[0200] In the cold start stage, in order to train the keyword combination generation model, a sample question and a corresponding sample answer need to be set, and in this embodiment, "How many is the net increase of population in 2015?" is taken as the sample question, and "The net increase of population in 2015 is 7.1 million" is taken as the sample answer. In actual application, in order to train the keyword combination generation model, a plurality of sample questions and corresponding sample answers need to be obtained, and in this embodiment, one of the sample questions and the corresponding sample answer is taken as an example.

[0201] Step 404: performing word segmentation processing and filtering stop word processing on the sample question to obtain at least one keyword corresponding to the sample question.

[0202] After performing word segmentation processing on the sample question "How many is the net increase of population in 2015?", four keywords "2015", "nationwide", "net increase of population" and "how many" are obtained, and then the stop words in the four keywords are filtered to obtain three final keywords "2015", "nationwide" and "net increase of population".

[0203] Step 406: determining at least one initial keyword combination according to the at least one keyword.

[0204] The three keywords "2015", "nationwide" and "net increase of population" are combined to obtain six initial keyword combinations, which are initial keyword combination P1 "2015", initial keyword combination P2 "nationwide", initial keyword combination P3 "net increase of population", initial keyword combination P4 "2015, nationwide", initial keyword combination P5 "2015, net increase of population" and initial keyword combination P6 "2015, nationwide, net increase of population".

[0205] Step 408: determining a set of associated documents corresponding to each initial keyword combination.

[0206] According to the initial keyword combination P1 "2015", the set of associated documents Q1 corresponding to the initial keyword combination P1 is determined; according to the initial keyword combination P2 "nationwide", the set of associated documents Q2 corresponding to the initial keyword combination P2 is determined; according to the initial keyword combination P3 "net increase of population", the set of associated documents Q3 corresponding to the initial keyword combination P3 is determined; according to the initial keyword combination P4 "2015, nationwide", the set of associated documents Q4 corresponding to the initial keyword combination P4 is determined; according to the initial keyword combination P5 "2015, net increase of population", the set of associated documents Q5 corresponding to the initial keyword combination P5 is determined; and according to the initial keyword combination P6 "2015, nationwide, net increase of population", the set of associated documents Q6 corresponding to the initial keyword combination P6 is determined.

[0207] Step 410: determining a seed keyword combination according to the sample answer and each set of associated documents.

[0208] The sample answer "The net increase in population nationwide in 2015 was 7.1 million" is matched with the sets of associated documents Q1-Q6, it is determined that the sets of associated documents Q4, Q5 and Q6 match the sample answer, and the initial keyword combinations P4, P5 and P6 corresponding to the sets of associated documents Q4, Q5 and Q6 are determined as the seed keyword combinations.

[0209] Step 412: determining a sample keyword combination of the sample question according to the seed keyword combination.

[0210] Further, the seed keyword combinations P4, P5 and P6 contain 2, 2 and 3 keywords respectively, and the seed keyword combination P6 "2015, nationwide, net increase in population" is determined as the sample keyword of the sample question.

[0211] Alternatively, the sample associated documents in the sets of associated documents corresponding to the seed keyword combinations P4, P5 and P6 are ranked as the 2nd, 4th and 1st respectively, and the seed keyword combination P6 "2015, nationwide, net increase in population" is determined as the sample keyword of the sample question.

[0212] Step 414: inputting the sample question and the sample keyword as a training sample into the keyword combination generation model and training the keyword combination generation model until a training stop condition is reached, to obtain a trained keyword combination generation model.

[0213] The keyword combination generation model is trained according to a plurality of sample questions and sample keyword combinations corresponding to the sample questions until the loss value of the keyword combination generation model is less than a preset threshold, and thus a trained keyword combination generation model is obtained.

[0214] Step 416: obtaining a to-be-processed question.

[0215] The user inputs the question "What are the hazards of smoking on the body?" for query, and the to-be-processed question obtained at this time is "What are the hazards of smoking on the body?".

[0216] Step 418: inputting the to-be-processed question into the trained keyword combination generation model.

[0217] The to-be-processed question "What are the hazards of smoking on the body?" is inputted into the trained keyword combination generation model.

[0218] Step 420: the keyword combination generation model outputs a keyword combination.

[0219] The keyword combination generation model generates the keyword combination "smoking, body, harm" as an input in response to the question to be processed "What are the harms of smoking to the body?"

[0220] The keyword combination generation model training method provided by the embodiments of the present application extracts the initial keyword combination of the sample question and determines the associated document set of each initial keyword combination, further determines the seed keyword combination according to the sample answer and the associated document set, and then determines the sample keyword combination. On this basis, the keyword combination generation model is trained according to the sample question and the sample keyword combination corresponding to the sample question until the training stopping condition is reached. In this way, not only the speed of labeling the keyword combination of the sample question can be improved, but also the accuracy of the sample keyword combination can be improved, effectively avoiding the problem that the keyword combination in the database built in some vertical fields does not match the prior knowledge of people. Further, the initial keyword combination is extracted from the sample question by means of word segmentation and stop word filtering, saving computing resources and improving the retrieval efficiency of subsequent retrieval according to the keyword combination; the sample associated document set is matched and determined in the management document set according to the sample answer, ensuring the matching degree of the sample answer and the sample associated document set, and solving the problem that the answer is not accurate in some vertical fields in the prior art due to manual labeling of keywords.

[0221] The keyword combination generation method provided by the present application inputs the question to be processed into the pre-trained keyword combination generation model, and then determines the keyword combination of the question to be processed, ensuring that the keyword combination of the question to be processed can be quickly and accurately obtained, ensuring that the problem of inaccurate keyword combination caused by manual labeling of keywords in the question to be processed can be avoided, and effectively avoiding the problem that the keyword combination in the database built in some vertical fields does not match the prior knowledge of people.

[0222] Corresponding to the keyword combination generation model training method embodiment described above, the present application also provides a keyword combination generation model training device embodiment, Figure 5 The structure schematic diagram of the keyword combination generation model training device provided by an embodiment of the present application is shown. As Figure 5 shown, the device 500 includes:

[0223] The first acquisition module 502 is configured to acquire a sample question and a sample answer corresponding to the sample question;

[0224] The extraction module 504 is configured to extract at least one initial keyword combination of the sample question and determine the associated document set corresponding to each initial keyword combination;

[0225] The seed keyword combination determination module 506 is configured to determine a seed keyword combination according to the sample answer and each set of associated documents;

[0226] The sample keyword combination determination module 508 is configured to determine a sample keyword combination of the sample question according to the seed keyword combination;

[0227] The training module 510 is configured to train the keyword combination generation model according to the sample question and the sample keyword combination until a training stop condition is reached.

[0228] In one or more embodiments of the present embodiment, the extraction module 504 is further configured to:

[0229] Search for initial associated documents corresponding to each initial keyword combination;

[0230] Screen the initial associated documents corresponding to each initial keyword combination to generate a set of associated documents corresponding to each initial keyword combination.

[0231] In one or more embodiments of the present embodiment, the seed keyword combination determination module 506 is further configured to:

[0232] Determine a sample set of associated documents according to the sample answer and each set of associated documents;

[0233] Determine the initial keyword combination corresponding to the sample set of associated documents as the seed keyword combination.

[0234] In one or more embodiments of the present embodiment, the seed keyword combination determination module 506 is further configured to:

[0235] Match the sample answer with the associated documents in each set of associated documents;

[0236] Determine the associated document matching the sample answer as a sample associated document, and determine the set of associated documents corresponding to the sample associated document as a sample set of associated documents.

[0237] In one or more embodiments of the present embodiment, the sample keyword combination determination module 508 is further configured to:

[0238] Obtain a sample associated document matching the sample answer from the set of associated documents corresponding to each seed keyword combination;

[0239] Obtain the ranking of the sample associated document in the set of associated documents corresponding to each seed keyword combination;

[0240] determine a sample keyword combination of the sample problem in the seed keyword combinations according to the ranking of the sample related documents.

[0241] In one or more embodiments of the present embodiment, the sample keyword combination determination module 508 is further configured to:

[0242] determine the sample keyword combination of the sample problem as the seed keyword combination corresponding to the sample related document with the highest ranking; and / or

[0243] determine the sample keyword combination of the sample problem as the seed keyword combination corresponding to the sample related document with the highest ranking.

[0244] In one or more embodiments of the present embodiment, the sample keyword combination determination module 508 is further configured to:

[0245] determine the sample keyword combination of the sample problem as the seed keyword combination containing the most keywords in the seed keyword combinations.

[0246] In one or more embodiments of the present embodiment, the extraction module 504 is further configured to:

[0247] perform word segmentation processing on the sample problem to obtain at least one keyword of the sample problem;

[0248] determine at least one initial keyword combination according to the at least one keyword.

[0249] In one or more embodiments of the present embodiment, the extraction module 504 is further configured to:

[0250] perform word segmentation processing on the sample problem to obtain an initial keyword set;

[0251] filter stop words in the initial keyword set to obtain at least one keyword of the sample problem.

[0252] In one or more embodiments of the present embodiment, the training module 510 is further configured to:

[0253] input the sample problem into a keyword combination generation model;

[0254] the keyword combination generation model generates a predicted keyword combination in response to the sample problem as input;

[0255] calculate a loss value according to the predicted keyword combination and the sample keyword combination;

[0256] adjust parameters of the keyword combination generation model according to the loss value.

[0257] The keyword combination generation model training device provided by the embodiment of the present application can improve the speed of labeling the keyword combination of the sample question and the accuracy of the sample keyword combination, and effectively avoid the problem that the keyword combination in the database built in some vertical field does not match the prior knowledge of people. Further, the initial keyword combination is extracted from the sample question by word segmentation and stop word filtering, which saves the computing resources and improves the retrieval efficiency of subsequent retrieval according to the keyword combination; the sample associated document set is matched and determined in the management document set according to the sample answer, which ensures the matching degree of the sample answer and the sample associated document set, and solves the problem that the answer is not accurate in some vertical field in the prior art due to manual labeling of the keyword.

[0258] The above is a schematic scheme of the keyword combination generation model training device of the embodiment. It should be noted that the technical scheme of the keyword combination generation model training device and the technical scheme of the keyword combination generation model training method described above belong to the same concept, and the details of the technical scheme of the keyword combination generation model training device that are not described in detail can be referred to the description of the technical scheme of the keyword combination generation model training method.

[0259] Corresponding to the keyword combination generation method embodiment described above, the present application also provides a keyword combination generation device embodiment, Figure 6 The structure of the keyword combination generation device provided by an embodiment of the present application is shown. As shown in the figure, Figure 6 The device 600 includes:

[0260] The second acquisition module 602 is configured to acquire a to-be-processed question;

[0261] The input module 604 is configured to input the to-be-processed question to a pre-trained keyword combination generation model, wherein the keyword combination generation model is trained by the keyword combination generation model training method described above;

[0262] The generation module 606 is configured to generate a keyword combination by the keyword combination generation model in response to the to-be-processed question as input.

[0263] The keyword combination generation device provided in the application can ensure that the keyword combination of the problem to be processed can be quickly and accurately obtained, and ensure that the problem of inaccurate keyword combination caused by manual marking of the keyword combination in the problem to be processed can be avoided, and the problem that the keyword combination in the database built in some vertical field does not match the prior knowledge of people can be effectively avoided.

[0264] The above is a schematic scheme of the keyword combination generation device of the embodiment. It should be noted that the technical scheme of the keyword combination generation device and the technical scheme of the keyword combination generation method described above belong to the same concept, and the details of the technical scheme of the keyword combination generation device that are not described in detail can be referred to the description of the technical scheme of the keyword combination generation method.

[0265] It should be noted that each component in the device claim should be understood as a functional module that must be established to implement each step of the program flow or each step of the method. Each functional module is not an actual functional division or separation limitation. The device claim defined by such a set of functional modules should be understood as a functional module architecture for implementing the solution by the computer program mainly recorded in the specification, and should not be understood as an entity device for mainly implementing the solution by hardware.

[0266] In an embodiment of the application, a computing device is also provided, which includes a memory, a processor, and computer instructions stored on the memory and executable on the processor, and the processor executes the computer instructions to implement the steps of the keyword combination generation model training method or the keyword combination generation method.

[0267] The above is a schematic scheme of the computing device of the embodiment. It should be noted that the technical scheme of the computing device and the technical scheme of the keyword combination generation model training method or the keyword combination generation method described above belong to the same concept, and the details of the technical scheme of the computing device that are not described in detail can be referred to the description of the technical scheme of the keyword combination generation model training method or the keyword combination generation method.

[0268] In an embodiment of the application, a computer readable storage medium is also provided, which stores computer instructions, and the computer instructions are executed by a processor to implement the steps of the keyword combination generation model training method or the keyword combination generation method.

[0269] The above is a schematic scheme of the computer readable storage medium of the embodiment. It should be noted that the technical scheme of the storage medium and the technical scheme of the keyword combination generation model training method or the keyword combination generation method described above belong to the same concept, and the details of the technical scheme of the storage medium that are not described in detail can be seen from the description of the technical scheme of the keyword combination generation model training method or the keyword combination generation method.

[0270] The embodiment of the application discloses a chip which stores computer instructions. When the computer instructions are executed by a processor, the steps of the keyword combination generation model training method or the keyword combination generation method described above are implemented.

[0271] The above describes specific embodiments of the application. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims can be performed in a different order and still accomplish desirable results. Additionally, the processes depicted in the figures do not necessarily require the particular order shown, or sequential order to achieve desirable results. In certain implementations, multitasking and parallel processing can be advantageous.

[0272] The computer instructions include computer program codes which can be in the form of source code, object code, executable files, or some intermediate forms, etc. The computer readable medium can include any entity or device capable of carrying the computer program codes, recording medium, U disk, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signal, telecommunication signal, and software distribution medium, etc. It should be noted that the contents included in the computer readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction, for example, in some jurisdictions, according to legislation and patent practice, the computer readable medium does not include electrical carrier signals and telecommunication signals.

[0273] It should be noted that for each method embodiment described above, in order to facilitate description, each is expressed as a combination of a series of actions, but those skilled in the art should know that the application is not limited by the order of the actions described, because according to the application, some steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should know that the embodiments described in the specification all belong to preferred embodiments, and the actions and modules involved are not necessarily essential to the application.

[0274] In the above-described embodiments, the description of each embodiment focuses on different aspects, and the parts not described in detail in a certain embodiment can be referred to the relevant description of other embodiments.

[0275] The preferred embodiments of the application disclosed above are only used to illustrate the application. Alternative embodiments do not describe all the details and limit the application to the specific embodiments described. Obviously, according to the content of the application, many modifications and changes can be made. The application selects and describes these embodiments in order to better explain the principles and practical applications of the application, so that those skilled in the art can well understand and utilize the application. The application is limited by the claims and their full scope and equivalents.

Claims

1. A method for training a keyword combination generation model, characterized in that, include: Obtain the sample question and the corresponding sample answer; Extract at least one initial keyword combination from the sample question, and determine the associated document set corresponding to each initial keyword combination, wherein the initial keyword combination and the associated document set correspond one-to-one; Based on the sample answers and each associated document set, a sample associated document set is determined; the initial keyword combination corresponding to the sample associated document set is determined as the seed keyword combination; Determining the sample keyword combination for the sample question based on the seed keyword combination includes: obtaining sample related documents that match the sample answer from the sample related document set corresponding to each seed keyword combination, and the ranking of the sample related documents, wherein the ranking of the sample related documents is determined based on the relevance of the sample related documents to the corresponding initial keyword combination; and determining the sample keyword combination for the sample question in the seed keyword combination based on the ranking of the sample related documents. The keyword combination generation model is trained based on the sample question and the sample keyword combination until the training stops.

2. The method as described in claim 1, characterized in that, The process of determining the set of associated documents corresponding to each initial keyword combination includes: Search for the initial associated documents corresponding to each initial keyword combination; The initial associated documents corresponding to each initial keyword combination are filtered to generate a set of associated documents corresponding to each initial keyword combination.

3. The method as described in claim 1, characterized in that, The step of determining the sample associated document set based on the sample answer and each associated document set includes: The sample answers are matched with the associated documents in each associated document set; The associated documents that match the sample answer are identified as sample associated documents, and the set of associated documents corresponding to the sample associated documents is identified as the sample associated document set.

4. The method as described in claim 1, characterized in that, The step of determining the sample keyword combination for the sample question in the seed keyword combination based on the ranking of the associated documents includes: The seed keyword combination corresponding to the sample-related documents whose ranking falls within a preset ranking range is determined as the sample keyword combination for the sample question; and / or The seed keyword combination corresponding to the highest-ranking associated document of the sample is determined as the sample keyword combination for the sample question.

5. The method as described in claim 1, characterized in that, The step of determining the sample keyword combination for the sample question based on the seed keyword combination includes: The seed keyword combination containing the most keywords is determined as the sample keyword combination.

6. The method as described in claim 1, characterized in that, The extraction of at least one initial keyword combination for the sample question includes: The sample question is segmented into words to obtain at least one keyword for the sample question; Determine at least one initial keyword combination based on the at least one keyword.

7. The method as described in claim 6, characterized in that, The step of segmenting the sample question to obtain at least one keyword for the sample question includes: The sample question is segmented into words to obtain an initial set of keywords; Filter the stop words in the initial keyword set to obtain at least one keyword for the sample question.

8. The method as described in claim 1, characterized in that, The step of training a keyword combination generation model based on the sample question and the sample keyword combination includes: The sample question is input as a keyword combination to generate a model; The keyword combination generation model generates a predicted keyword combination in response to the sample question as input; Calculate the loss value based on the predicted keyword combination and the sample keyword combination; The parameters of the keyword combination generation model are adjusted based on the loss value.

9. A method for generating keyword combinations, characterized in that, include: Get the issues to be processed; The problem to be processed is input into a pre-trained keyword combination generation model, wherein the keyword combination generation model is trained by the keyword combination generation model training method described in any one of claims 1-8. The keyword combination generation model generates keyword combinations in response to the problem to be processed as input.

10. A keyword combination generation model training device, characterized in that, include: The first acquisition module is configured to acquire a sample question and the sample answer corresponding to the sample question; The extraction module is configured to extract at least one initial keyword combination of the sample question and determine the associated document set corresponding to each initial keyword combination, wherein the initial keyword combination and the associated document set correspond one-to-one; The seed keyword combination determination module is configured to determine a sample associated document set based on the sample answer and each associated document set; and to determine the initial keyword combination corresponding to the sample associated document set as the seed keyword combination. A sample keyword combination determination module is configured to determine the sample keyword combination of the sample question based on the seed keyword combination. The sample keyword combination determination module is further configured to: obtain sample related documents matching the sample answer from the sample related document set corresponding to each seed keyword combination, and the ranking of the sample related documents, wherein the ranking of the sample related documents is determined based on the relevance of the sample related documents to the corresponding initial keyword combination; and determine the sample keyword combination of the sample question from the seed keyword combination based on the ranking of the sample related documents. The training module is configured to train a keyword combination generation model based on the sample question and the sample keyword combination until the training stops.

11. A keyword combination generation device, characterized in that, include: The second acquisition module is configured to acquire issues to be processed. The input module is configured to input the problem to be processed into a pre-trained keyword combination generation model, wherein the keyword combination generation model is trained by the keyword combination generation model training method described in any one of claims 1-8. The generation module is configured to generate keyword combinations in response to the question to be processed as input by the keyword combination generation model.

12. A computing device, comprising a memory, a processor, and computer instructions stored in the memory and executable on the processor, characterized in that, When the processor executes the computer instructions, it implements the steps of the method according to any one of claims 1-8 or 9.

13. A computer-readable storage medium storing computer instructions, characterized in that, When executed by a processor, the computer instructions implement the steps of the method according to any one of claims 1-8 or 9.

Images