Data processing method and device, problem recommendation method and device and computing equipment

Abstract

The embodiment of the invention provides a data processing method and device, a problem recommendation method and device and computing equipment, and the data processing method comprises the steps that a sample query text is received, the sample query text comprises a query text input into a recommendation system, and the recommendation system is used for obtaining a target recommendation problem based on the query text; generating a sample recommendation problem based on the sample query text by using a problem generation model; retrieval related questions related to the sample recommendation questions are retrieved from a knowledge base; obtaining an annotation label aiming at the sample query text and / or the retrieval related problem; and constructing training data for training at least one model in the recommendation system based on at least one of the sample query text, the sample recommendation problem, the retrieval related problem and the labeling tag. According to the method, automatic construction of the training data is realized, the problem of insufficient training data is effectively solved, the diversity and accuracy of the training data are improved, and meanwhile, the construction efficiency of the training data is improved.

Description

Technical Field

[0001] The embodiments of this specification relate to the technical field of natural language processing, and more particularly to a data processing and question recommendation method, apparatus, and computing device. Background Art

[0002] With the development of natural language processing (NLP) technology, simply describing fuzzy query text is mapped to a series of accurate and relevant recommended questions, effectively improving the response quality and efficiency of customer service systems. For example, when a user simply enters the query "refund," the specific request may not be clear due to the overly simple description. The specific request may be "How long does it take for the refund to be received?" or "How do I cancel a refund request?"

[0003] At present, by training a special recommendation system to output sample recommendation questions corresponding to the query text, the intelligent completion function of the query text is realized, which can efficiently and accurately describe the true intention of the query text.

[0004] However, sufficient training data that is relevant to real-world scenarios is essential for efficient and accurate recommendation systems. In reality, training a recommendation system from scratch often faces the challenge of lacking sufficient high-quality training data. Therefore, a data processing method that can effectively construct training data is urgently needed. Summary of the Invention

[0005] In view of this, embodiments of this specification provide a data processing method. One or more embodiments of this specification also relate to a question recommendation method, a data processing apparatus, a question recommendation device, a computing device, a computer-readable storage medium, and a computer program product to address technical deficiencies in the prior art.

[0006] According to a first aspect of an embodiment of this specification, a data processing method is provided, including: receiving a sample query text, the sample query text including a query text input into a recommendation system, the recommendation system being used to obtain a target recommendation question based on the query text; Use the question generation model to generate sample recommendation questions based on sample query text; Retrieve relevant questions related to the sample recommendation problem from the knowledge base; Obtaining annotation labels for sample query text and / or retrieval-related questions; Based on at least one of the sample query text, the sample recommendation question, the retrieval-related question, and the annotated label, training data for training at least one model in the recommendation system is constructed.

[0007] According to a second aspect of an embodiment of this specification, a question recommendation method is provided, including: receiving a target query text; Using a recommendation system, obtaining target recommendation questions related to the target query text, wherein at least one model in the recommendation system is trained according to the above data processing method; Displays the target recommendation question.

[0008] According to a third aspect of the embodiments of this specification, there is provided a data processing device, including: A first receiving module is configured to receive a sample query text, the sample query text including a query text input into the recommendation system, the recommendation system is used to obtain a target recommendation question based on the query text; A first generation module is configured to generate a sample recommendation question based on the sample query text using a question generation model; A first retrieval module is configured to retrieve retrieval-related questions related to the sample recommendation problem from a knowledge base; A first annotation module is configured to obtain annotation tags for the sample query text and / or retrieval-related questions; The first building module is configured to build training data for training at least one model in the recommendation system based on at least one of a sample query text, a sample recommendation question, a retrieval-related question, and an annotated label.

[0009] According to a fourth aspect of the embodiments of this specification, a question recommendation device is provided, including: A second receiving module is configured to receive a target query text; A second recommendation module is configured to obtain target recommendation questions related to the target query text using a recommendation system, wherein at least one model in the recommendation system is trained according to the above-mentioned data processing method; The second display module is configured to display the target recommendation question.

[0010] According to a fifth aspect of the embodiments of this specification, a computing device is provided, including: memory and processor; The memory is used to store computer programs / instructions, and the processor is used to execute the computer programs / instructions. When the computer programs / instructions are executed by the processor, the steps of the above-mentioned data processing method or question recommendation method are implemented.

[0011] According to a sixth aspect of the embodiments of this specification, a computer-readable storage medium is provided, which stores a computer program / instruction, which implements the steps of the above-mentioned data processing method or question recommendation method when executed by a processor.

[0012] According to a seventh aspect of the embodiments of this specification, a computer program product is provided, including a computer program / instruction, which implements the steps of the above-mentioned data processing method or question recommendation method when executed by a processor.

[0013] In one embodiment of this specification, the semantic understanding and generation capabilities of the question generation model are utilized to expand the query text, mine sample recommendation questions, and improve the diversity of training data. Based on the sample recommendation questions, retrieval-related questions related to the sample recommendation questions are retrieved from the knowledge base to complete labeling, obtain labeled labels for the sample query text and / or retrieval-related questions, and improve the label accuracy of the training data. Based on at least one of the sample query text, sample recommendation questions, retrieval-related questions, and labeled labels, training data is constructed. The diverse and high-quality training data effectively solves the problem of insufficient training data for training at least one model in the recommendation system. This solution not only realizes the automated construction of training data, but also continuously interactively inputs and feedbacks to generate training data, thereby improving the efficiency of training data construction. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 is a flow chart of a data processing method provided by one embodiment of this specification; Figure 2 This is a flow chart of a data processing method provided by one embodiment of this specification; Figure 3 This is a flowchart of a question recommendation method provided by one embodiment of this specification; Figure 4 This is a flowchart of a processing process of a method for recommending questions applied to customer service consumer services provided by an embodiment of this specification; Figure 5 This is a front-end schematic diagram of a method for recommending questions to customer service consumers, provided by one embodiment of this specification; Figure 6 This is a schematic diagram of the structure of a data processing device provided by one embodiment of this specification; Figure 7 This is a structural diagram of a question recommendation device provided by an embodiment of this specification; Figure 8 This is a structural block diagram of a computing device provided by one embodiment of this specification. DETAILED DESCRIPTION

[0015] The following description sets forth many specific details to facilitate a thorough understanding of this specification. However, this specification can be implemented in many other ways than those described herein, and those skilled in the art can make similar generalizations without violating the scope of this specification. Therefore, this specification is not limited to the specific implementations disclosed below.

[0016] The terms used in one or more embodiments of the present invention are for the purpose of describing specific embodiments only and are not intended to limit one or more embodiments of the present invention. The singular forms "a", "the" and "the" used in one or more embodiments of the present invention are also intended to include plural forms, unless the context clearly indicates otherwise. It should also be understood that the term "and / or" used in one or more embodiments of the present invention refers to and includes any or all possible combinations of one or more associated listed items.

[0017] It should be understood that although the terms first, second, etc. may be used to describe various information in one or more embodiments of the present invention, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from each other. For example, without departing from the scope of one or more embodiments of the present invention, the first may also be referred to as the second, and similarly, the second may also be referred to as the first. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".

[0018] In addition, it should be noted that the data involved in one or more embodiments of the present invention are information and data authorized by the user or fully authorized by all parties, and the statistics, use and processing of relevant data need to comply with the relevant laws, regulations and standards of relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0019] First, the terms involved in one or more embodiments of this specification are explained.

[0020] Large Language Model (LLM): An AI model based on deep learning techniques for processing and generating natural language. LLMs are typically trained on massive amounts of text data and are capable of performing well on a variety of language tasks, such as speech generation, translation, summarization, question answering, and conversation.

[0021] Transformer model: A neural network architecture based on the self-attention mechanism that abandons the traditional recurrent neural network and convolutional neural network structures. It can be trained in parallel, significantly improving the effect of processing long sequence information. It is the core of modern natural language tasks.

[0022] Bidirectional Encoder Representations from Transformers (BERT): A pre-trained language model based on Transformer that uses bidirectional context to better understand the semantic information of text.

[0023] Embedding: Embedding is a term used in machine learning and natural language processing. It refers to mapping high-dimensional or complex data (such as text and images) into a fixed-dimensional, low-dimensional vector space. These low-dimensional vectors typically preserve the semantics or feature relationships of the data, making them easier to compute and process.

[0024] Query: A piece of text or sentence entered to obtain specific information, solve a problem, or perform an action. A query can be any natural language expression, such as a question, statement, or simple keyword.

[0025] Query to Query (Q2Q): A query-to-query mapping technology. Given a query text, Q2Q technology is used to quickly find relevant data mapped to the text from massive data.

[0026] Knowledge base: When performing Q2Q recall online, the recall query comes from a high-quality question-and-answer knowledge base, which contains high-quality answers to various questions. When a user enters a query, the online algorithm will match knowledge in this knowledge base to answer the user's question.

[0027] Supervised Fine-Tuning (SFT) is an important stage in the training of large language models. It uses supervised training data to fine-tune the pre-trained large model so that the generated results are more in line with task requirements or user expectations.

[0028] Vector Retrieval System: This system is primarily used to rapidly process nearest neighbor search (NNS) and clustering tasks for massive amounts of high-dimensional vector data. It is particularly well-suited for large-scale vector retrieval scenarios, such as recommendation systems, image search, and vector similarity calculation in natural language processing.

[0029] In this specification, a data processing method is provided. This specification also involves a question recommendation method, a data processing device, a question recommendation device, a computing device, a computer-readable storage medium and a computer program product, which are described in detail one by one in the following embodiments.

[0030] See also Figure 1 , Figure 1 A flowchart of a data processing method provided by an embodiment of this specification is shown, including the following specific steps: Step 102: Receive a sample query text, where the sample query text includes a query text input into a recommendation system, and the recommendation system is used to obtain a target recommendation question based on the query text.

[0031] The embodiments of this specification are applied to the server side of a website, application or system platform that completes the training data construction function through data processing, including but not limited to: content sharing platforms, online customer service systems, e-commerce platforms, video playback applications, social media applications, and educational service websites. The server and the client interact through a training front-end, which is a user interface or application programming interface (API) for collecting input, displaying generated recommendation questions and receiving feedback. The training front-end can be implemented on a client with training requirements. The training front-end acts as an interactive bridge between the client with the training recommendation system requirements and the data processing on the server.

[0032] Sample query text is the natural language text of the query recommendation questions used to train the recommender system, typically a paragraph or sentence. Sample query text includes the query text input into the recommender system. The query text input into the recommender system represents the intent of the question. Compared to subsequent sample recommendation questions and search-related questions, sample query text is simple and vague. Sample query text can be question-type query text or non-question-type query text, without limitation here.

[0033] A recommendation system is a machine learning model system that has the ability to recommend questions based on query text. The recommendation system is used to obtain target recommendation questions based on query text. The recommendation system can contain multiple sub-models, such as classification models, recall models, and ranking models. Each sub-model is responsible for handling different tasks in the recommendation process: triggering, recalling, and ranking.

[0034] For example, a content sharing platform needs to provide an after-sales module with automatic question completion. Within this module, a recommendation system needs to be deployed to retrieve relevant questions from the knowledge base to assist the question generation model in generating appropriate recommended questions. To train this recommendation system, developers need to build training data to complete the model training. The developer enters the sample query text "refund" into the training frontend.

[0035] Receiving sample query text including query text input into the recommendation system provides data support for subsequent retrieval and generation of sample recommendation questions, and lays the foundation for subsequent construction of training data.

[0036] Step 104: Generate sample recommendation questions based on the sample query text using the question generation model.

[0037] A question generation model is a natural language model with question generation capabilities. Trained through machine learning, particularly deep learning, the model can extract key information from text and generate questions. Examples of question generation models include, but are not limited to, the Transformer model, the BERT model, and large language models.

[0038] A sample recommendation question is a natural language text that expands upon the sample query text, typically a paragraph or sentence. Sample recommendation questions typically represent the direction and content of the user's intended question. Compared to the sample query text, sample recommendation questions are more precise and clear.

[0039] Using the question generation model, sample recommendation questions are generated based on the sample query text. An optional method is to use a pre-trained question generation model to generate sample recommendation questions based on the sample query text, wherein the question generation model is pre-trained for the label recommendation questions based on the pre-trained query text and the corresponding pre-trained query text.

[0040] Using the question generation model, sample recommendation questions are generated based on the sample query text. Another optional approach is to use a large language model and adopt a prompt word-guided generation paradigm to generate sample recommendation questions based on the sample query text.

[0041] For example, by using a large language model and a prompt word-guided generation paradigm, based on the sample query text "refund", three corresponding sample recommendation questions are generated: "How to apply for a refund", "How to complain if the merchant does not give a refund" and "How long does it take to get a refund".

[0042] By using the question generation model, sample recommendation questions are generated based on the sample query text, which expands the target query text, mines sample recommendation questions, improves the diversity of training data, and provides more accurate retrieval basis for subsequent retrieval of retrieval-related questions from the knowledge base.

[0043] Step 106: Retrieve retrieval-related questions related to the sample recommendation question from the knowledge base.

[0044] A knowledge base is a structured collection of information. It can record a large number of high-quality, relevant questions based on the relationships between query text and related questions. These relationships are typically expressed as key-value pairs or knowledge graphs. Knowledge bases can be pre-built based on Frequently Asked Questions (FAQs), product documentation, user manuals, historical case studies, and more.

[0045] Retrieval-related questions related to the sample recommendation problem are questions that are semantically related to the sample recommendation problem.

[0046] Retrieve retrieval-related questions related to the sample recommendation question from the knowledge base. An optional method is to use query mapping to retrieve retrieval-related questions related to the sample recommendation question from the knowledge base. Further optionally, based on the feature similarity of text features between the sample recommendation question and each related question in the knowledge base, retrieve retrieval-related questions related to the sample recommendation question from the knowledge base. For example, the term frequency-inverse document frequency index (TF-IDF) is used to encode text features and the cosine similarity is used to calculate feature similarity. For another example, embedded coding is used to encode text features and the cosine similarity is used to calculate feature similarity.

[0047] Retrieve retrieval-related questions related to the sample recommendation problem from the knowledge base. Another optional approach is to retrieve retrieval-related questions related to the sample recommendation problem from the knowledge base based on the knowledge graph of the pre-built knowledge base.

[0048] For example, a pre-built question-and-answer knowledge base for after-sales service, containing a large number of high-quality after-sales-related questions, was used. Based on the cosine similarity between the embedded text features of three sample recommended questions ("How do I apply for a refund?", "How do I complain if the merchant doesn't refund?", and "How long does it take to get a refund?") and the text features of the relevant questions in the knowledge base, three related questions were retrieved from the knowledge base: "How do I apply for a return and refund?", "How do I complain if the merchant doesn't refund?", and "What is the refund period?"

[0049] Retrieve relevant questions related to the sample recommendation problem from the knowledge base to provide a reference basis for subsequent feedback to the training front-end to complete the feasibility evaluation of the sample recommendation problem, so that the training front-end can judge whether the generated sample recommendation questions are accurate based on high-quality questions in actual scenarios.

[0050] Step 108: Obtain annotation tags for the sample query text and / or retrieval-related questions.

[0051] The annotation label is a label that indicates whether the sample recommendation question can be mapped to the semantic intent of the retrieval-related question, and is used to indicate whether the sample recommendation question accurately reflects the intention of the user's question and / or whether the retrieval-related question is effective in the application of actual business scenarios. The annotation label is usually completed by manual data quality annotation based on the annotation rules designed for the actual business scenario. The annotation label usually includes the classification results of the sample query text, the classification results of the positive and negative samples in the sample recommendation question and / or the retrieval-related question, the feasibility ranking of the label of the sample recommendation question and / or the retrieval-related question, etc. Optionally, compared to directly sending the sample recommendation question to the training front end for annotation, the retrieval-related question is sent to the training front end, and high-quality question expressions are provided to annotate the annotation labels of the relevant sample recommendation question. This can explore the feasibility of the sample recommendation question in the application of actual business scenarios while satisfying the correlation between the sample recommendation question and the retrieval-related question, so that the recommendation system trained subsequently not only satisfies the correlation between the query text and the recommendation question, but also satisfies the feasibility in the application of actual business scenarios.

[0052] Obtain annotation labels for sample query text and / or retrieval-related questions. One optional method is to only feed back retrieval-related questions to the training front end, and receive annotation labels for retrieval-related questions fed back by the training front end. Another optional method is to feed back sample query text and retrieval-related questions to the training front end, and receive annotation labels for sample query text and / or retrieval-related questions. Another optional method is to feed back sample recommendation questions and retrieval-related questions to the training front end, and receive annotation labels for retrieval-related questions. Another optional method is to feed back sample query text, sample recommendation questions, and retrieval-related questions to the training front end, and receive annotation labels for sample query text and / or retrieval-related questions. There is no limitation here.

[0053] For example, the sample query text "refund" and three search-related questions are fed back to the training front-end and displayed according to the preset labeling rules: "If you are a user, would you click on the recommended question? (1) The recommended question is related to the query text and can solve the problem (2) It can ease the user's emotions: Query text: refund; Recommended questions: "How to apply for a return and refund", "How to complain if the merchant does not refund" and "What is the refund period".

[0054] Obtaining annotation labels for sample query text and / or retrieval-related questions provides an annotation basis for subsequent label annotation.

[0055] Step 110: Construct training data for training at least one model in the recommendation system based on at least one of the sample query text, the sample recommendation question, the retrieval-related question, and the annotated label.

[0056] The training data is a dataset used to train the recommendation system, which includes training input data and labeled output data, so that the recommendation system can learn how to correctly map input to output.

[0057] For example, the developer receives the labeling of sample recommendation questions fed back on the training front end based on the target query text ("refund") and three retrieval-related questions ("How to apply for a return and refund", "How to complain if the merchant does not refund" and "What is the refund period"): "Refund" - the question should be recommended; "How to apply for a refund" - label: accept / reject, "How to complain if the merchant does not refund" - label: accept / reject, "How long does it take to get a refund" - label: accept / reject.

[0058] Based on the labeled labels of the above sample query texts and sample recommendation questions, training data is constructed for training a recommendation system with three sub-functions: triggering, recalling, and retaking.

[0059] In the embodiments of this specification, the semantic understanding and generation capabilities of the question generation model are utilized to expand the query text, mine sample recommendation questions, and improve the diversity of training data. Based on the sample recommendation questions, retrieval-related questions related to the sample recommendation questions are retrieved from the knowledge base to complete labeling, obtain labeled labels for the sample query text and / or retrieval-related questions, and improve the label accuracy of the training data. Based on at least one of the sample query text, sample recommendation questions, retrieval-related questions, and labeled labels, training data is constructed. The diversified and high-quality training data effectively solves the problem of insufficient training data for training at least one model in the recommendation system. This solution not only realizes the automated construction of training data, but also continuously interactively inputs and feedbacks to generate training data, thereby improving the efficiency of training data construction.

[0060] In an optional embodiment of the present specification, the training data includes training input data and label output data; after step 110, the following specific steps are also included: using sample query text as training input data and using the labeled labels of sample recommendation questions as label output data to train at least one model in the recommendation system.

[0061] Training input data is the data used as model input during the recommendation system training process. Because sample query texts are raw, unprocessed natural language expressions, they may be brief or ambiguous, but they represent the user's intent or request. This allows the model to learn to understand diverse inputs and identify recommendations that align with the intent.

[0062] Labeled output data is the training data used as labeled output during the training of the recommendation system. Labeled data reflects whether the recommendation question reasonably and accurately expresses the user's intent and is effective in real-world applications. It serves to inform the model of the correct outputs, allowing it to adjust its parameters and achieve more accurate recommendation results.

[0063] For example, collect n sample query texts Q={ , ,……, } and the corresponding m sample recommendation problem annotation label D={ , ,……, The recommendation system is trained iteratively over multiple rounds using n sample query texts as training input data and m sample recommendation question labels as label output data.

[0064] In the embodiments of this specification, sample query text is used as training input data, and the labeled labels of sample recommendation questions are used as label output data to perform supervised training on at least one model in the recommendation system, thereby improving the model's understanding ability and recommendation accuracy. The model can learn diverse user intent expressions and generate highly relevant recommendation questions, thereby optimizing the customer service experience.

[0065] In an optional embodiment of the present specification, the recommendation system includes a classification model, and the annotated label of the sample query text includes a classification result label for the sample query text; after step 110, the following specific steps are also included: based on the sample query text and the annotated label of the sample query text, first training data is obtained; based on the first training data, the classification model is trained, and the classification model is used to obtain a classification result for the query text, so as to determine whether to obtain a target recommendation question based on the query text based on the classification result.

[0066] A classification model is a machine learning model that determines whether a query triggers question recommendations. It uses text features to determine whether a query belongs to a specific category, such as a question or non-question. A classification model can be built using a natural language classification model, such as a 12-layer BERT model.

[0067] The first training data is a supervised learning dataset used to train the classification model. It includes sample query texts and the classification result labels for these sample query texts. The classification result labels indicate whether the sample query texts are question-type queries. The classification result labels are annotated by the training frontend based on labeling rules designed for actual business scenarios. They serve as part of the training data to help the classification model learn how to classify query texts.

[0068] An optional method for training a classification model based on the first training data is: using the classification model to determine a classification result prediction of the sample query text based on the text features of the sample query text; calculating a classification loss value based on the classification result prediction and the classification result label; and training the classification model based on the classification loss value.

[0069] Text features of sample query text are information units extracted from the sample query text and used for machine learning model analysis. They retain the semantics or feature relationships of the sample query text, facilitating calculation and processing. Text features can include word frequency, grammatical structure, contextual information, and semantic vectors.

[0070] The classification result prediction of the sample query text is the prediction classification result output by the classification model, which indicates whether the sample query text is a question type query text. The classification result prediction is usually expressed in the form of a probability distribution.

[0071] Classification loss is a metric used to measure the difference between the classification result prediction and the classification result label. It reflects the classification model's current performance in determining whether a query triggers question recommendations. A smaller loss indicates a closer approximation to the actual situation. Classification loss includes, but is not limited to, cross-entropy loss and log-likelihood loss.

[0072] Using the classification model, based on the text features of the sample query text, the classification result prediction of the sample query text is determined. An optional method is to use the classification model to encode the text features of the sample query text to obtain the text features of the sample query text, and based on the text features of the sample query text, the classification result prediction of the sample query text is determined, wherein the text feature encoding method includes but is not limited to: word frequency-inverse text frequency index and embedded coding.

[0073] For example, the classification model is a classification model based on a 12-layer BERT model. Using this classification model, for n sample query texts Q={ , ,……, } Perform text feature encoding to obtain the text features of the sample query text, as shown in Formula 1: Formula 1 in, Represents the hidden state of each token in the sample query text, and extracts the hidden state of a special token as the global text feature of the entire sample query text: For example, to get the text features of "I work at A": tokenize the word segmentation: [CLS] I work at A [SEP]. By adding the [CLS] token, we can get the global text features.

[0074] Based on the text features of the sample query text, the classification result prediction of the sample query text is determined. Specifically, the global text features are passed into the fully connected layer and activation layer of the BERT model to obtain the classification result prediction represented by the probability distribution, as shown in Formula 2: Formula 2 Where W and b are the classifier parameters of the classification model.

[0075] Based on the classification result prediction and classification result label, the classification loss value is calculated. The classification loss value adopts the cross entropy loss value. The calculation formula is shown in Formula 3: Formula 3 Among them, i represents an example of a sample query text, Indicates the classification result label (0 or 1), Represents the classification result prediction, log is the logarithmic function, optimize the loss function, and complete the supervised training of the classification model.

[0076] In the embodiment of this specification, the classification model is trained based on the first training data including the classification result label, which significantly improves the accuracy of the classification model in determining whether the query text is a question type, thereby enhancing the accuracy and efficiency of the entire recommendation system.

[0077] In an optional embodiment of the present specification, the recommendation system includes a recall model, and the annotated labels of the retrieval-related questions indicate the triggering behavior labels for the retrieval-related questions; after step 110, the following specific steps are also included: according to the annotated labels of the retrieval-related questions, positive samples and / or negative samples of the sample query text are determined from the retrieval-related questions; second training data is obtained based on the sample query text and the positive samples and / or negative samples of the sample query text; the recall model is trained based on the second training data, and the recall model is used to determine candidate recommendation questions related to the query text from the knowledge base.

[0078] The recall model is a machine learning model that retrieves relevant questions from a knowledge base. It performs this recall based on the feature similarity between the query text features and the text features of relevant questions in the knowledge base. The recall model can be constructed by combining a natural language encoding model with a vector retrieval system. For example, a BERT-based embedding generative model combined with an open-source vector retrieval system can be used.

[0079] The labeling of retrieval-related questions indicates whether the retrieval-related question should be triggered by the recommendation system. Labels are binary or multi-class labels that indicate whether the retrieval-related question should be triggered by the recommendation system, reflecting the effectiveness of the question in real-world business scenarios. Labels are generated through manual or automated evaluation and are used to distinguish positive examples (effective recommendations) from negative examples (ineffective recommendations). Evaluation criteria include semantic relevance, user click-through rate, or business rules.

[0080] The second training data is a supervised learning dataset used to train the recall model. It includes retrieval-related questions and annotated labels indicating triggering behavior labels for retrieval-related questions. Positive samples are sample recommendation questions that are annotated as reflecting the intent of the sample query text and are effective in practical applications. Negative samples are sample recommendation questions that are annotated as not reflecting the intent of the sample query text and / or are ineffective in practical applications.

[0081] An optional method for training a recall model based on the second training data is: using the recall model, based on the text features of the sample query text and the text features of the retrieval-related questions corresponding to the positive sample, determine the positive prediction feature similarity, and based on the text features of the sample query text and the text features of the retrieval-related questions corresponding to the negative sample, determine the negative prediction feature similarity; based on the positive prediction feature similarity and the negative prediction feature similarity, calculate the recall loss value; and based on the recall loss value, train the recall model.

[0082] The text features of the query-related questions corresponding to the positive samples are information units extracted from the query-related questions corresponding to the positive samples and used for machine learning model analysis. They retain the semantic or feature relationships of the query-related questions corresponding to the positive samples, facilitating calculation and processing. Text features can include word frequency, grammatical structure, contextual information, and semantic vectors.

[0083] The text features of the retrieval-related questions corresponding to the negative samples are information units extracted from the retrieval-related questions corresponding to the negative samples and used for machine learning model analysis. They retain the semantic or feature relationships of the retrieval-related questions corresponding to the negative samples, facilitating calculation and processing. Text features can include word frequency, grammatical structure, contextual information, and semantic vectors.

[0084] Recall loss is a metric used to measure the difference between the similarity of positive prediction features and the similarity of negative prediction features. Recall loss reflects the recall model's current performance in recalling relevant questions from the knowledge base. A higher recall loss indicates that the model's predictions are closer to the ground truth. Recall loss includes, but is not limited to, contrastive loss and Euclidean distance loss.

[0085] Using the recall model, the positive prediction feature similarity is determined based on the text features of the sample query text and the text features of the retrieval-related questions corresponding to the positive sample, and the negative prediction feature similarity is determined based on the text features of the sample query text and the text features of the retrieval-related questions corresponding to the negative sample. An optional method is to use the recall model to encode the text features of the sample query text, the retrieval-related questions corresponding to the positive sample, and the retrieval-related questions corresponding to the negative sample, obtain the text features of the sample query text, the text features of the retrieval-related questions corresponding to the positive sample, and the text features of the retrieval-related questions corresponding to the negative sample, determine the positive prediction feature similarity based on the text features of the sample query text and the text features of the retrieval-related questions corresponding to the positive sample, and determine the negative prediction feature similarity based on the text features of the sample query text and the text features of the retrieval-related questions corresponding to the negative sample, wherein the text feature encoding method includes but is not limited to: word frequency-inverse text frequency index and embedded coding.

[0086] For example, the recall model is a combination of a BERT-based embedding generation model and an open source vector retrieval system. Using this recall model, we query n sample texts Q = { , ,……, }、N sample recommendation questions corresponding to the retrieval related questions D={ , ,……, } Perform text feature encoding to obtain the text features of the sample query text , the text features of the retrieval-related questions corresponding to N sample recommendation questions .

[0087] Based on the text features of the sample query text and the text features of the retrieval-related questions corresponding to the positive sample, the positive prediction feature similarity is determined, as shown in Formula 4: Formula 4 in is the retrieval-related question corresponding to the positive sample. Represents the norm of a vector.

[0088] Based on the text features of the sample query text and the text features of the retrieval-related questions corresponding to the negative samples, the negative prediction feature similarity is determined, as shown in Formula 5: Formula 5 in is the retrieval-related question corresponding to the negative sample. Represents the norm of a vector.

[0089] The recall loss value is calculated based on the positive prediction feature similarity and the negative prediction feature similarity. The loss function uses contrast loss. Given a positive sample d+ and multiple negative samples D-, the optimization goal is to maximize the positive prediction feature similarity and minimize the negative prediction feature similarity, as shown in Formula 6: Formula 6 Optimize the loss function to complete the comparative training of the recall model.

[0090] In the embodiment of this specification, the recall model is trained based on the second training data of positive and negative samples, which significantly improves the accuracy of the recall model in recalling relevant questions from the knowledge base and enhances the accuracy and efficiency of the entire recommendation system.

[0091] In an optional embodiment of the present specification, the recommendation system includes a ranking model, and the annotated labels of the retrieval-related questions include triggering behavior labels for the retrieval-related questions; after step 110, the following specific steps are also included: obtaining third training data based on the sample query text and the annotated labels of the retrieval-related questions corresponding to the sample query text; training the ranking model based on the third training data, and the ranking model is used to obtain the triggering probability of the candidate recommended questions, so as to sort the candidate recommended questions based on the triggering probability to determine the target recommended questions.

[0092] The ranking model is a machine learning model that ranks the relevance of relevant questions retrieved from the knowledge base. Essentially a classification model, it determines the trigger probabilities of candidate recommended questions based on the feature similarity between the textual features of the sample query and the textual features of multiple retrieval-related questions. It then ranks the multiple retrieval-related questions based on the trigger probabilities of the candidate recommended questions. Its input consists of a single query and a pair of retrieval-related questions. It predicts a trigger probability, ranks the questions based on the trigger probability, and selects the top-ranked retrieval-related questions as recommended questions. The ranking model can be constructed using a natural language classification model, for example, a 12-layer BERT model.

[0093] The trigger behavior label for the retrieval-related question is a ranking basis label used in the ranking model to identify whether the retrieval-related question should be recommended first, usually a probability value or a grade score.

[0094] The third training data is a supervised learning dataset used to train the ranking model. It consists of sample query texts, retrieved related questions, and their triggering behavior labels. Labeling the data establishes a ranking relationship between the query text and candidate questions, and the model learns to rank the candidate questions based on their triggering probability. Labels can be binary (trigger / not trigger) or continuous (probability). The triggering probability of a candidate recommended question is the confidence score output by the ranking model, ranging from 0 to 1, indicating the relevance of the candidate recommended question to the query text. The probability value is normalized using the Softmax or Sigmoid function and is used to rank the candidate questions by relevance. The probability threshold can be dynamically adjusted to control the number of recommendations.

[0095] Based on the third training data, a ranking model is trained. An optional method is: using the ranking model, based on the feature similarity between the text features of the sample query text and the text features of multiple retrieval-related questions, the trigger probability of the candidate recommendation question is obtained; based on the trigger probability and the trigger behavior label, the ranking loss value is determined; based on the ranking loss value, the ranking model is trained.

[0096] Ranking loss is a metric used to measure the gap between prediction feasibility ranking and label feasibility ranking. It reflects the ranking model's current performance in ranking the relevance of relevant questions retrieved from the knowledge base. Lower loss indicates a closer approximation to the ground truth. Ranking loss includes, but is not limited to, cross-entropy loss and log-likelihood loss.

[0097] Exemplarily, the ranking model is a classification model based on a 12-layer BERT model. Using this ranking model, text features of the sample query text and multiple search-related questions are encoded to obtain the text features of the sample query text and the text features of the multiple search-related questions, as shown in Formula 1.

[0098] The feature similarity between the text features of the sample query text and the text features of multiple retrieval-related questions is based on the feature similarity between the text features of the sample query text and the text features of multiple retrieval-related questions. The specific method is shown in Formulas 4 and 5.

[0099] Based on the feature similarity between the text features of the sample query text and the text features of multiple retrieval-related questions, the trigger probability of the candidate recommendation question is obtained. Based on the trigger probability and the triggering behavior label, the ranking loss value is determined. The ranking loss value adopts the cross-entropy loss value, and the calculation formula is shown in Formula 3. This loss function is optimized to complete the supervised training of the ranking model.

[0100] In the embodiment of this specification, the ranking model is trained based on the third training data, which significantly improves the accuracy of the ranking model in ranking the relevance of relevant questions recalled from the knowledge base, thereby enhancing the accuracy and efficiency of the entire recommendation system.

[0101] In an optional embodiment of the present specification, after step 110, the following specific steps are also included: constructing a correlation between the sample query text and the sample recommendation question based on the annotated tags; and adding the sample query text and the sample recommendation question to the knowledge base based on the correlation.

[0102] The mapping is the mapping between sample query text and sample recommendation questions and retrieval-related questions in the knowledge base. This mapping can be achieved by adding new key-value or knowledge graph pairs to the existing knowledge base, or by replacing existing entries to update the knowledge base content.

[0103] For example, a Q&A knowledge base for after-sales service contains a series of frequently asked questions (FAQs) about "refunds." For example, given the query text "refund," sample recommended questions include: "How do I apply for a refund?", "How do I file a complaint if the merchant refuses to refund?", and "How long does it take to get a refund?"

[0104] After labeling, the three sample recommendation questions are all considered valid (that is, their labeling labels are all positive), and a key-value pair relationship is constructed to add the sample query text and sample recommendation questions to the question-answering knowledge base.

[0105] In the embodiments of this specification, based on the annotation tags, the correlation between the sample query text and the sample recommended questions is constructed, and based on the correlation relationship, the sample query text and the sample recommended questions are added to the knowledge base, which significantly enhances the content richness and timeliness of the knowledge base. It not only ensures that the knowledge base can reflect the latest query text and recommended questions in a timely manner, but also improves the accuracy and relevance of subsequent question recommendations.

[0106] In an optional embodiment of the present specification, the question generation model is a large language model; step 104 includes the following specific steps: constructing a target prompt word based on the sample query text and preset guide words; inputting the target prompt word into the large language model so that the large language model generates a sample recommendation question based on the sample query text and the guide words.

[0107] In the cue-guided generation paradigm, guide words are instructions or descriptions used to guide large language models to produce specific types of output. These typically include specific instructions for the task, the expected output format, and any other information that helps the model understand the context and user intent.

[0108] The target prompt is a complete instruction constructed by combining a sample query with a guide word. It is directly input into the large language model to generate the desired output. The target prompt contains the necessary information to generate the recommendation question from the query text, allowing the large language model to understand the specific task to be performed, the input data, and the expected output format.

[0109] For example, the guide words are: "You are an expert in generating recommendation questions, and you can generate appropriate recommendation questions based on the user's input. The answers are returned in a format that can be loaded by JSON [xx, xx, xx, ...]. If the query is not suitable for generating recommendation questions, please return an empty list []. The user's query is: refund, please generate:".

[0110] Based on the sample query text "refund" and the above guide words, a target prompt word Prompt is constructed. The target prompt word Prompt is input into the large language model. Based on the sample query text, three corresponding sample recommendation questions are generated according to the guide words: "How to apply for a refund", "How to complain if the merchant does not refund", and "How long does it take to get a refund?"

[0111] In the embodiments of this specification, by combining guide words and sample query texts, target prompt words are constructed to guide the large language model to generate sample recommendation questions, thereby strengthening the large language model's recognition of the intent of the sample query texts, improving the accuracy of the sample recommendation questions, and increasing the diversity of the sample recommendation questions.

[0112] Refer to the above Figure 1 Example, Figure 2 FIG. 1 shows a flow chart of a data processing method provided by an embodiment of the present specification, as shown in FIG. Figure 2 As shown: Offline recommendation problem mining line: Using a large language model, we generate recommendation questions based on the query text. Using query mapping, we map the recommendation questions to the knowledge base question retrieval. The query text and the retrieval-related questions are fed back as data pairs to the training frontend for manual annotation. Based on the query text and the annotated labels, we construct training data to train the recommendation system, a smaller model (compared to a large language model).

[0113] Online Line: Using the recommendation system, trigger classification, recall and retake are performed sequentially based on the query text to query the target recommendation system and complete the question recommendation.

[0114] Offline large language model fine-tuning circuit: The user front-end labels the target recommendation questions for question recommendation. Online user clicks are used to collect online data to build fine-tuning data and fine-tune the large language model, thereby improving the model's ability to generate recommendation questions.

[0115] By leveraging large language models to generate recommendation questions, combined with query mapping techniques and manual annotation, we generate diverse and high-quality online data, providing reliable support for the development and optimization of recommendation systems. Furthermore, processing real-time online data continuously improves the offline large language model's question generation capabilities.

[0116] It has the following advantages: 1. Improving the diversity of training data: Traditional methods for obtaining cold-start data are often relatively simple. The first version of cold-start data often relies on existing business models, resulting in a significant gap between recommendation problems and actual business scenarios. By using a large language model to generate cold-start data for recommendation problems, we can design guiding phrases to enable the model to understand business needs while generating more diverse data, significantly improving data richness and generalization capabilities.

[0117] 2. Optimizing training data quality: To ensure high quality and consistent training data diversity and suitability for business scenarios, we employ query mapping technology to map generated recommendation questions to the knowledge base, ensuring accurate alignment with the knowledge base's business knowledge system. We also employ manual annotation and evaluation to pair the mapped query-related questions with the query text. This manual annotation process confirms the data's feasibility, further improving data accuracy and business suitability.

[0118] 3. Continuously Improving Data Diversity and Quality: To enable the large language model to continuously evolve in this business scenario and generate more diverse recommendation questions that better meet business needs, we designed an iterative optimization process. Specifically, we used online data from the previous version to fine-tune the large language model to generate new recommendation questions, and then conducted feasibility assessments through manual annotation to provide high-quality samples for further training of the recommendation system. We also introduced online user click data to help the large language model gain a deeper understanding of user needs, continuously improving the relevance of the generated recommendation questions and the model's capabilities.

[0119] See also Figure 3 , Figure 3 A flowchart of a question recommendation method provided by an embodiment of this specification is shown, including the following specific steps: Step 302: Receive target query text.

[0120] Step 304: Utilize the recommendation system to obtain target recommendation questions related to the target query text, wherein at least one model in the recommendation system is trained according to the above data processing method.

[0121] Step 306: Display the target recommendation question.

[0122] The user frontend is a user interface or application programming interface (API) used to collect input, display generated recommendation questions, and receive feedback. The user frontend can be implemented on clients that require recommendation questions. It serves as an interactive bridge between the client that requires recommendation questions and the data processing on the server.

[0123] The target query is the natural language text used to query recommendations, typically a paragraph or sentence. The target query typically represents the user's intended question. Compared to subsequent target recommendations, the target query is more descriptive and vague. The target query can be either a question or non-question query, without limitation.

[0124] The target recommendation questions related to the target query text are related questions retrieved from the knowledge base and having relevance to the target query text.

[0125] For example, after completing the previous recommendation system training and question-answer knowledge base mapping, a real-time recommendation module combining the recommendation system and question-answer knowledge base is launched in the after-sales module with automatic question supplementation. The user enters the after-sales module and enters the target query text "My order status". The server receives the target query text entered by the user front-end and uses the recommendation system to retrieve five target recommended questions related to the target query text from the knowledge base: "How do I check the status of my order?", "What should I do if my order is delayed?", "Where can I find the order status page?", "Will status updates for order cancellations or modifications be displayed?", and "Who should I contact if I encounter order status issues?", and then feeds these five target recommended questions back to the user front-end.

[0126] In the embodiments of this specification, a trained recommendation system is used to respond to user queries in real time, which not only accurately captures the user's intentions and needs, but also significantly improves the accuracy and efficiency of question recommendations.

[0127] In an optional embodiment of the present specification, the following specific steps are also included: obtaining user feedback data on the target recommendation question obtained by the recommendation system based on the target query text, the user feedback data including user triggered behavior and / or user expected questions; based on the target query text, the target recommendation question and the user feedback data, constructing fine-tuning data for fine-tuning the question generation model.

[0128] User feedback data is the behavioral feedback given by users to the target recommendation questions generated by the recommendation system, which is used to evaluate the recommendation effect and optimize the model. It includes the user's actively selected desired questions or behavioral data such as clicks and stay time.

[0129] User-triggered behaviors are specific actions generated when users interact with recommendation questions, reflecting their interest or satisfaction with the recommendation results, including clicks, favorites, ignores, complaints, and other actions.

[0130] User-desired questions are recommended questions explicitly selected or entered by the user that match the target query text and represent their actual needs. User-desired questions can be obtained through active user selection (e.g., checking a box) or manual input, and provide high-quality annotated data for supervised learning.

[0131] The fine-tuning data is a dataset used to fine-tune the question generation model. It contains fine-tuning input data and labeled output data, enabling the question generation model to better adapt to the query text to recommendation question generation in actual business scenarios.

[0132] For example, two user expected questions, "How do I check the status of my order?" and "Where can I find the order status page?", are selected from five target recommendation questions ("How do I check the status of my order?", "What should I do if my order is delayed?", "Where can I find the order status page?", "Will status updates for order cancellations or modifications be displayed?", and "Who should I contact if I encounter order status issues?") received from the user front-end feedback. Fine-tuning data for fine-tuning the question generation model is constructed based on the target query text "my order status" and the two user expected questions.

[0133] In the embodiments of this specification, by receiving label recommendation questions from user front-end feedback and constructing fine-tuning data, continuous fine-tuning of the question generation model is achieved. This not only enhances the adaptability of the question generation model to actual business scenarios, but also allows for more efficient and accurate construction of training data in subsequent processes, ensuring continuous optimization of the recommendation system, thereby improving user experience and satisfaction.

[0134] In an optional embodiment of the present specification, the fine-tuning data includes fine-tuning input data and label output data; after constructing the fine-tuning data for fine-tuning the question generation model based on the target query text and label-related questions, the following specific steps are also included: fine-tuning the question generation model using the target query text as the fine-tuning input data and the label recommendation question as the label output data.

[0135] Fine-tuning input data is the data used as input during the fine-tuning problem generation process. Because the target query text is raw, unprocessed natural language expression, it may be brief or ambiguous, but it represents the user's intent or request. This allows the model to learn to understand diverse inputs and generate recommendation questions that align with the intent.

[0136] Label output data is the fine-tuning data used as label output during the fine-tuning problem generation model process. Label recommendation problems reflect the reasonable and accurate expression of user intent and effectively establish standards in practical applications to inform the model of what output is correct, allowing the model to adjust its parameters and achieve more accurate recommendation generation.

[0137] For example, collect n target query texts Q={ , ,……, } and the corresponding m tag recommendation problem D={ , ,……, The question generation model is fine-tuned using n target query texts as fine-tuning input data and m label recommendation questions as label output data. The specific training loss function is shown in Formula 7: Formula 7 Among them, Q is the target query text, D is the tag recommendation problem, and P is the probability of predicting the tag recommendation problem D given the target query text Q. are the model parameters to be trained.

[0138] In the embodiments of this specification, training data is generated by the offline module to train the model of the online module. At the same time, the real-time data generated by the online module is annotated and fed back to optimize the large language model in the offline module. This closed-loop collaborative mechanism not only continuously improves the quality and diversity of recommended questions, but also promotes the self-optimization of question recommendations. The feedback of real-time data enables the model to continuously learn the latest user behavior and demand patterns, ensuring the high relevance and timeliness of recommended questions, thereby significantly improving user experience and system performance, and promoting overall iteration and optimization.

[0139] In an optional embodiment of the present specification, the recommendation system includes a classification model, a recall model and a ranking model; using the recommendation system, target recommendation questions related to the target query text are obtained, including the following specific steps: using the classification model to determine the classification result of the target query text based on the text features of the target query text; when the classification result is a question-type query text, using the recall model to retrieve recall-related questions from the knowledge base based on the text features of the target query text and the text features of each related question in the knowledge base; using the ranking model to obtain the triggering probability of the recall-related questions, ranking the recall-related questions based on the triggering probability of the recall-related questions, and determining the target recommendation questions.

[0140] The classification result of the target query text is the classification result of whether the target query text is a question type query text. Classification result.

[0141] Recall-related questions are questions retrieved from the knowledge base that are semantically related to the target query. Recall-related questions are determined based on the similarity between the textual features of the target query and the textual features of each relevant question in the knowledge base. Recall-related questions are high-efficiency, low-precision related questions, requiring low-efficiency, high-precision re-ranking in subsequent sorting.

[0142] The ranking results for recall-related questions are typically an ordered list, sorted from highest to lowest probability of triggering a recall-related question. This ranking ensures that the first few questions recommended to the user are the most likely to resolve the user's question or provide useful information. The ranking sub-model deeply analyzes the text features of each recall-related question, assesses its match with the target query text, and outputs a ranking result based on this analysis.

[0143] Exemplarily, a classification model is used to determine the classification result of the target query text "my order status" based on the text features of the target query text. A recall model is used to retrieve 50 recall-related questions from the knowledge base based on the text features of the target query text and the text features of each related question in the knowledge base. A ranking model is used to sort the recall-related questions based on the feature similarity between the text features of the sample query text and the text features of the 50 recall-related questions, and the ranking results of the 50 recall-related questions are obtained. The top 5 questions in the ranking results of the recall-related questions are determined as target recommendation questions: "How do I check the status of my order?", "What should I do if my order is delayed?", "Where can I find the order status page?", "Will the status update of the order cancellation or modification be displayed?", "Who should I contact if I encounter an order status problem?"

[0144] In the embodiments of this specification, the classification model, recall model and ranking model of the recommendation system are used to sequentially complete the trigger classification, recall and re-ranking processing, which can not only quickly respond to user query needs, but also provide highly relevant and personalized recommendation questions.

[0145] The above is a schematic scheme of a question recommendation method of this embodiment. It should be noted that the technical scheme of this question recommendation method and the technical scheme of the above-mentioned data processing method are based on the same concept. For details not described in detail in the technical scheme of the question recommendation method, please refer to the description of the technical scheme of the above-mentioned data processing method.

[0146] The following combined Figure 4 , taking the application of the question recommendation method provided in this specification in the customer service consumer business as an example, the question recommendation method is further explained. Figure 4A flowchart of a method for recommending questions to a customer service customer service provider according to an embodiment of the present disclosure is shown, including the following specific steps: Step 402: Receive sample query text of customer service business input by the training front end.

[0147] Step 404: Based on the sample query text and the preset guide words, a target prompt word is constructed, the target prompt word is input into the large language model, and based on the sample query text, a sample recommendation question is generated according to the guide words.

[0148] Step 406: Utilize query mapping technology to retrieve retrieval-related questions related to the sample recommendation question from the customer service consumer business knowledge base.

[0149] Step 408: Feedback the retrieval-related questions to the training front end.

[0150] Step 410: Receive the labeling labels of sample recommendation questions in the customer service consumer business based on the feedback of the training front end based on the retrieval of related questions, and construct training data for training the recommendation system based on the sample query text and the labeling labels of the sample recommendation questions, wherein the recommendation system includes a classification model, a recall model and a ranking model.

[0151] Step 412: Based on the sample query text and the annotated labels of the sample query text, obtain first training data, and train a classification model based on the first training data.

[0152] Step 414: According to the annotated labels of the retrieval-related questions, determine the positive samples and / or negative samples of the sample query text from the retrieval-related questions, obtain second training data based on the sample query text and the positive samples and / or negative samples of the sample query text, and train the recall model based on the second training data.

[0153] Step 416: Obtain third training data based on the sample query text and the annotated labels of the retrieval-related questions corresponding to the sample query text, and train a ranking model based on the third training data.

[0154] Step 418: Based on the annotated tags, a correlation relationship is constructed between the sample query text and the sample recommended questions. Based on the correlation relationship, the sample query text and the sample recommended questions are added to the customer service consumer business knowledge base.

[0155] Step 420: Receive the target query text of the customer service consumer business input by the user front end.

[0156] Step 422: Utilize the recommendation system to obtain target recommendation questions for customer service related to the target query text.

[0157] Step 424: Feedback the target recommendation questions of the customer service consumer business to the user front end.

[0158] Step 426: Receive the user's desired question selected from multiple target recommendation questions fed back by the user front end, and construct fine-tuning data for fine-tuning the large language model based on the target query text and the user's desired question.

[0159] Step 428: Fine-tune the large language model using the target query text as fine-tuning input data and the user's desired question as label output data.

[0160] In the embodiments of this specification, by generating recommendation questions using a large language model and combining query mapping technology with manual annotation, diverse and high-quality online data is generated, providing reliable support for high-quality training data for the development and optimization of recommendation systems. Simultaneously, the processing of real-time online data continuously improves the offline large language model's question generation capabilities.

[0161] With the above Figure 4 The embodiment of the specification corresponds to, Figure 5 FIG. 1 shows a front-end schematic diagram of a method for recommending questions to customer service consumers provided by an embodiment of this specification, such as Figure 5 As shown: On the customer service conversation page of the customer service consumer business, users can send messages to enjoy customer service.

[0162] Figure 4 In step 420, the target query text input by the user front end is: "My order status".

[0163] Figure 4 In step 422, five target recommendation problems are determined.

[0164] Figure 4 In step 424, the AI ​​assistant sends a conversation message to feed back the five target recommendation questions to the user front end.

[0165] Guess you want to ask: You may want to know the following questions: 1. How do I check the status of my order? 2. What should I do if my order is delayed? 3. Where can I find the order status page? 4. Will status updates for order cancellations or modifications be displayed? 5. Who should I contact if I encounter issues with my order status? Tip: Please select the issue that best describes your needs, or let us know if there's anything else we can help you with.

[0166] The user can click 1-5 below and select the corresponding user expected question to send directly to further complete the conversation communication.

[0167] Corresponding to the above method embodiment, this specification also provides a data processing device embodiment, Figure 6 FIG1 shows a schematic diagram of the structure of a data processing device provided by an embodiment of this specification. Figure 6 As shown, the device includes: A first receiving module 602 is configured to receive a sample query text, where the sample query text includes a query text input into the recommendation system, and the recommendation system is used to obtain a target recommendation question based on the query text; A first generating module 604 is configured to generate a sample recommendation question based on the sample query text using a question generation model; A first retrieval module 606 is configured to retrieve retrieval-related questions related to the sample recommendation question from the knowledge base; A first annotation module 608 is configured to obtain annotation tags for the sample query text and / or retrieval-related questions; The first construction module 610 is configured to construct training data for training at least one model in the recommendation system based on at least one of the sample query text, the sample recommendation question, the retrieval-related question, and the annotated label.

[0168] Optionally, the recommendation system includes a classification model, and the annotated label of the sample query text includes a classification result label for the sample query text; the device also includes: a first training module, configured to obtain first training data based on the sample query text and the annotated label of the sample query text, and train the classification model based on the first training data, the classification model being used to obtain a classification result for the query text, so as to determine whether to obtain a target recommendation question based on the query text based on the classification result.

[0169] Optionally, the recommendation system includes a recall model, and the annotated labels of the retrieval-related questions indicate triggering behavior labels for the retrieval-related questions; the device also includes: a second training module, which determines positive samples and / or negative samples of the sample query text from the retrieval-related questions based on the annotated labels of the retrieval-related questions; obtains second training data based on the sample query text and the positive samples and / or negative samples of the sample query text; trains the recall model based on the second training data, and the recall model is used to determine candidate recommendation questions related to the query text from the knowledge base.

[0170] Optionally, the recommendation system includes a ranking model, and the annotation labels of the retrieval-related questions include triggering behavior labels for the retrieval-related questions; the device also includes: a third training module, configured to obtain third training data based on the sample query text and the annotation labels of the retrieval-related questions corresponding to the sample query text; based on the third training data, the ranking model is trained, and the ranking model is used to obtain the triggering probability of the candidate recommended questions, so as to sort the candidate recommended questions based on the triggering probability to determine the target recommended questions.

[0171] Optionally, the device further includes: a knowledge base adding module configured to construct a correlation between the sample query text and the sample recommendation question based on the annotated tags; and add the sample query text and the sample recommendation question to the knowledge base based on the correlation.

[0172] Optionally, the question generation model is a large language model; the first generation module 604 is further configured to: construct a target prompt word based on the sample query text and preset guide words; input the target prompt word into the large language model, so that the large language model generates a sample recommendation question based on the sample query text and the guide words.

[0173] In the embodiments of this specification, the semantic understanding and generation capabilities of the question generation model are utilized to expand the query text, mine sample recommendation questions, and improve the diversity of training data. Based on the sample recommendation questions, retrieval-related questions related to the sample recommendation questions are retrieved from the knowledge base to complete labeling, obtain labeled labels for the sample query text and / or retrieval-related questions, and improve the label accuracy of the training data. Based on at least one of the sample query text, sample recommendation questions, retrieval-related questions, and labeled labels, training data is constructed. The diversified and high-quality training data effectively solves the problem of insufficient training data for training at least one model in the recommendation system. This solution not only realizes the automated construction of training data, but also continuously interactively inputs and feedbacks to generate training data, thereby improving the efficiency of training data construction.

[0174] The above is a schematic diagram of a data processing device according to this embodiment. It should be noted that the technical solution of the data processing device and the technical solution of the above-mentioned data processing method are based on the same concept. For details not described in detail in the technical solution of the data processing device, please refer to the description of the technical solution of the above-mentioned data processing method.

[0175] Corresponding to the above method embodiment, this specification also provides a question recommendation device embodiment, Figure 7 FIG. 1 shows a schematic diagram of a question recommendation device provided by an embodiment of this specification. Figure 7 As shown, the device includes: The second receiving module 702 is configured to receive a target query text; The second recommendation module 704 is configured to obtain target recommendation questions related to the target query text using a recommendation system, wherein at least one model in the recommendation system is trained according to the above data processing method; The second display module 706 is configured to display the target recommendation question.

[0176] Optionally, the device also includes: a fine-tuning module, configured to obtain user feedback data of the target recommendation question obtained by the recommendation system based on the target query text, the user feedback data including user triggered behavior and / or user expected questions; based on the target query text, the target recommendation question and the user feedback data, construct fine-tuning data for fine-tuning the question generation model.

[0177] In the embodiments of this specification, a trained recommendation system is used to respond to user queries in real time, which not only accurately captures the user's intentions and needs, but also significantly improves the accuracy and efficiency of question recommendations.

[0178] The above is a schematic diagram of a question recommendation device according to this embodiment. It should be noted that the technical solution of the question recommendation device and the technical solution of the question recommendation method described above are based on the same concept. For details not described in detail in the technical solution of the question recommendation device, please refer to the description of the technical solution of the question recommendation method described above.

[0179] Figure 8 8. The structure of a computing device provided in one embodiment of the present specification is shown in FIG. Components of the computing device 800 include, but are not limited to, a memory 810 and a processor 820. The processor 820 is connected to the memory 810 via a bus 830, and a database 850 is used to store data.

[0180] Computing device 800 also includes an access device 840 that enables computing device 800 to communicate via one or more networks 860. Examples of such networks include a Public Switched Telephone Network (PSTN), a Local Area Network (LAN), a Wide Area Network (WAN), a Personal Area Network (PAN), or a combination of communication networks such as the Internet. Access device 840 may include one or more of any type of wired or wireless network interface (e.g., a Network Interface Controller (NIC)), such as an IEEE 802.11 Wireless Local Area Network (WLAN) wireless interface, a Worldwide Interoperability for Microwave Access (Wi-MAX) interface, an Ethernet interface, a Universal Serial Bus (USB) interface, a cellular network interface, a Bluetooth interface, or a Near Field Communication (NFC) interface.

[0181] In one embodiment of the present specification, the above components of the computing device 800 and Figure 8 Other components not shown in the figure may also be connected to each other, for example, via a bus. Figure 8 The computing device structure block diagram shown is for illustrative purposes only and is not intended to limit the scope of this specification. Those skilled in the art may add or replace other components as needed.

[0182] Computing device 800 can be any type of stationary or mobile computing device, including a mobile computer or mobile computing device (e.g., a tablet computer, personal digital assistant, laptop computer, notebook computer, netbook computer, etc.), a mobile phone (e.g., a smartphone), a wearable computing device (e.g., a smartwatch, smart glasses, etc.), or other types of mobile devices, or a stationary computing device such as a desktop computer or personal computer (PC). Computing device 800 can also be a mobile or stationary server.

[0183] The processor 820 is configured to execute the following computer program / instruction, which, when executed by the processor, implements the steps of the above-mentioned data processing method or question recommendation method.

[0184] The above is a schematic diagram of a computing device according to this embodiment. It should be noted that the technical solution of this computing device is based on the same concept as the technical solutions of the aforementioned data processing method and question recommendation method. For details not described in detail in the technical solution of the computing device, please refer to the description of the technical solutions of the aforementioned data processing method or question recommendation method.

[0185] An embodiment of the present specification further provides a computer-readable storage medium storing a computer program / instruction, which implements the steps of the above-mentioned data processing method or question recommendation method when executed by a processor.

[0186] The above is a schematic diagram of a computer-readable storage medium according to this embodiment. It should be noted that the technical solution of this storage medium is based on the same concept as the technical solutions of the aforementioned data processing method and question recommendation method. For details not described in detail in the technical solution of the storage medium, please refer to the description of the technical solutions of the aforementioned data processing method or question recommendation method.

[0187] An embodiment of the present specification further provides a computer program product, including a computer program / instruction, which implements the steps of the above-mentioned data processing method or question recommendation method when executed by a processor.

[0188] The above is a schematic diagram of a computer program product according to this embodiment. It should be noted that the technical solution of this computer program product is based on the same concept as the technical solutions of the aforementioned data processing method and question recommendation method. For details not described in detail in the technical solution of the computer program product, please refer to the description of the technical solutions of the aforementioned data processing method or question recommendation method.

[0189] The foregoing description of this specification describes specific embodiments. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims can be performed in an order different from that described in the embodiments and still achieve the desired results. Furthermore, the processes depicted in the accompanying drawings do not necessarily require the specific order shown or the sequential order to achieve the desired results. In certain embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0190] The computer instructions include computer program code, which may be in source code form, object code form, executable file, or some intermediate form. The computer-readable medium may include any entity or device capable of carrying the computer program code, recording medium, USB flash drive, mobile hard drive, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signals, telecommunication signals, and software distribution media. It should be noted that the content of the computer-readable medium may be appropriately increased or decreased based on the requirements of patent practice. For example, in some regions, according to patent practice, computer-readable media does not include electric carrier signals and telecommunication signals.

[0191] It should be noted that for the aforementioned method embodiments, for the sake of simplicity of description, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the embodiments of this specification are not limited by the order of the actions described, because according to the embodiments of this specification, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in this specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the embodiments of this specification.

[0192] In the above embodiments, the description of each embodiment has its own focus. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0193] The preferred embodiments disclosed above are intended only to help illustrate this specification. The optional embodiments do not exhaustively describe all details, nor do they limit the invention to the specific embodiments described. Obviously, many modifications and variations can be made based on the content of the embodiments of this specification. This specification selects and specifically describes these embodiments in order to better explain the principles and practical applications of the embodiments of this specification, so that those skilled in the art can better understand and utilize this specification. This specification is limited only by the claims and their full scope and equivalents.

Claims

1. A data processing method, characterized in that: include: receiving a sample query text, the sample query text comprising a query text input into a recommendation system, the recommendation system being configured to obtain a target recommendation question based on the query text; Use the question generation model to generate sample recommendation questions based on sample query text; Retrieving retrieval-related questions related to the sample recommendation problem from a knowledge base; Obtaining annotation tags for the sample query text and / or the search-related questions; Based on at least one of the sample query text, the sample recommendation question, the retrieval-related question, and the annotated label, training data for training at least one model in the recommendation system is constructed.

2. The method according to claim 1, characterized in that The recommendation system includes a classification model, and the annotation label of the sample query text includes a classification result label of the sample query text; After constructing training data for training at least one model in the recommendation system based on at least one of the sample query text, the sample recommendation question, the retrieval-related question, and the annotated label, the method further includes: Obtaining first training data based on the sample query text and the annotated label of the sample query text; The classification model is trained based on the first training data, and the classification model is used to obtain a classification result for the query text, so as to determine whether to obtain a target recommendation question based on the query text based on the classification result.

3. The method according to claim 1, characterized in that The recommendation system includes a recall model, wherein the annotated labels of the retrieval-related questions indicate triggering behavior labels for the retrieval-related questions; After constructing training data for training at least one model in the recommendation system based on at least one of the sample query text, the sample recommendation question, the retrieval-related question, and the annotated label, the method further includes: Determining positive samples and / or negative samples of the sample query text from the retrieval-related questions according to the annotated labels of the retrieval-related questions; Obtaining second training data based on the sample query text and positive samples and / or negative samples of the sample query text; The recall model is trained based on the second training data, and the recall model is used to determine candidate recommendation questions related to the query text from the knowledge base.

4. The method according to claim 1, wherein The recommendation system includes a ranking model, and the annotation tags of the retrieval-related questions include triggering behavior tags for the retrieval-related questions; After constructing training data for training at least one model in the recommendation system based on at least one of the sample query text, the sample recommendation question, the retrieval-related question, and the annotated label, the method further includes: Obtaining third training data based on the sample query text and the annotated label of the retrieval-related question corresponding to the sample query text; The ranking model is trained based on the third training data, and the ranking model is used to obtain trigger probabilities of candidate recommendation questions, so as to rank the candidate recommendation questions based on the trigger probabilities to determine target recommendation questions.

5. The method according to any one of claims 1 to 4, characterized in that After constructing training data for training at least one model in the recommendation system based on at least one of the sample query text, the sample recommendation question, the retrieval-related question, and the annotated label, the method further includes: Based on the annotated tags, constructing a correlation relationship between the sample query text and the sample recommendation question; Based on the correlation, the sample query text and the sample recommended question are added to the knowledge base.

6. The method according to claim 1, characterized in that The question generation model is a language model; The method of using the question generation model to generate sample recommendation questions based on the sample query text includes: Constructing target prompt words based on the sample query text and preset guide words; The target prompt word is input into a large language model so that the large language model generates a sample recommendation question according to the guide word based on the sample query text.

7. A question recommendation method, characterized in that: include: receiving a target query text; Utilizing a recommendation system, obtaining target recommendation questions related to the target query text, wherein at least one model in the recommendation system is trained using the data processing method according to any one of claims 1 to 6; The target recommendation question is displayed.

8. The method according to claim 7, characterized in that Also includes: Obtaining user feedback data on target recommendation questions obtained by the recommendation system based on the target query text, wherein the user feedback data includes user triggering behavior and / or user expectation questions; Based on the target query text, the target recommendation question and the user feedback data, fine-tuning data for fine-tuning the question generation model is constructed.

9. A data processing device, characterized in that: include: A first receiving module is configured to receive a sample query text, wherein the sample query text includes a query text input into a recommendation system, and the recommendation system is used to obtain a target recommendation question based on the query text; A first generation module is configured to generate a sample recommendation question based on the sample query text using a question generation model; A first retrieval module is configured to retrieve retrieval-related questions related to the sample recommendation question from a knowledge base; A first annotation module is configured to obtain annotation tags for the sample query text and / or the retrieval-related question; The first building module is configured to build training data for training at least one model in the recommendation system based on at least one of the sample query text, the sample recommendation question, the retrieval-related question and the annotated label.

10. A question recommendation device, characterized in that: include: A second receiving module is configured to receive a target query text; a second recommendation module, configured to obtain target recommendation questions related to the target query text using a recommendation system, wherein at least one model in the recommendation system is trained using the data processing method according to any one of claims 1 to 6; The second display module is configured to display the target recommendation question.

11. A computing device, characterized in that include: memory and processor; The memory is used to store computer programs / instructions, and the processor is used to execute the computer programs / instructions. When the computer program / instructions are executed by the processor, the steps of the method according to any one of claims 1 to 8 are implemented.

12. A computer-readable storage medium, characterized in that It stores a computer program / instruction, which implements the steps of the method according to any one of claims 1 to 8 when executed by a processor.

13. A computer program product, characterized in that The method comprises a computer program / instruction which, when executed by a processor, implements the steps of the method according to any one of claims 1 to 8.

Images