Query text ranking method and device, computer device, and storage medium

Abstract

The application discloses a query text ranking method, which comprises the following steps: obtaining target dialogue text, performing recall processing on the target dialogue text to obtain candidate dialogue text; performing vector processing on the target dialogue text through a preset text model to obtain a target text vector, and obtaining a candidate text vector from a preset vector library; inputting the candidate text vector and the target text vector into the preset text model to obtain a candidate semantic vector and a target semantic vector; performing prediction processing on the candidate semantic vector and the target semantic vector to obtain a prediction value, and performing ranking processing on the candidate dialogue text according to the prediction value to obtain a text ranking result. The application obtains candidate dialogue text through a recall mode, so that the content of the candidate dialogue text is more comprehensive, the candidate text vector is obtained from the preset vector library, and therefore, the application realizes fast ranking of the candidate dialogue text, improves the speed of intelligent customer service reply, and further improves the experience of users.

Description

Technical Field

[0001] This invention relates to the field of text processing technology, and in particular to a method, apparatus, computer device, and storage medium for sorting query text. Background Technology

[0002] With the rapid development of the internet, artificial intelligence has also developed rapidly and is widely applied in various fields. Especially in intelligent customer service, it typically involves recognizing the intent of user questions and automatically displaying relevant business scripts to assist human customer service representatives or directly answer user questions.

[0003] Existing technologies often involve first recalling candidate dialogues, converting them into semantic vectors, and then sorting the semantic vectors using a semantic model. This sorting process is computationally complex and cannot quickly sort candidate dialogues, resulting in intelligent customer service failing to display relevant business dialogues in a timely manner, which seriously affects the user experience. Summary of the Invention

[0004] This invention provides a method, apparatus, computer device, and storage medium for sorting query text, in order to solve the problem of slow query text sorting speed and inability to respond in a timely manner in the prior art.

[0005] A method for sorting query text, comprising:

[0006] Obtain the current dialogue text, and perform completion processing on the current dialogue text to obtain the target dialogue text;

[0007] The target dialogue text is retrieved to obtain candidate dialogue texts;

[0008] The target dialogue text is processed by a preset text model to obtain a target text vector, and the candidate text vector corresponding to the candidate dialogue text is obtained from a preset vector library.

[0009] The candidate text vector and the target text vector are input into the preset text model to obtain the candidate semantic vector corresponding to the candidate text vector and the target semantic vector corresponding to the target text vector;

[0010] The candidate semantic vector and the target semantic vector are subjected to prediction processing to obtain prediction values, and the candidate dialogue text is sorted according to the prediction values ​​to obtain the text sorting result.

[0011] A query text sorting device, comprising:

[0012] The completion processing module is used to obtain the current dialogue text, complete the current dialogue text, and obtain the target dialogue text.

[0013] The text recall module is used to recall the target dialogue text to obtain candidate dialogue text;

[0014] The vector conversion module is used to perform vector processing on the target dialogue text through a preset text model to obtain the target text vector, and to obtain the candidate text vector corresponding to the candidate dialogue text from a preset vector library;

[0015] The semantic vector module is used to input the candidate text vector and the target text vector into the preset text model to obtain the candidate semantic vector corresponding to the candidate text vector and the target semantic vector corresponding to the target text vector;

[0016] The text sorting module is used to perform prediction processing on the candidate semantic vector and the target semantic vector to obtain prediction values, and sort the candidate dialogue text according to the prediction values ​​to obtain the text sorting result.

[0017] A computer device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the above-described query text sorting method.

[0018] A computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described query text sorting method.

[0019] This invention provides a method, apparatus, computer device, and storage medium for sorting query text. The method ensures the accuracy of candidate dialogue text retrieval by recalling target dialogue text, thereby improving the precision of user intent recognition. It improves the speed of subsequent candidate dialogue text sorting by directly obtaining candidate text vectors from a database. Semantic extraction of candidate and target text vectors is performed using a preset text model, thus obtaining target and candidate semantic vectors. Probabilistic prediction of candidate semantic vectors is performed based on the target semantic vector to obtain predicted values, and the candidate dialogue texts are sorted according to the predicted values, thereby obtaining the text sorting results, improving the speed of intelligent customer service responses, and thus enhancing the user experience. Attached Figure Description

[0020] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the description of the embodiments of the present invention will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0021] Figure 1 This is a schematic diagram of the application environment of the query text sorting method in one embodiment of the present invention;

[0022] Figure 2 This is a flowchart of a text sorting method according to an embodiment of the present invention;

[0023] Figure 3 This is a flowchart of step S10 in a text sorting method according to an embodiment of the present invention;

[0024] Figure 4 This is a flowchart of step S30 in a text sorting method according to an embodiment of the present invention;

[0025] Figure 5 This is a schematic diagram of a text sorting device according to an embodiment of the present invention;

[0026] Figure 6 This is a schematic diagram of a computer device according to an embodiment of the present invention. Detailed Implementation

[0027] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0028] The query text sorting method provided in this embodiment of the invention can be applied to, for example... Figure 1 In the application environment shown. Specifically, this query text sorting method is applied in a query text sorting device, which includes, for example, [details omitted]. Figure 1 The client and server shown communicate over a network to address the problem of slow text sorting and delayed responses in existing technologies. The server can be a standalone server or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms. The client, also known as the user terminal, refers to the program that provides categorization services to customers, corresponding to the server. The client can be installed on, but is not limited to, various personal computers, laptops, smartphones, tablets, and portable wearable devices.

[0029] In one embodiment, such as Figure 2As shown, a method for sorting query text is provided, which can be applied to... Figure 1 Taking the client as an example, the following steps are included:

[0030] S10: Obtain the target dialogue text, perform recall processing on the target dialogue text, and obtain candidate dialogue text.

[0031] Understandably, the target dialogue text is derived from the conversation between the user and the intelligent or human customer service representative. The candidate dialogue text is a question text similar to the target dialogue text.

[0032] Specifically, the target dialogue text is retrieved from the server and then processed for recall. This involves first correcting the target dialogue text by detecting errors in the characters. If errors are found, the target dialogue text is corrected to obtain the corrected dialogue text. Next, the corrected dialogue text is segmented into words, and the weights of these segmentation results are calculated to determine the user's query intent. Finally, semantic similarity matching is performed on the user's query intent, or similar text data is obtained through a defined thesaurus, thus making the subsequent recall more comprehensive. The process of recalling the user's query intent and similar corpus data involves text matching to retrieve all texts corresponding to the user's query intent and similar corpus data, and then identifying all retrieved texts as candidate dialogue texts.

[0033] S20: Perform vector processing on the target dialogue text using a preset text model to obtain the target text vector, and retrieve the candidate text vector corresponding to the candidate dialogue text from a preset vector library.

[0034] Understandably, the target text vector is the vector corresponding to the target dialogue text. The preset text model is a pre-set model used to convert text into vectors, such as the BERT model. The candidate text vector is the vector corresponding to the candidate dialogue text.

[0035] Specifically, after obtaining all candidate dialogue texts, a preset text model is retrieved from the server. The target dialogue text is input into the preset text model, which performs word segmentation on the target dialogue text, breaking it down into multiple words. First, a CLS flag is added before each of the segmented words as an identifier representing the overall semantics of the target dialogue text. Then, a SEP flag is added after each of the segmented words as a separator, forming an input vector. Next, each word, CLS flag, and SEP flag in the input vector is vectorized, i.e., each word, CLS flag, and SEP flag is encoded, representing each word, CLS flag, and SEP flag as a number, and then represented by a vector to obtain the representation vector corresponding to each word, CLS flag, and SEP flag, thus obtaining the target text vector corresponding to the target dialogue text. Further, candidate text vectors corresponding to the candidate dialogue texts are retrieved from a preset vector library using a lookup table or matching method.

[0036] S30: Input the candidate text vector and the target text vector into the preset text model to obtain the candidate semantic vector corresponding to the candidate text vector and the target semantic vector corresponding to the target text vector.

[0037] Understandably, the candidate semantic vector is a vector representing the semantics of the candidate dialogue text. For example, if Xiaoming ate an apple, the semantics of that sentence would be represented by a vector. The target semantic vector is a vector representing the semantics of the target dialogue text. The preset text model includes a preset number (e.g., 12 or 24) of transformer layers.

[0038] Specifically, after obtaining the candidate text vectors, the candidate and target text vectors are input into a pre-defined text model. The transformer layer within this model performs semantic extraction on the candidate and target text vectors. This involves multiple attention processes, projecting the Q, K, and V vectors through various linear transformations, and finally concatenating the different attention results to obtain a combined vector. The combined vector is then normalized to prevent degradation after multiple attention processes. A pre-feedback neural network in a fully connected layer performs a non-linear transformation on the normalized combined vector, effectively activating it. This activated combined vector is then linearly transformed to map to a high-dimensional space. After residual processing, and repeating this process a predetermined number of times, the candidate and target text vectors are transformed into candidate and target semantic vectors.

[0039] S40, perform prediction processing on the candidate semantic vector and the target semantic vector to obtain prediction values, and sort the candidate dialogue text according to the prediction values ​​to obtain text sorting results.

[0040] Understandably, the text ranking result is the sorting of all candidate dialogue texts according to their predicted values ​​from largest to smallest. The predicted values ​​are obtained by using the softmax layer in the pre-defined text model to predict all candidate semantic vectors and the target semantic vector.

[0041] Specifically, after obtaining the target semantic vector and candidate semantic vectors, all candidate semantic vectors and the target semantic vector are input into the softmax layer of a preset text model. The softmax layer maps all candidate semantic vectors and the target semantic vector to values ​​between [0, 1]. That is, the softmax layer performs probability prediction on all candidate semantic vectors based on the target semantic vector, obtaining the predicted value corresponding to each candidate semantic vector. Then, all predicted values ​​are sorted in descending order, and all candidate semantic vectors are sorted according to the order of the predicted values, thus obtaining the text ranking result. For example, with 10 candidate semantic vectors, the softmax layer performs prediction processing on the candidate semantic vectors, obtaining a set of data [0.8, 0.6, 0.5, 0.3, 0.7, 0.9, 0.8, 0.4, 0.7, 0.5]. The candidate dialogue text is then sorted based on this set of data to obtain the text ranking result.

[0042] This invention provides a text ranking method that, by recalling target dialogue text, ensures the accuracy of candidate dialogue text retrieval, thereby improving the precision of user intent recognition. By directly obtaining candidate text vectors from a database, the subsequent ranking speed of candidate dialogue texts is improved. Semantic extraction is performed on candidate and target text vectors using a preset text model, thus obtaining target and candidate semantic vectors. Probabilistic prediction is performed on candidate semantic vectors based on the target semantic vector to obtain predicted values, and the candidate dialogue texts are ranked according to these predicted values, thereby obtaining the text ranking results, improving the speed of intelligent customer service responses, and ultimately enhancing the user experience.

[0043] In one embodiment, such as Figure 3 As shown, step S10, which involves obtaining the target dialogue text, includes:

[0044] S101, Obtain the initial dialogue text, perform word segmentation on the initial dialogue text, and obtain at least one word to be processed in the initial dialogue text.

[0045] Understandably, the words to be processed are the result of segmenting the initial dialogue. A directed acyclic graph (DAG) is a graph without loops and with direction. The initial dialogue text is the text of the conversation between the user and customer service.

[0046] Specifically, the initial dialogue text is retrieved from the server, and Chinese word segmentation algorithm is used to segment the initial dialogue text. Based on the contextual features, the initial dialogue text is segmented using a full segmentation path selection method, which means listing all possible segmentation results and selecting the best segmentation path. All segmentation results are then combined into a directed acyclic graph. By using the segmentation results as nodes and assigning weights to the edges between words, the path with the minimum weight is found as the final result. For example, word frequency can be used as the weight, and the path with the highest total word frequency can be considered the best path. This process yields at least one word to be processed corresponding to the initial dialogue text.

[0047] S102, perform entity recognition on the words to be processed to obtain entity recognition results corresponding to the words to be processed.

[0048] Understandably, entity recognition models can be obtained through supervised training of models, such as those built on neural networks, using labeled text. Part-of-speech tagging is the process of assigning part-of-speech labels to words based on a part-of-speech encoding table. Entity recognition is the process of extracting entity information from dialogue text. The entity recognition result is the entity information extracted from the dialogue text.

[0049] Specifically, after obtaining the words to be processed, part-of-speech tagging is performed on all the words using a part-of-speech encoding table. Each word or phrase is labeled with a part-of-speech tag, such as adjective, verb, noun, etc. This allows the words to be processed to incorporate more useful information in subsequent processing. The dialogue text to be processed after part-of-speech tagging is input into the entity recognition model. The entity recognition model performs entity recognition on the dialogue text to be processed. For example, the entity type of each word to be processed is determined based on its part of speech, and then the entity type is determined as the entity recognition result. That is, based on the context features and the relationship between the part of speech of sentences and words, important entity information is extracted from the given dialogue text, such as time, place, people, etc. Time can be a time entity, place can be a place entity, and people can be a name entity, etc.

[0050] S103, the initial dialogue text is filtered according to the entity recognition results corresponding to each word to be processed to obtain the current dialogue text.

[0051] Specifically, after obtaining the entity recognition results corresponding to each word to be processed, the noise words such as stop words and interjections, words with low frequency of occurrence, and sentences with incoherent sentences in all entity recognition results are filtered out by a pre-set dictionary. All the entity recognition results obtained after filtering are then organized into dialogue text, and the current dialogue text can be obtained.

[0052] S104, perform completion processing on the current dialogue text to obtain the target dialogue text.

[0053] Specifically, after obtaining the current dialogue text, all current dialogue texts are concatenated in text order to obtain the dialogue text to be completed. Keywords are extracted from the dialogue text to be completed and recorded as referential entities. The positions of referential words in the dialogue text to be completed are determined, and the referential words in the current dialogue text are replaced, that is, the referential words are replaced with their corresponding referential entities based on context information. In this way, by replacing all referential words in the dialogue text to be completed, the target dialogue text is obtained.

[0054] This invention, through word segmentation of the initial conversation text, achieves the acquisition of words to be processed within the initial dialogue text. Part-of-speech tagging is performed on the words to be processed using a part-of-speech encoding table, and entity recognition is performed on the words to be processed using an entity recognition model, thereby acquiring the entity recognition results. The initial dialogue text is filtered based on the entity recognition results corresponding to each word to be processed, thus acquiring the current dialogue text. By completing the current dialogue text, the target dialogue text is acquired.

[0055] In one embodiment, step S104, namely, performing completion processing on the current dialogue text to obtain the target dialogue text, includes:

[0056] S1041, Obtain the historical dialogue text and the historical reply text corresponding to the historical dialogue text; the historical dialogue text refers to the text of the previous round of dialogue of the current dialogue text; the historical dialogue text and the historical reply text both correspond to the same historical text tag.

[0057] S1042, the historical dialogue text, the historical reply text, and the current dialogue text are concatenated to obtain the initial text.

[0058] In essence, historical dialogue text refers to the text of the previous round of dialogue, i.e., the text of the previous round of the current dialogue. Historical reply text refers to the text corresponding to customer service replies in the historical dialogue. Historical text tags are tags used to distinguish historical dialogue text from the current dialogue text, such as token type 0. Both historical dialogue text and historical reply text correspond to the same historical text tag.

[0059] Specifically, after obtaining the current dialogue text, the historical dialogue text and its corresponding historical reply text are retrieved from the server. Historical text tags are then set for the historical dialogue text and historical reply text. For example, setting `tokentype` to 0 means that the text tags for both historical dialogue text and historical reply text are 0, indicating that they all correspond to the same historical text tag. All historical dialogue text, historical reply text, and the current dialogue text are then concatenated in chronological order, separated by delimiters (such as spaces or / ), thus obtaining the initial text.

[0060] S1043, obtain the current text tag corresponding to the current dialogue text, and concatenate the historical text tag and the current text tag to obtain the initial tag.

[0061] Understandably, the current text tag is a tag used to distinguish the historical text from the current text, such as token type 1. The initial tag is the tag obtained by concatenating the historical text tag and the current text tag, such as initial tags 0, 0, 1.

[0062] Specifically, after obtaining the initial text, the current text tag corresponding to the current dialogue text is retrieved from the server through table lookup or matching. For example, the token type can be set to 1 for the current dialogue text tag. After the current dialogue text tag is set, the historical text tags and the current text tags are concatenated in the order of historical dialogue text, historical reply text, and current dialogue text in the initial text to obtain the initial tag.

[0063] S1044, Input the initial text and the initial tag into the preset text query model, and obtain the reference entity position corresponding to the initial text and the position to be completed corresponding to the current dialogue text output by the preset text query model.

[0064] Understandably, the preset text query model is a pre-set model used to find the locations of referential entities and the locations to be completed in the initial text. The location of the referential entity is the position of the entity referred to by the pronoun in the initial text. The location to be completed is the position in the initial text that needs to be completed or the position of the pronoun in the initial text.

[0065] Specifically, after obtaining the initial labels, the initial text and initial labels are input together into a preset text query model. The preset text query model first encodes the initial text, then maps it to a high-dimensional space through a three-layer transformation module. Then, it queries the referential entity positions and the positions to be completed in the current dialogue text through a fully connected layer and a normalization layer, obtaining the referential entity positions in the initial text and the positions to be completed in the current dialogue text. It then calculates the probabilities of the positions to be completed and their corresponding referential entity positions, associating the position with the highest probability with the referential entity position. This process is repeated for all positions to be completed in the current dialogue text and their corresponding referential entity positions. A referential entity position can be associated with at least one position to be completed, and a position to be completed can only be associated with one referential entity position.

[0066] S1045, extract the reference entity text corresponding to the reference entity position from the initial text, and perform text completion on the current dialogue text according to the reference entity text and the position to be completed to obtain the target dialogue text.

[0067] Understandably, the entity reference text refers to the text that refers to the entity's location. The target dialogue text is the current dialogue text after the location to be completed has been filled in.

[0068] Specifically, after obtaining the referential entity location and the location to be completed, the referential entity text corresponding to the referential entity location is extracted from the initial text; that is, the referential entity text is extracted from the referential entity location. First, the start and end positions of the referential entity text are determined to obtain its length. Then, the start and end positions of the location to be completed are determined to obtain its length. Next, based on the obtained association information, all locations to be completed associated with the referential entity location are retrieved, and text completion is performed on the current dialogue text, that is, the referential entity text is filled or replaced into the locations to be completed. After filling or replacing all locations to be completed in sequence, the target dialogue text is obtained.

[0069] This invention, through simultaneous input of initial text and initial tags into a preset text query model, obtains the referential entity location and the location to be completed, thereby acquiring the referential entity location in the initial text and the location to be completed in the current dialogue text. By extracting the referential entity text from the referential entity location and supplementing it to the location to be completed according to the referential completion location, the target dialogue text is obtained. This further avoids the problem of intelligent customer service being unable to respond to users due to missing subjects.

[0070] In one embodiment, step S10, namely, recalling the target dialogue text to obtain candidate dialogue text, includes:

[0071] S105, perform text detection on the target dialogue text to obtain text detection results; the text detection results include detection failure results; the detection failure results indicate that there are errors in the target dialogue text.

[0072] Understandably, text detection results are used to characterize whether errors exist in the target dialogue text.

[0073] Specifically, after obtaining the target dialogue text, text detection is performed on it. This involves detecting whether there are misidentified words or sentences in the target dialogue text. Specifically, the target dialogue text is first segmented, and the words with the highest similarity are matched from the bag-of-words database. The matched words and segmented words are then replaced, and the dialogue text before and after replacement is scored to obtain the text detection result. The text detection result includes successful detection results and failed detection results. Successful detection results indicate that all content of the target audio text is correct, while failed detection results indicate that the content of the target dialogue text contains errors.

[0074] S106, perform correction processing on the target dialogue text corresponding to the detection failure result to obtain the corrected dialogue text.

[0075] S107, Obtain a preset intent recognition model, and perform intent recognition on the correction dialogue text through the preset intent recognition model to obtain the text intent recognition result.

[0076] Understandably, corrected dialogue text is obtained by correcting erroneous words and phrases. Text intent recognition results are used to represent the user's intent, such as the user wanting to purchase a certain item. The preset intent recognition model is pre-set and trained using a large amount of sample data, continuously iterating the model parameters to continuously improve the model's recognition accuracy.

[0077] Specifically, after obtaining the text detection results, the failed detection results are filtered out, and the target dialogue text corresponding to the failed detection results is determined. The target dialogue text corresponding to the failed detection results is then corrected, that is, erroneous words or sentences are replaced. Based on contextual information, the target dialogue text corresponding to the failed detection results is predicted to obtain at least one predicted replacement word or sentence. All predicted replacement words or sentences are then replaced with the positions corresponding to the erroneous words or sentences in the target dialogue text corresponding to the failed detection results, thus obtaining the corrected dialogue text. Further, an intent recognition model is retrieved from the server, and the corrected dialogue text is input into the intent recognition model. The intent recognition model then performs intent recognition on the corrected text to obtain the text intent recognition result. For example, if a user asks how much a blue helmet costs, the preset intent recognition model identifies the user's intent, thus determining that the user is asking about the price of a blue helmet.

[0078] S108, Obtain a set of candidate dialogue texts, wherein the set of candidate dialogue texts includes at least one candidate dialogue text.

[0079] S109, perform semantic similarity matching between the candidate dialogue text and the text intent recognition result to obtain semantic similarity matching result.

[0080] S1010, the candidate dialogue text whose semantic similarity matching result is greater than or equal to the preset similarity threshold is determined as the candidate dialogue text.

[0081] Understandably, the candidate dialogue text set is the collection of texts containing all response statements. The candidate dialogue text is the text of the response statements. The semantic similarity matching result is used to characterize the semantic similarity between the candidate dialogue text and the text intent recognition result.

[0082] Specifically, after obtaining the text intent recognition result, a preset set of candidate dialogue texts is retrieved from the server or a third-party platform. This set includes at least one candidate dialogue text. The intent recognition result is then semantically similar to all candidate dialogue texts; that is, the semantic similarity between the text intent recognition result and all candidate dialogue texts is calculated, such as by calculating the Euclidean distance or cosine similarity between them. Candidate dialogue texts whose semantic similarity matching results are greater than or equal to a preset similarity threshold are identified as candidate dialogue texts.

[0083] This invention, through text detection of the target dialogue text, determines whether it is intended for correction, thus ensuring the accuracy of user intent recognition. A preset intent recognition model is used to identify the intent in the correction dialogue text, thereby quickly obtaining the text intent recognition result. By performing semantic similarity matching between the text intent recognition result and candidate dialogue texts, candidate dialogue texts are determined, ensuring the accuracy of the candidate dialogue texts.

[0084] In one embodiment, before step S20, that is, before vector processing of the target dialogue text using a preset text model, the following steps are included:

[0085] S201, Obtain a sample training dataset, wherein the sample training dataset includes at least one text training data; one text training data corresponds to one sample label.

[0086] Understandably, text training data can be collected from different websites or databases using web scraping techniques, or it can be sent by the user from the client to the server. A sample training dataset is then constructed based on all the acquired text training data. Each piece of text training data is assigned a sample label, which is a numerical value used to represent the text training data. In the context of intelligent customer service Q&A, the text training data represents each round of dialogue between the user and the customer service representative. The sample label is a sample value representing the text of each round of dialogue, and this sample value is directly set based on the text of each round of dialogue.

[0087] S202, Obtain a preset training model, and use the preset training model to perform vector transformation on the text training data to obtain sample text vectors.

[0088] S203, input the sample text vector into the preset training model, obtain the sample semantic vector corresponding to the sample text vector, and predict the sample semantic vector to obtain the predicted label.

[0089] Understandably, the sample text vector is obtained by vectorizing the text training data using a pre-set training model. The sample semantic vector is obtained by semantic extraction from the sample text vector using a pre-set training model. The predicted label is a numerical value used to represent the semantic vector of the sample. The pre-set training model is a pre-configured BERT model. This BERT model includes an encoding layer for vector transformation, a transformer layer for semantic extraction, and a prediction layer (softmax layer) for probability prediction.

[0090] Specifically, the pre-trained model is extracted from the server, and all text training data is input into it. First, the encoding layer performs word segmentation on all text training data. Then, a CLS flag is added before each word as an identifier representing the overall semantics of the target dialogue text, and a SEP flag is added after each word as a separator, forming an input vector. Each word, CLS flag, and SEP flag in this input vector is vectorized to obtain representation vectors corresponding to each word, CLS flag, and SEP flag, thus obtaining sample text vectors corresponding to the text training data. Further, the transformer layer of the pre-trained model performs semantic extraction on the sample text vectors, i.e., attention processing, residual processing, and linear mapping processing are performed sequentially on the sample text vectors to obtain sample semantic vectors corresponding to all sample text vectors. Finally, the prediction layer performs probability prediction on all sample semantic vectors to obtain the predicted labels corresponding to each sample semantic vector.

[0091] S204, determine the prediction loss value of the text training data based on the sample label and the prediction label corresponding to the same text training data.

[0092] Understandably, the prediction loss is generated during the prediction of the predictive features of the text training data, and it is used to characterize the difference between the sample features and the predicted features.

[0093] Specifically, after obtaining the predicted labels, all sample labels corresponding to the text training data are arranged according to the order of the text training data in the sample training dataset. Then, the predicted labels associated with the text training data are compared with the sample labels of the text training data with the same sequence. That is, according to the text training data, the sample label corresponding to the first text training data is compared with the predicted label corresponding to the first text training data, and the loss value between the sample label and the predicted label is determined by the loss function. Then, the sample label corresponding to the second text training data is compared with the predicted label corresponding to the second text training data, until all sample labels and predicted labels have been compared, and the prediction loss value of the text training data can be determined.

[0094] S205, when the predicted loss value does not reach the preset convergence condition, iteratively update the initial parameters in the preset training model until the predicted loss value reaches the convergence condition, and record the converged preset training model as a preset text model.

[0095] Understandably, the convergence condition can be the condition that the predicted loss value is less than a set threshold, that is, when the predicted loss value is less than the set threshold, training stops; the convergence condition can also be the condition that the predicted loss value is very small after 500 calculations and will not decrease further, that is, when the predicted loss value is very small after 500 calculations and will not decrease further, training stops, and the converged preset training model is recorded as a preset text model.

[0096] Specifically, after determining the predicted loss value of the text training data, if the predicted loss value does not reach the preset convergence condition, the initial parameters of the preset training model are adjusted according to the predicted loss value, and a portion of the text training data is re-input into the preset training model after adjusting the initial parameters, so that the predicted loss value of the text training data reaches the preset convergence condition through gradient descent. Then, a portion of the text training data in the sample training dataset is selected, and the above steps S201 to S204 are executed to obtain the predicted loss value corresponding to the preset training model with adjusted initial parameters. If the predicted loss value does not reach the preset convergence condition, the initial parameters of the preset training model are adjusted again according to the predicted loss value, so that the predicted loss value of the preset training model with adjusted initial parameters reaches the preset convergence condition.

[0097] Thus, after training the preset training model with all text training data, the output of the preset training model can continuously approach the accurate result, making the prediction accuracy higher and higher, until the prediction loss value of all text training data reaches the preset convergence condition, and the converged preset training model is recorded as the preset text model.

[0098] This invention acquires text training data and sample labels, and trains a preset training model by selecting a portion of the text training data from the sample training dataset each time, thereby achieving the training of the preset training model using the gradient descent method. Iterative training of the preset training model using gradient descent ensures that the prediction loss value reaches the convergence condition, thus achieving the training of the preset text model and improving the prediction accuracy of the preset text model.

[0099] In one embodiment, such as Figure 4 As shown, step S30, which involves obtaining the candidate semantic vector corresponding to the candidate text vector, includes:

[0100] S301, The candidate text vector is processed by the attention layer in the preset text model to obtain the attention vector.

[0101] Understandably, the attention vector is obtained by applying attention to the candidate text vector through the attention layer in the transformer layer.

[0102] Specifically, after obtaining the candidate text vectors, multiple sets of three weight matrices W in the attention layer are used. Q W K W V The candidate text vectors are processed to obtain their corresponding Query vector, Keys vector, and Values ​​vector. The relevance score between candidate text vectors is calculated using the dot product method, i.e., the dot product is calculated between each candidate text vector in Q and each candidate text vector in K. The relevance scores between candidate text vectors are then normalized using the softmax function, transforming the scores into a probability distribution between [0, 1]. Based on this probability distribution, the corresponding Values ​​value is multiplied to obtain the attention vector. In this way, the attention vectors corresponding to all candidate text vectors are obtained.

[0103] S302, input all the attention vectors into the residual layer in the preset text model, and obtain the residual vectors output by the residual layer.

[0104] S303, the residual vector is linearly mapped through the fully connected layer in the preset text model to obtain the candidate semantic vector.

[0105] Understandably, the residual vector is obtained by concatenating and normalizing all attention vectors.

[0106] Specifically, after obtaining the attention vectors, multiple attention vectors are concatenated, and the concatenated attention vectors are normalized through a first residual connection layer to obtain residual vectors, thus avoiding attention vector degradation during the residual processing. Then, the normalized attention vectors are processed through the feedforward network and prediction layer in the fully connected layer; that is, the ReLU function is used to activate the normalized attention vectors, and a linear mapping is used to map the activated attention vectors to a high-dimensional space. After residual processing and repeating this process for a predetermined number of transformer layers, candidate semantic vectors are obtained.

[0107] This invention achieves the acquisition of attention vectors through multiple sets of attention processing. By performing residual processing on all attention vectors in a single layer, the residual vectors are determined. Through linear mapping, the residual vectors are mapped to a higher-order space, thereby enabling the extraction of candidate semantic vectors.

[0108] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0109] In one embodiment, a query text sorting device is provided, which corresponds one-to-one with the query text sorting method in the above embodiments. For example... Figure 5 As shown, the query text sorting device includes a text retrieval module 11, a vector transformation module 13, a semantic vector module 14, and a text sorting module 15. Detailed descriptions of each functional module are as follows:

[0110] Text recall module 11 is used to obtain target dialogue text, perform recall processing on the target dialogue text, and obtain candidate dialogue text;

[0111] The vector conversion module 12 is used to perform vector processing on the target dialogue text through a preset text model to obtain the target text vector, and to obtain the candidate text vector corresponding to the candidate dialogue text from a preset vector library.

[0112] The semantic vector module 13 is used to input the candidate text vector and the target text vector into the preset text model to obtain the candidate semantic vector corresponding to the candidate text vector and the target semantic vector corresponding to the target text vector;

[0113] The text sorting module 14 is used to perform prediction processing on the candidate semantic vector and the target semantic vector to obtain prediction values, and to sort the candidate dialogue text according to the prediction values ​​to obtain the text sorting result.

[0114] In one embodiment, the text recall module 11 includes:

[0115] The word segmentation unit is used to acquire the initial dialogue text, perform word segmentation on the initial dialogue text, and obtain at least one word to be processed in the initial dialogue text.

[0116] An entity recognition unit is used to perform entity recognition on the word to be processed and obtain an entity recognition result corresponding to the word to be processed.

[0117] The filtering unit is used to filter the initial dialogue text based on the entity recognition results corresponding to each word to be processed, so as to obtain the current dialogue text.

[0118] The text completion unit is used to complete the current dialogue text to obtain the target dialogue text.

[0119] In one embodiment, the text completion unit includes:

[0120] The acquisition unit is used to acquire historical dialogue text and historical reply text corresponding to the historical dialogue text; the historical dialogue text refers to the text of the previous round of dialogue of the current dialogue text; the historical dialogue text and the historical reply text both correspond to the same historical text tag;

[0121] The splicing unit is used to splice the historical dialogue text, the historical reply text, and the current dialogue text to obtain the initial text;

[0122] The tag unit is used to obtain the current text tag corresponding to the current dialogue text, and to concatenate the historical text tag and the current text tag to obtain the initial tag;

[0123] The query unit is used to input the initial text and the initial tag into a preset text query model, and obtain the reference entity position corresponding to the initial text and the position to be completed corresponding to the current dialogue text output by the preset text query model.

[0124] The completion unit is used to extract the referential entity text corresponding to the referential entity position from the initial text, and to complete the current dialogue text according to the referential entity text and the position to be completed, so as to obtain the target dialogue text.

[0125] In one embodiment, the text recall module 11 includes:

[0126] A text detection unit is used to perform text detection on the target dialogue text and obtain a text detection result; the text detection result includes a detection failure result; the detection failure result indicates that there is an error in the target dialogue text.

[0127] The text correction unit is used to correct the target dialogue text corresponding to the detection failure result to obtain the corrected dialogue text.

[0128] An intent recognition unit is used to acquire a preset intent recognition model, and to perform intent recognition on the correction dialogue text through the preset intent recognition model to obtain a text intent recognition result.

[0129] The candidate dialogue text unit is used to obtain a candidate dialogue text set, wherein the candidate dialogue text set includes at least one candidate dialogue text.

[0130] A semantic matching unit is used to perform semantic similarity matching between the candidate dialogue text and the text intent recognition result to obtain a semantic similarity matching result;

[0131] The candidate dialogue text unit is used to identify candidate dialogue texts whose semantic similarity matching results are greater than or equal to a preset similarity threshold.

[0132] In one embodiment, the vector transformation module 12 includes:

[0133] A sample acquisition unit is used to acquire a sample training dataset, wherein the sample training dataset includes at least one text training data; one text training data corresponds to one sample label.

[0134] A text vector unit is used to acquire a preset training model, and to perform vector transformation on the text training data through the preset training model to obtain sample text vectors.

[0135] The semantic vector prediction unit is used to input the sample text vector into the preset training model, obtain the sample semantic vector corresponding to the sample text vector, and predict the sample semantic vector to obtain the predicted label.

[0136] The prediction loss value unit is used to determine the prediction loss value of the text training data based on the sample label and the prediction label corresponding to the same text training data.

[0137] The model convergence unit is used to iteratively update the initial parameters in the preset training model when the predicted loss value does not reach the preset convergence condition, until the predicted loss value reaches the convergence condition, and then record the converged preset training model as a preset text model.

[0138] In one embodiment, the semantic vector module 13 includes:

[0139] An attention unit is used to perform attention processing on the candidate text vector through the attention layer in the preset text model to obtain an attention vector;

[0140] The normalization unit is used to input all the attention vectors into the residual layer in the preset text model and obtain the residual vectors output by the residual layer.

[0141] The linear mapping unit is used to perform linear mapping processing on the residual vector through the fully connected layer in the preset text model to obtain the candidate semantic vector.

[0142] Specific limitations regarding the query text sorting device can be found in the limitations of the query text sorting method above, and will not be repeated here. Each module in the aforementioned query text sorting device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device in hardware form, or stored in the memory of a computer device in software form, so that the processor can call and execute the operations corresponding to each module.

[0143] In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 6 As shown, the computer device includes a processor, memory, network interface, and database connected via a system bus. The processor provides computing and control capabilities. The memory includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores the operating system, computer programs, and the database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The database stores the data used in the query text sorting method described in the above embodiment. The network interface communicates with external terminals via a network connection. When the computer program is executed by the processor, it implements a query text sorting method.

[0144] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the query text sorting method described in the above embodiment.

[0145] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the query text sorting method described in the above embodiment.

[0146] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0147] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is used as an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.

[0148] The above-described embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included within the protection scope of the present invention.

Claims

1. A method for sorting query text, characterized in that, include: Obtain the target dialogue text, and perform recall processing on the target dialogue text to obtain candidate dialogue text; The target dialogue text is processed by a preset text model to obtain a target text vector, and a candidate text vector corresponding to the candidate dialogue text is obtained from a preset vector library. The candidate text vector and the target text vector are input into the preset text model to obtain the candidate semantic vector corresponding to the candidate text vector and the target semantic vector corresponding to the target text vector; The candidate semantic vector and the target semantic vector are subjected to prediction processing to obtain prediction values, and the candidate dialogue texts are sorted according to the prediction values ​​to obtain text sorting results; The step of obtaining the candidate semantic vector corresponding to the candidate text vector includes: The candidate text vectors are processed by the attention layer in the preset text model to obtain the attention vectors; All attention vectors are input into the residual layer of the preset text model to obtain the residual vectors output by the residual layer; The residual vector is linearly mapped through the fully connected layer in the preset text model to obtain the candidate semantic vector.

2. The query text sorting method as described in claim 1, characterized in that, The acquisition of the target dialogue text includes: Obtain the initial dialogue text, perform word segmentation on the initial dialogue text, and obtain at least one word to be processed in the initial dialogue text; Entity recognition is performed on the words to be processed to obtain entity recognition results corresponding to the words to be processed; Based on the entity recognition results corresponding to each word to be processed, the initial dialogue text is filtered to obtain the current dialogue text; The current dialogue text is completed to obtain the target dialogue text.

3. The query text sorting method as described in claim 2, characterized in that, The step of completing the current dialogue text to obtain the target dialogue text includes: Obtain the historical dialogue text and the corresponding historical reply text; the historical dialogue text refers to the text of the previous round of dialogue in the current dialogue text; the historical dialogue text and the historical reply text both correspond to the same historical text tag; The historical dialogue text, the historical reply text, and the current dialogue text are concatenated to obtain the initial text; Obtain the current text tag corresponding to the current dialogue text, and concatenate the historical text tag and the current text tag to obtain the initial tag; Input the initial text and the initial label into a preset text query model to obtain the reference entity position corresponding to the initial text and the position to be completed corresponding to the current dialogue text output by the preset text query model; Extract the referential entity text corresponding to the referential entity position from the initial text, and perform text completion on the current dialogue text based on the referential entity text and the position to be completed to obtain the target dialogue text.

4. The query text sorting method as described in claim 1, characterized in that, The process of recalling the target dialogue text to obtain candidate dialogue texts includes: The target dialogue text is subjected to text detection to obtain text detection results; the text detection results include detection failure results; the detection failure results indicate that the target dialogue text contains errors. The target dialogue text corresponding to the detection failure result is corrected to obtain the corrected dialogue text; Obtain a preset intent recognition model, and use the preset intent recognition model to perform intent recognition on the correction dialogue text to obtain the text intent recognition result; Obtain a set of candidate dialogue texts, wherein the set of candidate dialogue texts includes at least one candidate dialogue text; The candidate dialogue text and the text intent recognition result are semantically similar to each other to obtain a semantic similarity matching result. Candidate dialogue texts whose semantic similarity matching results are greater than or equal to a preset similarity threshold are identified as candidate dialogue texts.

5. The query text sorting method as described in claim 1, characterized in that, Before performing vector processing on the target dialogue text using a preset text model, the process includes: Obtain a sample training dataset, which includes at least one text training data; one text training data corresponds to one sample label. Obtain a preset training model, and use the preset training model to perform vector transformation on the text training data to obtain sample text vectors; The sample text vector is input into the preset training model to obtain the sample semantic vector corresponding to the sample text vector, and the sample semantic vector is predicted to obtain the predicted label. The prediction loss value of the text training data is determined based on the sample label and the prediction label corresponding to the same text training data; When the predicted loss value does not reach the preset convergence condition, the initial parameters in the preset training model are iteratively updated until the predicted loss value reaches the convergence condition. Then, the converged preset training model is recorded as the preset text model.

6. A query text sorting device, characterized in that, include: The text recall module is used to obtain the target dialogue text, perform recall processing on the target dialogue text, and obtain candidate dialogue text. The vector conversion module is used to perform vector processing on the target dialogue text through a preset text model to obtain the target text vector, and to obtain the candidate text vector corresponding to the candidate dialogue text from a preset vector library. The semantic vector module is used to input the candidate text vector and the target text vector into the preset text model to obtain the candidate semantic vector corresponding to the candidate text vector and the target semantic vector corresponding to the target text vector; The text sorting module is used to perform prediction processing on the candidate semantic vector and the target semantic vector to obtain prediction values, and sort the candidate dialogue text according to the prediction values ​​to obtain the text sorting result. The semantic vector module includes: An attention unit is used to perform attention processing on the candidate text vector through the attention layer in the preset text model to obtain an attention vector; The normalization unit is used to input all the attention vectors into the residual layer in the preset text model and obtain the residual vectors output by the residual layer. The linear mapping unit is used to perform linear mapping processing on the residual vector through the fully connected layer in the preset text model to obtain the candidate semantic vector.

7. The query text sorting device as described in claim 6, characterized in that, The text recall module includes: The word segmentation unit is used to acquire the initial dialogue text, perform word segmentation on the initial dialogue text, and obtain at least one word to be processed in the initial dialogue text. An entity recognition unit is used to perform entity recognition on the word to be processed and obtain an entity recognition result corresponding to the word to be processed. The filtering unit is used to filter the initial dialogue text based on the entity recognition results corresponding to each word to be processed, so as to obtain the current dialogue text. The text completion unit is used to complete the current dialogue text to obtain the target dialogue text.

8. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the query text sorting method as described in any one of claims 1 to 5.

9. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the query text sorting method as described in any one of claims 1 to 5.

Images