Query rewriting method and its device, equipment, and medium

Abstract

The present application relates to a query rewriting method and its apparatus, device, and medium in the field of computer technology. The method comprises: obtaining a query text input by a user; using a text classification model to determine a question text matching the query text from a preset shopping guide database, obtaining the question text and its corresponding answer text; determining keywords in the question text and its corresponding answer text, wherein the keywords are descriptive information of product features; constructing the keywords as replacement text for the query text, executing a product query with the replacement text, and recalling a list of products matching the replacement text and pushing it to the user. The present application introduces shopping guide-like question and answer data to redirect the query text, which can optimize the user's query results.

Description

Technical Field

[0001] The present application relates to the field of e-commerce technology, and in particular to a query rewriting method and its corresponding device, computer equipment, and computer-readable storage medium. Background Art

[0002] In e-commerce search scenarios, buyers can enter arbitrary query text, and the search engine will search for matching product text based on the query text and provide the corresponding products as search results. However, the query text entered by buyers is often arbitrary and cannot directly reflect the buyer's actual intention. There are often numerous inconsistencies between the query text and the product text, resulting in too few or even zero products corresponding to the recalled product text. Such search results fail to meet the buyer's needs, wasting the buyer's search.

[0003] Traditionally, when there are too few or even zero recalled products, a redirected product query operation is typically used. This involves replacing the current query text with the replacement text to execute the product query. However, the correlation between the replacement text and the query text cannot be determined, resulting in the recall of recalled products failing to meet the needs of buyers.

[0004] In view of this, the applicant has been engaged in research in related fields for a long time and has made corresponding explorations to solve industry problems in the e-commerce field. Summary of the Invention

[0005] The primary purpose of the present application is to solve at least one of the above problems and to provide a query rewriting method and its corresponding device, computer equipment, and computer-readable storage medium.

[0006] In order to meet the various objectives of this application, this application adopts the following technical solutions:

[0007] A query rewriting method provided to meet one of the purposes of this application includes the following steps:

[0008] Get the query text entered by the user;

[0009] Using a text classification model to determine a question text that matches the query text from a preset shopping guide database, and obtaining the question text and its corresponding answer text;

[0010] Determining keywords in the question text and its corresponding answer text, wherein the keywords are descriptive information of product characteristics;

[0011] The keyword is constructed as a replacement text for the query text, a product query is performed using the replacement text, and a product list matching the replacement text is retrieved and pushed to the user.

[0012] In a further embodiment, before obtaining the query text input by the user, the following steps are also included:

[0013] Using question texts in a preset shopping guide database, replacing some word units in the question texts with masking marks to construct training samples, and training the codec network to a convergence state so that it is suitable for predicting the replaced word units corresponding to the masking marks;

[0014] A twin-tower model is constructed using the encoders in the same two encoding and decoding networks as the text classification model, and two different question texts in the shopping guide database are used to form sentence pairs. Their supervision labels are set accordingly, and the text classification model is trained to a convergence state so that it is suitable for determining the similarity of the two text descriptions in the sentence pair. The two encoders in the twin-tower model share weights, and the supervision label represents whether the two question text descriptions are similar.

[0015] In a further embodiment, the step of using a text classification model to determine a question text matching the query text from a preset shopping guide database includes the following steps:

[0016] Constructing sentence pairs with the question text in a preset shopping guide database and the query text;

[0017] Using a text classification model to extract the deep semantic information corresponding to the two texts in the sentence pair to obtain corresponding encoding representations;

[0018] Perform average pooling on the corresponding encoding representations of the two texts in the sentence pair and calculate the similarity between the two pooling results;

[0019] Filter out sentence pairs whose similarity meets the preset threshold and obtain the question text.

[0020] In a further embodiment, the step of determining keywords in the question text and its corresponding answer text includes the following steps:

[0021] Matching a preset product word list to determine the product words mentioned in the question text and its corresponding answer text;

[0022] Count the frequency of product words and select product words that meet the preset threshold as keywords.

[0023] In a further embodiment, the step of constructing a twin-tower model using the encoders in the same two encoding and decoding networks as the text classification model, using two different question texts in the shopping guide database to form sentence pairs, setting their supervision labels accordingly, and training the text classification model to a convergence state includes the following steps:

[0024] Obtaining a single training sample and its supervisory label from a prepared training set, wherein the training sample includes a sentence pair consisting of two question texts with similar descriptions in a shopping guide database, and the supervisory label indicates that the two question texts in the sentence pair of the training sample have similar descriptions;

[0025] Inputting the training sample into a text classification model to extract deep semantic information corresponding to the two question texts in the sentence pair, and obtaining corresponding encoding representations;

[0026] Perform average pooling on the corresponding encoding representations of the two texts in the training sample and calculate the similarity between the two pooling results;

[0027] The supervisory label of the training sample is used to determine the loss value of the similarity. When the loss value does not reach a preset threshold, the weight of the text classification model is updated, and other training samples are continuously called to perform iterative training until the model converges.

[0028] In a further embodiment, before obtaining the query text input by the user, the following steps are also included:

[0029] Obtaining text information of multiple products in a product database, wherein the text information is product description information;

[0030] Segmenting each of the text information to obtain corresponding segmentation tokens;

[0031] The word frequencies corresponding to each segmented word grammar are counted, and the segmented word grammar corresponding to the word frequencies that meet the preset threshold are selected as product words to construct a product word list.

[0032] In a further embodiment, before obtaining the query text input by the user, it also includes: obtaining chat data between the buyer user and the seller user in the e-commerce shopping guide scenario, determining the question text and its corresponding answer text, and constructing a shopping guide database.

[0033] On the other hand, a query rewriting device provided to meet one of the purposes of the present application includes a text acquisition module, a text matching module, a keyword determination module and a list push module, wherein the text acquisition module is used to obtain the query text input by the user; the text matching module is used to use a text classification model to determine the question text matching the query text from a preset shopping guide database, and obtain the question text and its corresponding reply text; the keyword determination module is used to determine the keywords in the question text and its corresponding reply text, wherein the keywords are descriptive information of the product characteristics; the list push module is used to construct the keywords as replacement text for the query text, execute product query with the replacement text, and recall the product list matching the replacement text and push it to the user.

[0034] In a further embodiment, the text acquisition module further includes: a codec network training module, which is used to use the question text in a preset shopping guide database, replace some of the words in the question text with masking marks to construct training samples, and train the codec network to a convergent state so that it is suitable for predicting the replaced words corresponding to the masking marks; a text classification model training module, which is used to construct a double-tower model as the text classification model using the encoders in the same two codec networks, use two different question texts in the shopping guide database to form a sentence pair, set their supervision labels accordingly, and train the text classification model to a convergent state so that it is suitable for determining the similarity of the two text descriptions in the sentence pair. The two encoders in the double-tower model share weights, and the supervision label represents whether the two question text descriptions are similar.

[0035] In a further embodiment, the text matching module includes: a sentence pair construction submodule, which is used to construct a sentence pair with the question text in a preset shopping guide database and the query text; a first coding representation submodule, which is used to use a text classification model to extract the deep semantic information corresponding to the two texts in the sentence pair and obtain the corresponding coding representation; a first similarity calculation submodule, which is used to perform average pooling on the coding representations corresponding to the two texts in the sentence pair and calculate the similarity between the two pooling results; a sentence pair screening submodule, which is used to screen out sentence pairs whose similarity meets a preset threshold and obtain the question text therein.

[0036] In a further embodiment, the keyword determination module includes: a vocabulary matching submodule, which is used to match a preset product vocabulary to determine the product words mentioned in the question text and its corresponding reply text; a keyword screening submodule, which is used to count the word frequency of product words and screen out product words corresponding to the word frequency that meets the preset threshold as keywords.

[0037] In a further embodiment, the text classification model training module includes: a sample acquisition submodule, which is used to obtain a single training sample and its supervisory label from a prepared training set, wherein the training sample includes a sentence pair consisting of two question texts with similar descriptions in a shopping guide database, and the supervisory label represents that the two question texts in the sentence pair of the training sample are similar in description; a second encoding representation submodule, which is used to input the training sample into the text classification model to extract the deep semantic information corresponding to the two question texts in the sentence pair, and obtain the corresponding encoding representation; a second similarity calculation submodule, which is used to average pool the encoding representations corresponding to the two texts in the training sample, and calculate the similarity between the two pooling results; an iterative training submodule, which is used to use the supervisory label of the training sample to determine the loss value of the similarity. When the loss value does not reach a preset threshold, the text classification model is weighted updated, and other training samples are continued to be called to perform iterative training until the model converges.

[0038] In a further embodiment, the text acquisition module also includes: an information acquisition module for acquiring text information of multiple commodities in a commodity database, wherein the text information is the description information of the commodities; a text segmentation module for segmenting each of the text information to obtain corresponding segmentation word elements; a word list construction module for counting the word frequency corresponding to each segmentation word element, screening out the segmentation word elements corresponding to the word frequency that meets the preset threshold as commodity words, and constructing a commodity word list.

[0039] In a further embodiment, before the text acquisition module, it also includes: a database construction module, which is used to obtain chat data between buyer users and seller users in the e-commerce shopping guide scenario, determine the question text and its corresponding answer text, and construct a shopping guide database.

[0040] On the other hand, a computer device provided to meet one of the purposes of the present application includes a central processing unit and a memory, wherein the central processing unit is used to call and run a computer program stored in the memory to execute the steps of the query rewriting method described in the present application.

[0041] On the other hand, a computer-readable storage medium is provided to meet another purpose of the present application, which stores a computer program implemented according to the query rewriting method in the form of computer-readable instructions. When the computer program is called and executed by a computer, it executes the steps included in the method.

[0042] The technical solution of this application has many advantages, including but not limited to the following:

[0043] This application adopts a text classification model to determine the question text that matches the query text entered by the user from a preset shopping guide database, obtain the question text and its corresponding reply text, determine the keywords that describe the characteristics of the product, construct a replacement text with the keywords, replace the query text to execute the product query, and recall the product list that matches the replacement text and push it to the user. It can be understood that in the e-commerce search scenario, the corresponding product is queried based on the query text entered by the user, and in the e-commerce shopping guide scenario, the question text with the nature of shopping guide and its corresponding reply text, the two scenarios are based on the user's description to determine the corresponding product. Therefore, the question text that matches the query text is equivalent to a different description of the query text. Based on this, the keywords that describe the characteristics of the product in the question text and its corresponding reply text can accurately represent the purchase intention expressed by the query text to a certain extent. Therefore, the replacement text constructed with the keyword can replace the query text to execute the product query, effectively solving the scenario where the recalled products are too few or even zero. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:

[0045] Figure 1 A flowchart of a typical embodiment of the query rewriting method of the present application;

[0046] Figure 2 A schematic diagram of the model structure of a text classification model in an embodiment of the present application;

[0047] Figure 3 Schematic diagram of the construction and training process of the text classification model in the embodiment of the present application;

[0048] Figure 4 A schematic diagram of a process for determining a question text that matches a query text using a text classification model in an embodiment of the present application;

[0049] Figure 5 A schematic diagram of a process for determining keywords in a question text and its corresponding answer text in an embodiment of the present application;

[0050] Figure 6 A schematic diagram of the training process of the text classification model in an embodiment of the present application;

[0051] Figure 7 A schematic diagram of the process of constructing a product vocabulary in an embodiment of the present application;

[0052] Figure 8 This is a functional block diagram of the query rewriting device of this application;

[0053] Figure 9 This is a schematic diagram of the structure of a computer device used in this application. DETAILED DESCRIPTION

[0054] The following describes in detail embodiments of the present application, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present application, and are not to be construed as limiting the present application.

[0055] It will be understood by those skilled in the art that, unless expressly stated otherwise, the singular forms "a", "an", "said" and "the" used herein may also include the plural forms. It should be further understood that the term "comprising" used in the specification of the present application refers to the presence of the features, integers, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or groups thereof. It should be understood that when we refer to an element as being "connected" or "coupled" to another element, it may be directly connected or coupled to the other element, or there may be intermediate elements. In addition, "connected" or "coupled" as used herein may include wireless connections or wireless couplings. The term "and / or" used herein includes all or any units and all combinations of one or more associated listed items.

[0056] It will be understood by those skilled in the art that, unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by those skilled in the art to which this application belongs. It should also be understood that terms such as those defined in common dictionaries should be understood to have meanings consistent with their meanings in the context of the prior art and will not be interpreted in an idealized or overly formal sense unless specifically defined as herein.

[0057] Those skilled in the art will appreciate that the terms "client," "terminal," and "terminal device" as used herein include both devices that are wireless signal receivers, i.e., devices that only have wireless signal receivers without transmission capabilities, and devices that have receiving and transmitting hardware capable of two-way communication over a two-way communication link. Such devices may include: cellular or other communication devices such as personal computers and tablet computers, which have single-line displays, multi-line displays, or cellular or other communication devices without multi-line displays; PCS (Personal Communications Service), which may combine voice, data processing, fax, and / or data communication capabilities; PDA (Personal Digital Assistant), which may include a radio frequency receiver, a pager, Internet / Intranet access, a web browser, a notepad, a calendar, and / or a GPS (Global Positioning System) receiver; and conventional laptop and / or palmtop computers or other devices, which have and / or include a radio frequency receiver. As used herein, the terms "client," "terminal," or "terminal device" may be portable, transportable, installed in a vehicle (air, sea, and / or land), or adapted and / or configured to operate locally, and / or in a distributed manner, at any other location on Earth and / or in space. As used herein, the terms "client," "terminal," or "terminal device" may also refer to a communication terminal, an Internet access terminal, or a music / video playback terminal, such as a PDA, an MID (Mobile Internet Device), and / or a mobile phone with a music / video playback function, or may refer to a smart TV, a set-top box, or the like.

[0058] The hardware referred to by names such as "server", "client", and "service node" in this application is essentially an electronic device with capabilities equivalent to those of a personal computer. It is a hardware device that has the necessary components revealed by the von Neumann principle, such as a central processing unit (including an arithmetic unit and a controller), a memory, an input device, and an output device. Computer programs are stored in its memory, and the central processing unit loads the program stored in the external memory into the internal memory for execution, executes the instructions in the program, and interacts with the input and output devices to complete specific functions.

[0059] It should be noted that the concept of "server" referred to in this application can also be extended to server clusters. Based on the network deployment principles understood by those skilled in the art, the servers described should be logically divided. In physical space, these servers can be independent of each other but callable through interfaces, or integrated into a single physical computer or a computer cluster. Those skilled in the art should understand this flexibility and should not use it to constrain the implementation of the network deployment method of this application.

[0060] Unless expressly specified, one or more technical features of the present application can be deployed on a server for implementation and accessed by a client through a remote call to obtain an online service interface provided by the server, or can be directly deployed and run on a client for implementation.

[0061] Unless expressly specified otherwise, the neural network models referenced or may be referenced in this application may be deployed on a remote server and remotely called on the client, or may be deployed and directly called on a client with sufficient device capabilities. In some embodiments, when it runs on the client, its corresponding intelligence may be obtained through transfer learning to reduce the requirements for the client's hardware operating resources and avoid excessive occupation of the client's hardware operating resources.

[0062] Unless explicitly specified, the various data involved in this application can be stored remotely on a server or on a local terminal device, as long as they are suitable for being called by the technical solution of this application.

[0063] Those skilled in the art should be aware that although the various methods of this application are described based on the same concept and thus exhibit commonality, unless otherwise specified, these methods can be independently executed. Similarly, the various embodiments disclosed in this application are all based on the same inventive concept. Therefore, concepts with the same expression, as well as concepts that are appropriately transformed for convenience despite different expression, should be understood as equivalent.

[0064] Unless expressly stated to be mutually exclusive, the various embodiments disclosed in this application may be cross-combined with the relevant technical features of the various embodiments to flexibly construct new embodiments, as long as such combination does not deviate from the creative spirit of this application and can meet the needs of the prior art or resolve certain deficiencies in the prior art. Those skilled in the art should be aware of such flexibility.

[0065] A query rewriting method of the present application can be programmed as a computer program product and deployed in a client or server for execution. For example, in the exemplary application scenario of the present application, it can be deployed and implemented in the server of an e-commerce platform. The method can be executed by accessing the interface opened after the computer program product is run and performing human-computer interaction with the process of the computer program product through a graphical user interface.

[0066] See also Figure 1 The query rewriting method of the present application, in its typical embodiment, includes the following steps:

[0067] Step S1100: obtaining a query text input by a user;

[0068] In an exemplary e-commerce scenario, a cross-border e-commerce platform provides an independent website service. Products are rented on the cross-border e-commerce platform's independent website, where an online store is deployed and product information is listed for purchase by platform buyers. To facilitate buyers' search for products in the online store, the cross-border e-commerce platform also provides a product search service. Buyers can enter a query text through the product search service provided by the online store and submit it to the product search service to search for products matching the query text in the online store.

[0069] When a buyer enters a query text, the page program encapsulates the query text into a product search request and submits it to the product search service. The independent site server providing the product search service receives the product search request and parses it to obtain the query text.

[0070] In an alternative embodiment based on this, before the page program of the buyer's terminal device encapsulates the query text, or after the server parses the query text from the product search request, the query text can be pre-processed to clean it and filter out invalid characters to facilitate subsequent product searches. The text pre-processing can include removing stop words, removing spaces, and other routine operations that do not change the original meaning.

[0071] Furthermore, the e-commerce search scenario that this application aims to solve is that the query text entered by the user has too few search results, or even 0. Therefore, after the server obtains the query text, it drives the search engine pre-built in the server, performs a character matching search with the query text, obtains the product corresponding to the matching product text as the search result, and determines that the search result is less than a preset threshold, or even 0, to perform the technical implementation of the subsequent steps. The preset threshold indicates that the search results are too few to meet the actual needs of the user, for example, one or two. Those skilled in the art can flexibly set the preset threshold according to the disclosure herein. The character matching can be an exact match and / or a fuzzy match, and the search engine can use Elasticsearch, which is a distributed search and analysis engine at the core of Elastic Stack.

[0072] Step S1200: using a text classification model to determine a question text matching the query text from a preset shopping guide database, and obtaining the question text and its corresponding answer text;

[0073] Generally speaking, cross-border e-commerce platforms establish chat windows for each independent site. Each site is assigned its own program entry point, allowing online stores hosted on each site to chat with buyers. Buyers can access the chat window of the corresponding online store through this program entry point and chat with sellers there. For example, in a common e-commerce shopping guide scenario, buyers can enter text describing their needs and send it to the chat window. After the seller user checks, if the user's needs can be directly determined based on this, the corresponding products in the store can be pushed to the chat window as a reply; if the user's needs cannot be directly determined, the product characteristics of the store's products, such as product attributes, product categories, product brands, product names, etc., can be further based on the product characteristics of the store, such as product attributes, product categories, product brands, product names, etc., and a text containing descriptive information of the product characteristics can be sent to the chat window as a reply. After the buyer user checks, he can determine whether the product characteristics described in the reply text are what he needs, and enter the corresponding text and send it to the chat window. In this way, through the chat between the two, the seller user can continuously provide the buyer user with text containing descriptive information of the product characteristics. In the process, based on the text input by the buyer user, it is confirmed whether the descriptive information of the product characteristics provided is what the buyer user needs. This cycle repeats over and over again. Finally, the seller user can determine the user's needs and push the corresponding products in the store to the chat window as a reply.

[0074] Based on this, chat data between buyer and seller users in e-commerce shopping guide scenarios can be obtained. In one embodiment, all text entered by the buyer into the chat window can be used as question text. If multiple texts are entered in a short period of time, they are concatenated into a long text. Subsequent text entered by the seller into the chat window can be used as the corresponding answer text. If multiple texts are entered, they are concatenated into a long text. The question text and its corresponding answer text in the determined chat data are associated with the online store that generated the chat data, thereby constructing a shopping guide database.

[0075] The text classification model is a double-tower model, and the specific model structure is as follows: Figure 2 As shown, the system includes two input paths, each receiving the two sentences in the sentence pair. Each input path must pass through the corresponding encoder and pooling layer, and then the similarity is calculated based on the pooling results output by the two pooling layers as the model output. The two corresponding encoders share weights, and the two corresponding pooling layers perform average pooling. The encoder is suitable for vectorizing the input text and transforming learning in multi-layer semantic spaces. It can be selected from a variety of known models, including but not limited to BERT, RNN, Bi LSTM, Bi GRU, RoBERTa, ALBert, ERNI E, BERT-WWM, etc.

[0076] Each question text in the shopping guide database is used to form a plurality of sentence pairs corresponding to the query text. Each sentence pair is used as the input of the text classification model to obtain the similarity between the query text and each question text, thereby determining the question text whose similarity exceeds a preset threshold and is deemed to match the query text. Further, the matched question text and its corresponding answer text are obtained.

[0077] Step S1300: Determine keywords in the question text and its corresponding answer text, where the keywords are descriptive information of product features;

[0078] According to the online store associated with the question text and its corresponding answer text, the product word corresponding to the online store is obtained from the preset product word table, and the question text and its corresponding answer text are character matched with the product word to determine the mentioned product word.

[0079] Furthermore, a bag-of-words model may be used to count the frequency of each of the mentioned product words, and select product words whose frequency meets a preset threshold as keywords. The preset threshold may be set as needed by those skilled in the art.

[0080] The product word list stores product words corresponding to each online store. The product words are partial text in the text information of the corresponding product and can describe the product characteristics of the corresponding product. The construction of the product word list can be further explained in the subsequent embodiments, and this step will not be discussed here for the time being.

[0081] Step S1400: construct the keyword as a replacement text for the query text, execute a product search with the replacement text, and retrieve a product list matching the replacement text and push it to the user.

[0082] A replacement text is constructed based on the keyword, for example: "keyword 1" or "keyword 2" or "keyword 3", and the query text called by the current product query is replaced with the replacement text. The product query is executed, and products that match the replacement text are recalled to construct a product list, and the product list is pushed to the user.

[0083] It can be seen from the typical embodiments of the present application that the technical solution of the present application has many advantages, including but not limited to the following aspects:

[0084] This application adopts a text classification model to determine the question text that matches the query text entered by the user from a preset shopping guide database, obtain the question text and its corresponding reply text, determine the keywords that describe the characteristics of the product, construct a replacement text with the keywords, replace the query text to execute the product query, and recall the product list that matches the replacement text and push it to the user. It can be understood that in the e-commerce search scenario, the corresponding product is queried based on the query text entered by the user, and in the e-commerce shopping guide scenario, the question text with the nature of shopping guide and its corresponding reply text, the two scenarios are based on the user's description to determine the corresponding product. Therefore, the question text that matches the query text is equivalent to a different description of the query text. Based on this, the keywords that describe the characteristics of the product in the question text and its corresponding reply text can accurately represent the purchase intention expressed by the query text to a certain extent. Therefore, the replacement text constructed with the keyword can replace the query text to execute the product query, effectively solving the scenario where the recalled products are too few or even zero.

[0085] See also Figure 3 In a further embodiment, before step S1100, obtaining the query text input by the user, the following steps are also included:

[0086] Step S1010: Using question texts in a preset shopping guide database, replacing some word units in the question texts with masking marks to construct training samples, and training the codec network to a convergence state so that it is suitable for predicting the replaced word units corresponding to the masking marks;

[0087] The encoding and decoding network is Bert and adopts an Encoder-Decoder (encoder-decoder) structure.

[0088] Obtain the question text from the shopping guide database, and replace some of the word units in the question text with masking marks based on the masking mechanism. The masking mechanism is for each word unit in the question text:

[0089] With a probability of 85%, the original word is retained unchanged.

[0090] 15% probability, use the following replacement:

[0091] With a probability of 80%, the character [MASK], i.e., the masking identifier, is used to replace the current word.

[0092] With a 20% probability, the original word is retained unchanged.

[0093] The replaced question text is used as a training sample, and the replaced word units corresponding to the masked tokens are used as supervision labels to train the encoder-decoder network to a convergent state. Specifically;

[0094] A single training sample is obtained and input into the codec network. The training sample is bidirectionally encoded by the encoder of the codec network, and the deep semantic information of the training sample is extracted to obtain a corresponding encoding feature vector. The deep semantic information includes each word in the training sample, and based on the attention interaction in its context, each element in the encoding feature vector corresponds to the semantic information obtained. The encoding feature vector is decoded by the decoder of the codec network to predict the word corresponding to each covered mark in the training sample. The supervisory label of the training sample is used to determine the loss value of the word corresponding to each covered mark. When the loss value does not reach the preset threshold, the weight of the codec network is updated, and other training samples are continuously called to perform iterative training until the codec network converges.

[0095] It is not difficult to understand that after training to convergence, the resulting codec network can accurately predict the replaced word corresponding to the masked mark. Based on this, it is ensured that the encoder in the codec network performs bidirectional encoding on the text input to it, and the resulting encoded feature vector can accurately represent the semantics of the text.

[0096] Step S1020: Construct a twin-tower model using the encoders in the same two encoding and decoding networks as the text classification model, use two different question texts in the shopping guide database to form a sentence pair, set their supervision labels accordingly, and train the text classification model to a convergence state so that it is suitable for determining the similarity of the two text descriptions in the sentence pair. The two encoders in the twin-tower model share weights, and the supervision label represents whether the two question text descriptions are similar.

[0097] The encoder in the codec network trained to convergence in the above step is constructed with another encoder that is identical to it. The two encoders are each followed by a pooling layer, and the two pooling layers are together followed by a similarity calculation module. Thus, a double-tower model is constructed as the text classification model, as shown in the reference diagram. Figure 2 As shown, the two encoder weights are shared and both pooling layers perform average pooling.

[0098] Two question texts with similar descriptions are manually selected from the shopping guide database to form a sentence pair as a positive sample, and two question texts with dissimilar descriptions are manually selected to form a sentence pair as a negative sample. The positive and negative samples are labeled with corresponding supervisory labels. The supervisory labels indicate whether the two question texts are similar. For example, the supervisory label of the positive sample is 1, and the supervisory label of the negative sample is 0. Thus, the text classification model is trained to a convergent state using the positive and negative samples and their corresponding supervisory labels. Specifically:

[0099] A single positive sample or negative sample is obtained as a training sample and input into a text classification model. The two encoders of the text classification model respectively perform bidirectional encoding on the two question texts in the sentence pair, extract the deep semantic information of each question text, and obtain a corresponding encoded feature vector. The deep semantic information includes each word in the question text. Based on the attention interaction in its context, the corresponding semantic information obtained, each element in the encoded feature vector represents the semantic representation of the corresponding word in the question text. The encoded feature vectors corresponding to the two question texts are respectively input into two pooling layers for average pooling to obtain the corresponding pooling results. The two-way pooling results are input into the similarity calculation module to calculate the similarity. It can be understood that the similarity represents the degree of semantic similarity between the two question texts in the input sentence pair. Furthermore, the supervisory label of the training sample is used to determine the loss value of the similarity. When the loss value does not reach the preset threshold, the weight of the text classification model is updated, and other training samples are continuously called to perform iterative training until the text classification model converges.

[0100] In this embodiment, the encoder of the text classification model is derived from the encoder of a codec network trained to convergence. This codec network uses training samples constructed by replacing some tokens in the question text with masked markers. By rationalizing the noise in the training samples and adjusting the difficulty of the codec network in understanding the input text, the encoder in the converged codec network encodes the input text, resulting in an encoded representation that accurately represents the semantics of the input text. Furthermore, the text classification model has a simple structure, is easily trained to convergence, and performs efficiently.

[0101] See also Figure 4 In a further embodiment, step S1200, the step of using a text classification model to determine a question text matching the query text from a preset shopping guide database, includes the following steps:

[0102] Step S1210: forming a sentence pair with the question text in the preset shopping guide database and the query text;

[0103] Each question text in the shopping guide database is used to form a plurality of sentence pairs corresponding to the query text, and each sentence pair is used as the input of the text classification model.

[0104] Step S1220: Using a text classification model to extract deep semantic information corresponding to the two texts in the sentence pair, and obtain corresponding encoding representations;

[0105] For each sentence pair, two encoders of the text classification model are used to extract the deep semantic information corresponding to the question text and the query text in the sentence pair respectively, and obtain the corresponding encoded feature vector, namely the encoded representation. The deep semantic information includes each word in the corresponding text. Based on the attention interaction in its context, the corresponding semantic information obtained, each element in the encoded feature vector represents the semantic representation of the corresponding word in the text.

[0106] Step S1230: Perform average pooling on the corresponding encoded representations of the two texts in the sentence pair, and calculate the similarity between the two pooling results;

[0107] Furthermore, two pooling layers of the text classification model are used to perform average pooling on the encoded feature vectors corresponding to the question text and the query text, respectively, to obtain two corresponding pooling results. Then, a similarity calculation module is used to calculate the similarity between the two pooling results. The similarity calculation module can be implemented by any one of the large-scale vector retrieval engines such as Faiss, ElasticSearch, Milvus, etc., or by any ready-made algorithm such as cosine similarity, dot product, Manhattan distance, Euclidean distance, etc.

[0108] Step S1240: Filter out sentence pairs whose similarity meets a preset threshold, and obtain the question text therein.

[0109] As described above, a text classification model can be used to calculate a corresponding similarity for each sentence pair. The similarity represents the degree of semantic similarity between the question text and the query text in the sentence pair. Thus, sentence pairs whose similarity meets a preset threshold can be screened out to obtain the question text therein. This question text is the text with the highest degree of semantic similarity to the query text. The preset threshold can be set as needed by those skilled in the art with reference to the disclosure herein.

[0110] In this embodiment, the text classification model implemented in this application is used to calculate the similarity between each question text in the shopping guide database and the query text. Based on this, sentence pairs whose similarity meets a preset threshold are selected. The question text in this sentence pair has a high degree of semantic similarity with the query text. This is a fully intelligent implementation with efficient execution and no human resources are required.

[0111] See also Figure 5 In a further embodiment, step S1300, the step of determining keywords in the question text and its corresponding answer text, includes the following steps:

[0112] Step S1310: Matching a preset product word list to determine product words mentioned in the question text and its corresponding answer text;

[0113] According to the online store associated with the question text and its corresponding answer text, the product word corresponding to the online store is obtained from the preset product word table, and the question text and its corresponding answer text are character matched with the product word to determine the mentioned product word.

[0114] Step S1320: Count the word frequencies of the product words, and select product words corresponding to the word frequencies that meet a preset threshold as keywords.

[0115] The bag-of-words model may be used to count the word frequencies corresponding to the mentioned product words, and product words whose word frequencies meet a preset threshold may be selected as keywords. The preset threshold may be set as needed by those skilled in the art.

[0116] In this embodiment, by character matching the question text and its corresponding answer text with the product words of the corresponding online store in the product word list, the mentioned product words are determined, and then based on the word frequency corresponding to the product words, the product words corresponding to the word frequency that meets the preset threshold are screened out as keywords, which is efficient and convenient.

[0117] See also Figure 6In a further embodiment, step S1020, constructing a twin-tower model as the text classification model using the encoders in the two identical encoding and decoding networks, using two different question texts in the shopping guide database to form sentence pairs, setting their supervision labels accordingly, and training the text classification model to a convergence state, includes the following steps:

[0118] Step S1021: Obtain a single training sample and its supervisory label from a prepared training set, wherein the training sample includes a sentence pair consisting of two question texts with similar descriptions in a shopping guide database, and the supervisory label indicates that the two question texts in the sentence pair of the training sample have similar descriptions;

[0119] The training set can be prepared in advance. For example, two question texts with similar descriptions can be manually selected from a shopping guide database to form sentence pairs as training samples. These training samples are labeled with corresponding supervisory labels. The supervisory labels indicate that the two question texts are similar. For example, the supervisory label of the training sample is 1. Based on this, a sufficient number of training samples are constructed, and the corresponding supervisory labels are labeled for each training sample, so that the training set is constructed using each training sample and its supervisory label.

[0120] Based on this, training samples and their supervision labels can be obtained one by one from the training set to train the text classification model.

[0121] Step S1022: Input the training sample into a text classification model to extract deep semantic information corresponding to the two question texts in the sentence pair, and obtain corresponding encoding representations;

[0122] Two encoders of a text classification model are used to respectively extract deep semantic information corresponding to the two question texts in the sentence pair to obtain corresponding encoded feature vectors, namely the encoded representation. The deep semantic information includes each word in the question text. Based on the attention interaction in its context, corresponding to the semantic information obtained, each element in the encoded feature vector represents the semantic representation of the corresponding word in the question text.

[0123] Step S1023: perform average pooling on the corresponding encoded representations of the two texts in the training sample, and calculate the similarity between the two pooling results;

[0124] Two pooling layers of the text classification model are used to perform average pooling on the encoded feature vectors corresponding to the two question texts in the training sample to obtain two corresponding pooling results. Then, a similarity calculation module is used to calculate the similarity between the two pooling results. The similarity calculation module can be implemented by any large-scale vector retrieval engine such as Faiss, ElasticSearch, Milvus, etc., or by any ready-made algorithm such as cosine similarity, dot product, Manhattan distance, Euclidean distance, etc.

[0125] Step S1024: using the supervisory label of the training sample to determine the loss value of the similarity; when the loss value does not reach a preset threshold, performing weight update on the text classification model, and continuing to call other training samples to perform iterative training until the model converges.

[0126] A preset cross entropy loss function is called, which can be flexibly set by those skilled in the art based on prior knowledge or experimental experience. The cross entropy loss value of the similarity is calculated based on the supervisory label of the training sample. When the loss value reaches a preset threshold, it indicates that the text classification model has been trained to a convergence state, and the model training can be terminated. When the loss value does not reach the preset threshold, it indicates that the model has not converged, and then the model is gradient updated according to the loss value. The weight parameters of each link of the model are usually corrected through back propagation to make the model closer to convergence. Then, the next training sample in the training set is continued to be called to perform iterative training on the text classification model until the text classification model is trained to a convergence state.

[0127] In this embodiment, the training process of the text classification model is disclosed. After training to convergence, the text classification model learns the ability to calculate the similarity between two texts in a sentence pair input to it.

[0128] See also Figure 7 In a further embodiment, before step S1100, obtaining the query text input by the user, the following steps are also included:

[0129] Step S1000: Acquire text information of multiple commodities in a commodity database, wherein the text information is description information of the commodities;

[0130] Each independent site in the cross-border e-commerce platform independently maintains a product database of its own online store's products, and the product database is used to store each product in the online store and its corresponding product information.

[0131] The product information generally includes two categories: product images and text information. Product images are generally used to display the corresponding product. Text information generally refers to all product description information stored in association with the product and suitable for provision in text form, including but not limited to any one or more of the product title text, product attribute data, product details text, product image tags, etc. In terms of usage, the product description information is generally used to describe any specific information such as the brand, name, model, category, purpose, function, nature, selling point, usage, etc. of the product.

[0132] Based on this, the text information of each product is obtained from the product database of each online store.

[0133] Step S1001: Segment each of the text information to obtain corresponding segmentation word units;

[0134] The search engine mode of the Jieba word segmentation algorithm is used to segment each of the text information at a granular level to obtain corresponding word units. Those skilled in the art will understand that the precise mode of the Jieba word segmentation algorithm is to implement the most precise segmentation of the text information, segmenting the corresponding word units so as to avoid redundant data in the word units. However, the search engine mode, based on the precise mode, further segments the long words in the word units to achieve a finer granularity of the word units.

[0135] Based on this, the segmented word units corresponding to each of the text information can be obtained.

[0136] Step S1002: Count the word frequencies corresponding to the various segmented word units, select the segmented word units corresponding to the word frequencies that meet a preset threshold as product words, and construct a product word list.

[0137] For each online store, we use the TF-IDF algorithm to calculate the word frequency of each segmentation token corresponding to the online store. Specifically, taking a segmentation token as an example, we calculate the frequency of its occurrence in all text messages, namely TF (term frequency). We also count the number of text messages in which the segmentation token appears. The total number of text messages is divided by the number of text messages, and the IDF (inverse document frequency) is calculated. The word frequency of the segmentation token is then calculated as the TF corresponding to the segmentation token multiplied by the IDF.

[0138] Filter out segmented word units whose word frequency meets a preset threshold as product words, associate the product words with corresponding online stores, and construct a product word list. The preset threshold can be set as needed by those skilled in the art.

[0139] In this embodiment, a fine-grained word segmentation algorithm is used to segment product text information, enabling efficient and effective acquisition of more segmentation tokens, ensuring a richer selection of product terms from subsequent segmentation token filtering. Furthermore, a TF-IDF algorithm is employed to calculate the word frequency corresponding to each segmentation token. The resulting word frequency accurately reflects the importance of each segmentation token to the textual information in which it appears, ensuring the accuracy of subsequent product terms filtered based on the segmentation token frequency. This ensures that these product terms can accurately describe the product characteristics of the corresponding product.

[0140] See also Figure 8 , a query rewriting device provided to meet one of the purposes of the present application is a functional embodiment of the query rewriting method of the present application, the device includes a text acquisition module 1100, a text matching module 1200, a keyword determination module 1300 and a list push module 1400, wherein the text acquisition module 1100 is used to obtain the query text input by the user; the text matching module 1200 is used to use a text classification model to determine the question text matching the query text from a preset shopping guide database, and obtain the question text and its corresponding answer text; the keyword determination module 1300 is used to determine the keywords in the question text and its corresponding answer text, and the keywords are descriptive information of the product features; the list push module 1400 is used to construct the keywords as replacement text for the query text, execute product query with the replacement text, and recall the product list matching the replacement text and push it to the user.

[0141] In a further embodiment, the text acquisition module 1100 further includes: a codec network training module, which is used to use the question text in a preset shopping guide database, replace some of the words in the question text with masking marks to construct training samples, and train the codec network to a convergent state so that it is suitable for predicting the replaced words corresponding to the masking marks; a text classification model training module, which is used to construct a twin-tower model as the text classification model using the same two encoders in the codec network, use two different question texts in the shopping guide database to form a sentence pair, set their supervision labels accordingly, and train the text classification model to a convergent state so that it is suitable for determining the similarity of the two text descriptions in the sentence pair. The two encoders in the twin-tower model share weights, and the supervision label represents whether the two question text descriptions are similar.

[0142] In a further embodiment, the text matching module 1200 includes: a sentence pair construction submodule, which is used to construct a sentence pair with the question text in a preset shopping guide database and the query text; a first coding representation submodule, which is used to extract the deep semantic information corresponding to the two texts in the sentence pair using a text classification model to obtain the corresponding coding representation; a first similarity calculation submodule, which is used to perform average pooling on the coding representations corresponding to the two texts in the sentence pair and calculate the similarity between the two pooling results; a sentence pair screening submodule, which is used to screen out sentence pairs whose similarity meets a preset threshold and obtain the question text therein.

[0143] In a further embodiment, the keyword determination module 1300 includes: a vocabulary matching submodule, which is used to match a preset product vocabulary to determine the product words mentioned in the question text and its corresponding answer text; a keyword screening submodule, which is used to count the word frequency of product words and screen out product words corresponding to the word frequency that meets the preset threshold as keywords.

[0144] In a further embodiment, the text classification model training module includes: a sample acquisition submodule, which is used to obtain a single training sample and its supervisory label from a prepared training set, wherein the training sample includes a sentence pair consisting of two question texts with similar descriptions in a shopping guide database, and the supervisory label represents that the two question texts in the sentence pair of the training sample are similar in description; a second encoding representation submodule, which is used to input the training sample into the text classification model to extract the deep semantic information corresponding to the two question texts in the sentence pair, and obtain the corresponding encoding representation; a second similarity calculation submodule, which is used to average pool the encoding representations corresponding to the two texts in the training sample, and calculate the similarity between the two pooling results; an iterative training submodule, which is used to use the supervisory label of the training sample to determine the loss value of the similarity. When the loss value does not reach a preset threshold, the text classification model is weighted updated, and other training samples are continued to be called to perform iterative training until the model converges.

[0145] In a further embodiment, the text acquisition module 1100 further includes: an information acquisition module for acquiring text information of multiple commodities in a commodity database, wherein the text information is descriptive information of the commodities; a text segmentation module for segmenting each of the text information to obtain corresponding segmentation word elements; a word list construction module for counting the word frequency corresponding to each segmentation word element, screening out the segmentation word elements corresponding to the word frequency that meets the preset threshold as commodity words, and constructing a commodity word list.

[0146] In a further embodiment, before the text acquisition module 1100, it also includes: a database construction module, which is used to obtain chat data between buyer users and seller users in the e-commerce shopping guide scenario, determine the question text and its corresponding answer text, and construct a shopping guide database.

[0147] In order to solve the above technical problems, the embodiment of the present application also provides a computer device. Figure 9 As shown, a schematic diagram of the internal structure of a computer device. The computer device includes a processor, a computer-readable storage medium, a memory, and a network interface connected via a system bus. Among them, the computer-readable storage medium of the computer device stores an operating system, a database, and computer-readable instructions. The database may store a control information sequence. When the computer-readable instructions are executed by the processor, the processor may implement a query rewriting method. The processor of the computer device is used to provide computing and control capabilities to support the operation of the entire computer device. The memory of the computer device may store computer-readable instructions. When the computer-readable instructions are executed by the processor, the processor may execute the query rewriting method of the present application. The network interface of the computer device is used to connect and communicate with a terminal. Those skilled in the art will understand that Figure 9 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0148] In this embodiment, the processor is used to execute Figure 8 The memory stores the program code and various data required to execute the specific functions of each module and its submodule in the query rewriting device. The network interface is used to transmit data between user terminals or servers. The memory in this embodiment stores the program code and data required to execute all modules / submodules in the query rewriting device of this application, and the server can call the server's program code and data to execute the functions of all submodules.

[0149] The present application also provides a storage medium storing computer-readable instructions. When the computer-readable instructions are executed by one or more processors, the one or more processors execute the steps of the query rewriting method of any embodiment of the present application.

[0150] Those skilled in the art will appreciate that all or part of the processes in the above-mentioned embodiments of the present application can be implemented by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the above-mentioned embodiments of the method. The aforementioned storage medium can be a computer-readable storage medium such as a magnetic disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM).

[0151] In summary, this application introduces question-and-answer data with a shopping guide nature to redirect the query text, which can optimize the user's query results.

[0152] Those skilled in the art will appreciate that the steps, measures, and schemes in the various operations, methods, and processes discussed in this application may be interchanged, modified, combined, or deleted. Furthermore, other steps, measures, and schemes in the various operations, methods, and processes discussed in this application may also be interchanged, modified, rearranged, decomposed, combined, or deleted. Furthermore, steps, measures, and schemes in the prior art that are similar to those disclosed in this application may also be interchanged, modified, rearranged, decomposed, combined, or deleted.

[0153] The above description is only part of the implementation methods of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A query rewriting method, characterized in that: The steps include: Get the query text entered by the user; A text classification model is used to determine a question text that matches the query text from a preset shopping guide database, and obtain the question text and its corresponding answer text. The shopping guide database is constructed based on chat data between buyers and sellers in an e-commerce shopping guide scenario and includes multiple pairs of question texts and their corresponding answer texts sent by the buyers and sellers; Determining keywords in the question text and its corresponding answer text, wherein the keywords are descriptive information of product characteristics; The keyword is constructed as a replacement text for the query text, a product query is performed using the replacement text, and a product list matching the replacement text is retrieved and pushed to the user.

2. The query rewriting method according to claim 1, wherein: Before obtaining the query text entered by the user, the following steps are also included: Using question texts in a preset shopping guide database, replacing some word units in the question texts with masking marks to construct training samples, and training the codec network to a convergence state so that it is suitable for predicting the replaced word units corresponding to the masking marks; A twin-tower model is constructed using the encoders in the same two encoding and decoding networks as the text classification model, and two different question texts in the shopping guide database are used to form sentence pairs. Their supervision labels are set accordingly, and the text classification model is trained to a convergence state so that it is suitable for determining the similarity of the two text descriptions in the sentence pair. The two encoders in the twin-tower model share weights, and the supervision label represents whether the two question text descriptions are similar.

3. The query rewriting method according to claim 1, wherein: The step of using a text classification model to determine a question text matching the query text from a preset shopping guide database includes the following steps: Constructing sentence pairs with the question text in a preset shopping guide database and the query text; Using a text classification model to extract the deep semantic information corresponding to the two texts in the sentence pair to obtain corresponding encoding representations; Perform average pooling on the corresponding encoding representations of the two texts in the sentence pair and calculate the similarity between the two pooling results; Filter out sentence pairs whose similarity meets the preset threshold and obtain the question text.

4. The query rewriting method according to claim 1, wherein: The step of determining keywords in the question text and its corresponding answer text includes the following steps: Matching a preset product word list to determine the product words mentioned in the question text and its corresponding answer text; Count the frequency of product words and select product words that meet the preset threshold as keywords.

5. The query rewriting method according to claim 2, wherein: The steps of constructing a twin-tower model as the text classification model using the encoders in the same two encoding and decoding networks, using two different question texts in the shopping guide database to form sentence pairs, setting their supervision labels accordingly, and training the text classification model to a convergence state include the following steps: Obtaining a single training sample and its supervisory label from a prepared training set, wherein the training sample includes a sentence pair consisting of two question texts with similar descriptions in a shopping guide database, and the supervisory label indicates that the two question texts in the sentence pair of the training sample have similar descriptions; Inputting the training sample into a text classification model to extract deep semantic information corresponding to the two question texts in the sentence pair, and obtaining corresponding encoding representations; Perform average pooling on the corresponding encoding representations of the two texts in the training sample and calculate the similarity between the two pooling results; The supervisory label of the training sample is used to determine the loss value of the similarity. When the loss value does not reach a preset threshold, the weight of the text classification model is updated, and other training samples are continuously called to perform iterative training until the model converges.

6. The query rewriting method according to claim 1, wherein: Before obtaining the query text entered by the user, the following steps are also included: Obtaining text information of multiple products in a product database, wherein the text information is product description information; Segmenting each of the text information to obtain corresponding segmentation tokens; The word frequencies corresponding to each segmented word grammar are counted, and the segmented word grammar corresponding to the word frequencies that meet the preset threshold are selected as product words to construct a product word list.

7. The query rewriting method according to claim 1, wherein: Before obtaining the query text input by the user, the method also includes: obtaining chat data between buyer users and seller users in the e-commerce shopping guide scenario, determining the question text and its corresponding answer text, and constructing a shopping guide database.

8. A query rewriting device, characterized in that: include: The text acquisition module is used to obtain the query text input by the user; A text matching module is configured to use a text classification model to determine a question text that matches the query text from a preset shopping guide database, and obtain the question text and its corresponding answer text. The shopping guide database is constructed based on chat data between buyers and sellers in an e-commerce shopping guide scenario and includes multiple pairs of question texts and their corresponding answer texts sent by the buyers and sellers; A keyword determination module, configured to determine keywords in the question text and its corresponding answer text, wherein the keywords are descriptive information of product features; The list push module is used to construct the keyword as a replacement text for the query text, execute a product query with the replacement text, and recall a product list matching the replacement text and push it to the user.

9. A computer device comprising a central processing unit and a memory, characterized in that: The central processing unit is configured to call and run a computer program stored in the memory to execute the steps of the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that It stores a computer program implemented according to the method described in any one of claims 1 to 7 in the form of computer-readable instructions, and when the computer program is called and executed by a computer, the steps included in the corresponding method are executed.

Images