A method for generating a search ranking model, a ranking display method, an apparatus and equipment

Abstract

This invention discloses a search ranking model generation method, ranking display method, apparatus, and device. The method includes: acquiring multiple historical input items entered by a user in a search box, a list of historical suggestion words corresponding to each historical input item, and historical search items determined by the user in the historical suggestion word list; forming positional bias-eliminating sample data based on the historical input items, the historical suggestion word list, and the historical search items; and training a deep learning model based on the positional bias-eliminating sample data, using a ranking loss function as the optimization objective, to obtain a search ranking model. This method, by focusing on positional bias information in the sample data, can generate a search ranking model that better conforms to relevance ranking, thereby facilitating the obtaining of suggestion word ranking results that better meet user needs through this search ranking model.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a method for generating a search ranking model, a ranking display method, an apparatus, and a device. Background Technology

[0002] During a search, the system can capture the user's input intent based on limited input and provide suggestions that meet the user's needs. These suggestions can help users complete the input process before searching quickly and efficiently.

[0003] In existing technologies, search suggestions are generated by mining user behavior log information. However, suggestions in behavior log information are mixed with users' subjective biases regarding location. If suggestions are determined directly based on behavior log information, the positional bias in the training data cannot be eliminated, resulting in poor suggestion ranking and failing to meet users' search needs. Summary of the Invention

[0004] This invention provides a search ranking model generation method, ranking display method, apparatus, and device. By focusing on the positional deviation information in sample data, a search ranking model that better conforms to relevance ranking can be generated, thereby facilitating the obtaining of prompt word ranking results that better meet user needs through this search ranking model.

[0005] In a first aspect, embodiments of the present invention provide a method for generating a search ranking model, the method comprising:

[0006] Get multiple historical input items entered by the user in the search box, a list of historical suggestion words corresponding to each historical input item, and historical search items determined by the user in the list of historical suggestion words;

[0007] Based on historical input items, historical suggestion lists, and historical search items, sample data for eliminating positional bias is generated;

[0008] Based on the sample data to eliminate positional bias, a deep learning model is trained with the ranking loss function as the optimization objective to obtain the search ranking model.

[0009] Secondly, embodiments of the present invention provide a search sorting and display method, the method comprising:

[0010] Get the target input item entered by the user in the search box; and in the search engine, determine one or more target suggestion words that match the target input item;

[0011] The target input item and the corresponding target prompt words are input into the search ranking model provided in any embodiment of the present invention to obtain a target prompt word list generated by ranking each target prompt word;

[0012] Search results are displayed based on a list of target keywords.

[0013] Thirdly, embodiments of the present invention provide a search ranking model generation apparatus, the apparatus comprising:

[0014] The search information acquisition module is used to acquire multiple historical input items entered by the user in the search box, a list of historical suggestion words corresponding to each historical input item, and historical search items determined by the user in the list of historical suggestion words;

[0015] The module for generating sample data to eliminate positional deviation is used to generate sample data to eliminate positional deviation based on historical input items, historical suggestion word lists, and historical search items.

[0016] The search ranking model training module is used to train a deep learning model based on sample data with the elimination of positional bias, with the ranking loss function as the optimization objective, to obtain the search ranking model.

[0017] Fourthly, embodiments of the present invention provide a search sorting and display device, the device comprising:

[0018] The target suggestion word determination module is used to obtain the target input item entered by the user in the search box; and in the search engine, determine one or more target suggestion words that match the target input item;

[0019] The target prompt word list determination module is used to input the target input item and the corresponding target prompt words into the search ranking model provided in any embodiment of the present invention to obtain a target prompt word list generated by ranking each target prompt word;

[0020] The suggestion word list display module is used to search and display based on the target suggestion word list.

[0021] Fifthly, embodiments of the present invention provide an electronic device, the electronic device comprising:

[0022] At least one processor; and

[0023] A memory that is communicatively connected to at least one processor; wherein,

[0024] The memory stores a computer program that can be executed by at least one processor, such that the at least one processor can execute the search ranking model generation method provided in any embodiment of the present invention; or, the search ranking display method of the present invention.

[0025] In a sixth aspect, embodiments of the present invention provide a computer-readable storage medium storing computer instructions, which are used to cause a processor to execute and implement the search ranking model generation method provided in any embodiment of the present invention; or, the search ranking display method of embodiments of the present invention.

[0026] The technical solution of this invention involves acquiring multiple historical input items entered by the user in the search box, a list of historical suggestion words corresponding to each historical input item, and historical search items determined by the user in the list of historical suggestion words. Based on the historical input items, the list of historical suggestion words, and the historical search items, sample data with eliminated positional bias is formed. Based on this sample data, a deep learning model is trained with a ranking loss function as the optimization objective to obtain a search ranking model. This method, by focusing on positional bias information in the sample data, can generate a search ranking model that better reflects relevance ranking, thereby facilitating the acquisition of suggestion word ranking results that better meet user needs.

[0027] It should be understood that the description in this section is not intended to identify key or important features of the embodiments of the present invention, nor is it intended to limit the scope of the embodiments of the present invention. Other features of the embodiments of the present invention will become readily apparent from the following description. Attached Figure Description

[0028] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0029] Figure 1a This is a flowchart of a search ranking model generation method provided in Embodiment 1 of the present invention;

[0030] Figure 1b This is a schematic diagram of the deep learning model structure provided in Embodiment 1 of the present invention;

[0031] Figure 2a This is a flowchart of another search ranking model generation method provided in Embodiment 2 of the present invention;

[0032] Figure 2b This is a schematic diagram of the prompt word list label provided in Embodiment 2 of the present invention for eliminating positional deviation sample data;

[0033] Figure 2c This is a schematic diagram of the word2vec model training process provided in Embodiment 2 of the present invention;

[0034] Figure 3 This is a flowchart of another search ranking model generation method provided in Embodiment 3 of the present invention;

[0035] Figure 4 This is a flowchart of a search sorting and display method provided in Embodiment 4 of the present invention;

[0036] Figure 5 This is a schematic diagram of a search ranking model generation device according to Embodiment 5 of the present invention;

[0037] Figure 6 This is a schematic diagram of a search sorting and display device according to Embodiment Six of the present invention;

[0038] Figure 7 This is a schematic diagram of the structure of an electronic device that implements the search ranking model generation and search ranking display methods of this invention. Detailed Implementation

[0039] To enable those skilled in the art to better understand the embodiments of the present invention, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0040] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0041] Example 1

[0042] Figure 1aThis is a flowchart of a search ranking model generation method provided in Embodiment 1 of the present invention. This embodiment can be used to rank multiple prompts related to the input item by using the generated search ranking model during a search. The method can be executed by a search ranking model generation device, which can be implemented in hardware and / or software. The device can be configured in an electronic device, such as a laptop or desktop computer.

[0043] like Figure 1a As shown in the figure, the search ranking model generation method provided in this embodiment includes the following steps:

[0044] Step 110: Obtain multiple historical input items entered by the user in the search box, a list of historical suggestion words corresponding to each historical input item, and historical search items determined by the user in the list of historical suggestion words.

[0045] Specifically, during a user's use of a search engine, the search engine records user data and generates a search log. For example, the search engine forms a search log based on the content the user has entered in the search box, the content displayed to the user, and the content the user selects from the displayed content. The content the user has entered in the search box can be input items. The content displayed to the user can be a list of suggestion words, which can consist of multiple suggestion words. The content the user selects from the displayed content can be search terms. From the search log over a period of time, multiple historical input items entered by the user in the search box, a historical suggestion word list corresponding to each historical input item, and historical search terms selected by the user from the historical suggestion word list can be obtained.

[0046] The suggestion words can be single words, phrases, or sentences. The suggestion word list can contain multiple suggestions, such as 10. Search logs over a period of time can be automatically recorded and generated by the search engine, such as 50 days of search logs.

[0047] In this embodiment, because the user's input, the search engine's returned results (the content displayed to the user), and the user's interactive actions on the results (the user's choices within the displayed content) respond to different interfaces of the search engine, they cannot be directly recorded as a single log entry. Therefore, it is necessary to associate the user's input, the search engine's returned results, and the user's interactive actions on the results to form a complete search log. Specifically, a unified session ID can be set for the search logs formed by input and returned results and the search logs formed by interactive actions. The necessary information from the two logs can be associated through the session ID. In this way, when mining the logs, it is possible to completely obtain multiple historical input items entered by the user in the search box, the historical suggestion list corresponding to each historical input item, and the historical search items determined by the user in the historical suggestion list.

[0048] Step 120: Based on historical input items, historical suggestion word list, and historical search items, generate sample data to eliminate positional bias.

[0049] In this embodiment, when a user uses a search engine, they select a search term from the suggestion list based on their subjective intention. This search term is not necessarily the most relevant to the user's target search content. For example, the first word in the suggestion list may not be the most relevant word to the user's target search content. When a user subjectively selects the first word as a search term, this search term creates a positional bias compared to the truly most relevant word. Therefore, historical input items, the historical suggestion list corresponding to the historical input items, and the historical search terms selected from the suggestion list are combined to form sample data to eliminate positional bias.

[0050] Step 130: Based on the sample data for eliminating positional bias, train a deep learning model with the ranking loss function as the optimization objective to obtain the search ranking model.

[0051] Typically, deep learning models used for search ranking are single relevance models and do not take into account positional bias.

[0052] In this embodiment, the deep learning model used for search ranking can be configured as two parts, enabling the model to consider both the relevance between historical input items and suggestions in the historical suggestion list, and positional bias. The deep learning model's network includes a long short-term memory network, at least one dense network, at least one fully connected layer, and at least one dropout layer. The ranking loss function can use pairwise, listwise, or pointwise ranking loss functions. By combining these two parts and using the ranking loss function as the optimization objective, a search ranking model that better conforms to relevance ranking is obtained, i.e., the search ranking model.

[0053] For example, Figure 1b This is a schematic diagram of the deep learning model structure provided in Embodiment 1 of the present invention. Figure 1b As shown, field-1 to field-m are relevance network units, and the position part is a positional bias network unit. In this embodiment, the positional bias-eliminated sample data can be used as the input to both the relevance network unit and the positional bias network unit; based on the positional bias-eliminated sample data... Figure 1b The deep learning model shown is trained to generate a search ranking model. Therefore, when using the search ranking model to generate a list of prompt words in practical applications, positional bias can be eliminated, and a prompt word ranking result that better suits the user's needs can be obtained. Among them, the relevance network unit can include three single-layer dense networks (Dense), one long short-term memory network (LSTM), two 256-layer dense networks (Dense(256)), two random dropout layers, and two fully connected layers (Dense(1)). The positional bias network unit can include a 128-layer dense network (Dense(123)), a 256-layer dense network, a dropout layer, and a fully connected layer. Finally, the result of the relevance network unit is multiplied with the result of the positional bias network unit to obtain the output of the search ranking model. Among them, ReLU and sigmoid are activation functions. The dropout layer can be used to remove training units of the neural network from the network with a certain probability during the deep learning training process, thereby achieving the effect of preventing overfitting.

[0054] The technical solution of this embodiment obtains multiple historical input items entered by the user in the search box, a list of historical suggestion words corresponding to each historical input item, and historical search items determined by the user in the list of historical suggestion words. Based on the historical input items, the list of historical suggestion words, and the historical search items, sample data for eliminating positional bias is formed. Based on the sample data for eliminating positional bias, a deep learning model is trained with a ranking loss function as the optimization objective to obtain a search ranking model. This method, by focusing on the positional bias information in the sample data, can generate a search ranking model that better conforms to relevance ranking, thereby facilitating the obtaining of suggestion word ranking results that better meet user needs through this search ranking model.

[0055] Example 2

[0056] Figure 2a This is a flowchart of another search ranking model generation method provided by Embodiment 2 of the present invention. The technical solutions of the embodiments of the present invention are further refinements based on the technical solutions of any of the above embodiments. The technical solutions of the embodiments of the present invention can be combined with various optional implementation methods in any of the above embodiments.

[0057] Specifically, in an optional embodiment of the present invention, sample data for eliminating positional bias is formed based on historical input items, a list of historical suggestion words, and historical search items, including:

[0058] Based on historical search terms, tags are set for each suggestion in the historical suggestion list to obtain suggestion tags for the historical suggestion list;

[0059] Based on the historical input items, the historical prompt word list, and the prompt word labels, as well as the correspondence among the historical input items, the historical prompt word list, and the prompt word labels, sample data for eliminating positional bias is formed.

[0060] In an optional embodiment of the present invention, sample data for eliminating positional bias is formed based on historical input items, a list of historical prompt words, and historical search items, including:

[0061] Obtain multiple associated input items corresponding to each historical input item, and input the historical input items and their corresponding multiple associated input items into the preset word vector model in sequence to obtain the corresponding input item vectors;

[0062] Based on each input item vector, determine the similarity between each historical input item and its corresponding multiple associated input items;

[0063] Similarity is used as a contextual semantic feature, and this contextual semantic feature is added to the sample data to eliminate positional bias.

[0064] In an optional embodiment of the present invention, before obtaining multiple associated input items corresponding to each historical input item, and sequentially inputting the historical input items and the corresponding multiple associated input items into a preset word vector model, the method further includes:

[0065] Obtain the context information corresponding to each historical input item, determine the time interval information between each search item in the context information, and split the context information according to the time interval information to obtain multiple corpus information;

[0066] If the corpus information does not meet the preset corpus limitation conditions, the time interval information between each search item in the corpus information is determined, and the corpus information is split according to the time interval information until all the obtained corpus information meets the preset corpus limitation conditions.

[0067] If all the information in the corpus meets the preset corpus constraints, then a preset word vector model is generated based on the information in the corpus.

[0068] In an optional embodiment of the present invention, sample data for eliminating positional bias is formed based on historical input items, a list of historical prompt words, and historical search items, including:

[0069] Determine the data length corresponding to each historical input item and each suggestion word in the historical suggestion word list;

[0070] Get the number of clicks a user makes on each suggestion in the historical suggestion list within a preset time period;

[0071] Based on the data length and click volume, a search length feature is generated and added to the sample data to eliminate positional bias.

[0072] like Figure 2a As shown, another search ranking model generation method provided in this embodiment includes the following steps:

[0073] Step 201: Obtain multiple historical input items entered by the user in the search box, a list of historical suggestion words corresponding to each historical input item, and historical search items determined by the user in the list of historical suggestion words.

[0074] Step 202: Based on the historical search terms, set tags for each suggestion in the historical suggestion list to obtain the suggestion tag of the historical suggestion list.

[0075] In this embodiment, 'q' can represent a historical input item, 'd' can represent the corresponding historical suggestion word list, and 'l' can represent a set label. The label for each suggestion word in the historical suggestion word list can be set to 0 or 1. Specifically, the suggestion word selected by the user in the suggestion word list, i.e., the suggestion word corresponding to the historical search item, is labeled as 1. Other suggestion words are labeled as 0. The process of setting suggestion word labels can be represented by the following formula:

[0076]

[0077] Where, d i It can represent the i-th prompt word in the prompt word list. i This can be represented as the label for the i-th cue word. q t This can be represented as a search term.

[0078] The advantage of this setup is that it clearly represents the relationship between the input item, the list of suggestions corresponding to the input item, and the suggestion labels in the list of suggestions, and it can also mark the location of the search item.

[0079] Step 203: Based on the historical input items, the historical prompt word list, and the prompt word labels, as well as the correspondence among the historical input items, the historical prompt word list, and the prompt word labels, form sample data for eliminating positional deviation.

[0080] Specifically, a historical input item, a corresponding list of prompt words, and a corresponding set of labels can constitute a sample data set for eliminating positional bias. A sample data set for eliminating positional bias can then be represented by t, where t =<q,d,l> . Figure 2b This is a schematic diagram of the prompt word list label provided in Embodiment 2 of the present invention for eliminating positional deviation sample data. For example... Figure 2b As shown, q can represent a historical input item. The leftmost d1 to d10 in the diagram represent the sequence numbers of each prompt in the prompt list from top to bottom. The rightmost 1 and 0 represent the labels of each prompt in the prompt list. 1 represents the label of the prompt selected by the user in the prompt list, i.e., the label of the prompt corresponding to the historical search item; 0 represents the label of other unselected prompts. For example, if the input q is "dat", then multiple words or phrases starting with "dat" in the diagram are prompts, and these multiple prompts constitute the prompt list. When the user selects "datagrand" as the search item, the label of this prompt is set to 1, and the labels of other prompts are set to 0. The positional bias elimination sample dataset can be composed of multiple positional bias elimination sample data t, which can be represented as T, where T = [t1, t2, t3, ..., t n ], where n represents the number of sample data to eliminate positional bias.

[0081] Furthermore, since d and l contain multiple data points, q can be mapped to each data point in d and l to form an expanded sample dataset with eliminated positional bias, which can be represented by S.

[0082] Step 204: Obtain the context information corresponding to each historical input item, determine the time interval information between each search item in the context information, and split the context information according to the time interval information to obtain multiple corpus information.

[0083] In this embodiment, the context information can also be obtained from the search logs mentioned above. The context information corresponding to the historical input item can consist of the search items before and after the historical input item, that is, it can be text composed of multiple search items. The time interval can be the duration between two user searches.

[0084] Specifically, within the contextual information, the time intervals of each search item may be the same or different, allowing the contextual text to be segmented by time interval. For example, the maximum time interval can be used for segmentation, thus obtaining multiple segments of corpus information. The maximum time interval is 1 hour.

[0085] Step 205: If the corpus information does not meet the preset corpus limitation conditions, determine the time interval information between each search item in the corpus information, and split the corpus information according to the time interval information until all the obtained corpus information meets the preset corpus limitation conditions.

[0086] In this embodiment of the invention, the preset corpus limitation condition can be to limit the length of the corpus information. For example, the length of the corpus information should not exceed 20 search terms. Splitting can be to further divide the corpus information whose length exceeds the corpus limitation condition.

[0087] Specifically, in step 204, dividing the context information by the maximum time interval may result in some corpus information being too long. Excessively long corpus information can be further divided by selecting different time intervals. Each selected time interval can be the current maximum time interval between search items in the corpus information, until all the divided corpus information meets the preset corpus constraints.

[0088] Step 206: If all the information in the corpus meets the preset corpus constraints, then train and generate a preset word vector model based on the information in the corpus.

[0089] In this embodiment, the preset word vector model can be the word2vec model or the FastText model.

[0090] For example, a word2vec model can be trained using corpus information that meets preset corpus constraints. The Genism library can be used to build the word2vec model. The word2vec model is trained using the continuous bag-of-words (CBOW) algorithm. The Genism library can read the segmented corpus information and encode it into multiple basic vectors. These basic vectors are then input into the CBOW algorithm, and through continuous training, a word2vec model with semantic weights conforming to the corpus information is obtained.

[0091] Gensim is a commonly used tool library in natural language processing. It is mainly used to learn topic vector representations of text hidden layers from raw unstructured text in an unsupervised manner.

[0092] Figure 2c This is a schematic diagram of the word2vec model training process provided according to Embodiment 2 of the present invention. Figure 2cAs shown in the embodiment of the present invention, multiple corpus information are obtained by dividing the context information according to the maximum time interval information; if the corpus information does not meet the preset corpus limitation conditions, the time interval information between each search item in the corpus information is determined, and the corpus information is divided again according to the maximum time interval information until all the obtained corpus information meets the preset corpus limitation conditions; finally, a word2vec model is trained and generated based on each corpus information.

[0093] The advantage of this setup is that by dividing the contextual information into corpora by the maximum time interval, search terms with shorter time intervals can be grouped into a corpus, thereby increasing the relevance between search terms in each corpus and thus enabling the training of a better word2vec model.

[0094] Step 207: Obtain multiple associated input items corresponding to each historical input item, and input the historical input items and their corresponding multiple associated input items into the preset word vector model in sequence to obtain the corresponding input item vectors.

[0095] In this embodiment, the multiple associated input items corresponding to each historical input item can be several input items preceding each historical input item. The input item vector can be the encoded vector obtained by encoding the historical input item and its corresponding multiple associated input items through a pre-built word vector model. For example, the 64 input items preceding each historical input item can be used as the corresponding multiple associated input items, and each historical input item and its corresponding 64 associated input items can be input into the trained word2vec model to obtain the vectors of each historical input item and the corresponding 64 associated encoded vectors.

[0096] Step 208: Based on each input item vector, determine the similarity between each historical input item and its corresponding multiple associated input items;

[0097] Specifically, the similarity between each historical input vector and its corresponding 64 associated input vectors can be calculated. Examples include cosine similarity and distance similarity. Each historical input can correspond to 64 cosine similarity scores.

[0098] Step 209: Use similarity as a contextual semantic feature and add the contextual semantic feature to the sample data to eliminate positional bias.

[0099] Specifically, step 208 yields 64 cosine similarities corresponding to each historical input item. These cosine similarities can be used as a set of 64-dimensional contextual semantic features. Multiple sets of contextual semantic features corresponding to historical input items are added to the position-bias-reduced sample data as part of the position-bias-reduced sample dataset. This supplements the sample data with relevance information between historical input items, thereby improving the model's ability to rank relevance.

[0100] Step 210: Determine the data length corresponding to each prompt word in the historical input item and historical prompt word list.

[0101] In this embodiment, the data length can be the number of characters. For example, the number of characters contained in a historical input item. The length of the historical input item, the length of each suggestion word in the historical suggestion word list, and the length of the historical search item can be determined based on the historical input items and the historical suggestion word list.

[0102] Furthermore, after determining the data length, the ratio of the length of each suggestion word in the historical suggestion word list to the length of the corresponding historical input content is determined, as well as the ratio of the length of each suggestion word in the historical suggestion word list to the average length of the historical search terms is determined.

[0103] Step 211: Obtain the number of clicks by the user on each suggestion word in the historical suggestion word list within a preset time period.

[0104] In this embodiment, the number of times a user clicks on each suggestion word in the historical suggestion word list each day within a preset time period can be used as the daily click count. For example, the preset time period can be 50 days.

[0105] Step 212: Based on the data length and click volume, generate a search length feature and add the search length feature to the sample data for eliminating positional bias.

[0106] Specifically, the search length feature is defined as follows: the length of historical input items, the length of each suggestion word in the historical suggestion word list, the ratio of the length of each suggestion word in the historical suggestion word list to the corresponding length of the historical input item, the ratio of the length of each suggestion word in the historical suggestion word list to the average length of historical search items, and the click count of each suggestion word in the historical suggestion word list. This search length feature is then added to the positional bias elimination sample data as part of the positional bias elimination sample dataset. This supplements the positional bias elimination sample data with information on the correlation between the data lengths of input items and suggestion words.

[0107] Step 213: Based on the sample data for eliminating positional bias, train a deep learning model with the ranking loss function as the optimization objective to obtain the search ranking model.

[0108] In this embodiment of the invention, the positional bias sample data eliminated through the above steps includes 64-dimensional contextual semantic features and a 5-dimensional search length feature. These two together constitute 69-dimensional feature information, which, combined with S, forms the training features. The training features are input into a preset deep learning model, and the model is trained through forward propagation, target loss function optimization, and then backpropagation. An evaluation metric is calculated at the end of each round; training is complete when the evaluation metric exceeds the expected value, resulting in a search ranking model. The 69-dimensional feature information can be used as input to the relevance network unit, and S can be used as input to the positional bias network unit.

[0109] For example, the pairwise sorting loss function can be used to train the model, with an initial learning rate of 0.001, a batch size of 64, and an epoch count of 200.

[0110] The technical solution of this embodiment obtains multiple historical input items entered by the user in the search box, a list of historical suggestion words corresponding to each historical input item, and historical search items determined by the user in the list of historical suggestion words; based on the historical search items, labels are set for each suggestion word in the list of historical suggestion words to obtain suggestion word labels for the list of historical suggestion words; based on the correspondence between historical input items, the list of historical suggestion words, and suggestion word labels, and the correspondence between the three, sample data for eliminating positional deviation is formed; contextual information corresponding to each historical input item is obtained; and a semantic analysis model is trained using the contextual information to obtain a contextual semantic analysis model. Next, based on the contextual semantic analysis model, the similarity between historical input items and multiple preset related input items corresponding to the historical input items is determined; the similarity is used as a contextual semantic feature and added to the sample data for eliminating positional bias; the data length corresponding to each historical input item and each prompt word in the historical prompt word list is determined; the number of clicks by the user on each prompt word in the historical prompt word list within a preset time period is obtained; based on the data length and the number of clicks, a search length feature is formed and added to the sample data for eliminating positional bias; based on the sample data for eliminating positional bias, a deep learning model is trained with the ranking loss function as the optimization objective to obtain the search ranking model.

[0111] In summary, the technical solution of this invention associates user input, search engine results, and user interactions with those results to form a search log. Multiple historical input items, a list of historical suggestion words corresponding to each historical input item, and historical search terms determined by the user within the historical suggestion word list are obtained from the search log. Contextual semantic features and search length features are generated based on the historical input items, the historical suggestion word list, and the historical search terms. These features are then added to the positional bias elimination sample data to form a complete positional bias elimination sample dataset. The positional bias elimination sample dataset is then used to train a deep learning model. Through forward propagation, target loss function optimization, and backpropagation, a search ranking model is generated when the model's evaluation metric exceeds the expected value. Finally, the search ranking model is used to rank search suggestion words. After a user inputs information into the search engine, the search engine can recall coarse-ranked results. The search ranking model further ranks these coarse-ranked results to obtain a more relevant suggestion word list, which is then displayed to the user, completing the search suggestion word ranking process.

[0112] The acquisition, storage, and application of historical search content involved in the technical solutions of this invention comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0113] Example 3

[0114] Figure 3 This is a flowchart of another search ranking model generation method provided by Embodiment 3 of the present invention. The technical solutions of the embodiments of the present invention are further refinements based on the technical solutions of any of the above embodiments. The technical solutions of the embodiments of the present invention can be combined with various optional implementation methods in any of the above embodiments.

[0115] Specifically, in one optional embodiment of the present invention, a deep learning model is trained based on the sample data for eliminating positional bias, with the ranking loss function as the optimization objective, to obtain a search ranking model, including:

[0116] The ranking loss function is determined based on the number of samples corresponding to the sample data for eliminating positional bias, sample ranking information, sample position information, prompt word labels, sample weight information, and the ranking score output by the search ranking model for each sample.

[0117] Based on the sample data to eliminate positional bias, a deep learning model is trained with the ranking loss function as the optimization objective to obtain the search ranking model.

[0118] like Figure 3 As shown, the third embodiment of this method for generating a search ranking model specifically includes the following steps.

[0119] Step 310: Obtain multiple historical input items entered by the user in the search box, a list of historical suggestion words corresponding to each historical input item, and historical search items determined by the user in the list of historical suggestion words;

[0120] Step 320: Based on historical input items, historical suggestion word list, and historical search items, generate sample data to eliminate positional bias;

[0121] Step 330: Determine the ranking loss function based on the number of samples corresponding to the sample data for eliminating positional bias, sample ranking information, sample position information, prompt word labels, sample weight information, and the ranking score output by the search ranking model for each sample.

[0122] In this embodiment of the invention, the target loss function can be a classic ranking loss function. For example, the target loss function can be composed of the classic PairWise loss function plus normalized discounted cumulative gain (nDCG) information. The PairWise loss function can be composed of cross-entropy and sample weights. The expression for the target loss function can be expressed as:

[0123]

[0124] Where p,n represents a single prompt word sample in the sample dataset S for eliminating positional bias; |S| can be represented as the number of samples corresponding to the sample dataset for eliminating positional bias; P p,n Let |ΔnDCG| represent the probability that sample p is ranked before n; |ΔnDCG| represents the absolute value of the change in nDCG caused by swapping the ranking order of samples p and n; w p,n This represents sample weight information, which is also a measure of sample importance.

[0125] Among them, P p,n The expression for can be represented by the following formula:

[0126]

[0127] Where f p and f n These are the ranking scores for sample p and sample n, respectively, in the search ranking model.

[0128] The expression for |ΔnDCG| can be represented by the following formula:

[0129]

[0130] In this embodiment of the invention, paired samples f can be selected. p >f nIf p is listed before n, and p is the clicked suggestion word, then rel p =1, the expression for |ΔnDCG| can be simplified to the following formula:

[0131]

[0132] Here, rel can represent the cue word label, and rank can represent the sample location information.

[0133] Combining the above formulas, the loss function formula can also be expressed as:

[0134]

[0135] Step 340: Based on the sample data for eliminating positional bias, train a deep learning model with the ranking loss function as the optimization objective to obtain the search ranking model.

[0136] The technical solution of this embodiment obtains multiple historical input items entered by the user in the search box, a list of historical suggestion words corresponding to each historical input item, and historical search items determined by the user in the list of historical suggestion words. Based on the historical input items, the list of historical suggestion words, and the historical search items, positional bias elimination sample data is formed. Based on the number of samples, sample ranking information, sample position information, suggestion word labels, sample weight information, and the ranking score output by the search ranking model for each sample, a ranking loss function is determined. Based on the positional bias elimination sample data, a deep learning model is trained with the ranking loss function as the optimization objective to obtain the search ranking model. This method optimizes the deep learning model by determining the target loss function, resulting in a search ranking model that better conforms to relevance ranking.

[0137] The acquisition, storage, and application of historical search content involved in the technical solutions of this invention comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0138] Example 4

[0139] Figure 4 This is a flowchart of a search ranking and display method according to Embodiment 4 of the present invention. This embodiment can be applied to situations where a user enters content in a search engine, and a ranked list of search suggestions is generated and displayed to the user based on the user's input. This method can be executed by a search ranking and display device, which can be implemented in hardware and / or software. The device can be configured in an electronic device, such as a laptop or desktop computer.

[0140] like Figure 4As shown in the figure, the search sorting and display method provided in this embodiment four specifically includes the following steps:

[0141] Step 410: Obtain the target input item entered by the user in the search box; and in the search engine, determine one or more target suggestion words that match the target input item.

[0142] In this embodiment, relevant knowledge data can be imported into the search engine to form a prefix tree. The prefix tree can then be used to retrieve one or more target suggestion words that match the target input. The knowledge data can consist of a large number of characters, words, phrases, and sentences. Words can be composed of characters, which can then be combined into phrases and sentences. A prefix tree is a multi-way tree data structure used for fast retrieval; each node can represent a character, and it can be used to retrieve words and sentences with a common character prefix.

[0143] Among them, the relevant knowledge data can be enterprise knowledge data, such as data consisting of a large number of enterprise names, enterprise profiles, enterprise news and other information.

[0144] Step 420: Input the target input items and the corresponding target prompt words into the search ranking model provided in any embodiment of the present invention to obtain a target prompt word list generated by ranking each target prompt word.

[0145] Specifically, the search ranking model provided in any embodiment of the present invention can sort a list of target prompt words from the corresponding target prompt words according to the target input item.

[0146] This search ranking model can better capture contextual semantic information and eliminate positional bias in the training data, thereby obtaining more realistic relevance information from the target prompts and ranking a list of prompts that better meet user requirements.

[0147] Step 430: Display the search results based on the target keyword list.

[0148] Specifically, the search engine can obtain a list of target suggestion words and select the first preset number of suggestion words from the list to display to the user for selection.

[0149] The technical solution of this embodiment involves obtaining the target input item entered by the user in the search box; determining one or more target suggestion words matching the target input item in the search engine; inputting the target input item and the corresponding target suggestion words into the search ranking model of any embodiment of the present invention to obtain a target suggestion word list generated by ranking the target suggestion words; and displaying the search results based on the target suggestion word list. This method, through the search ranking model in any of the above embodiments, provides users with a list of suggestion words that are more relevant to the user's search content and conform to the relevance ranking.

[0150] Example 5

[0151] Figure 5 This is a schematic diagram of a search ranking model generation device according to Embodiment 5 of the present invention. Figure 5 As shown, the device includes: a search information acquisition module 501, a sample data generation module for eliminating positional bias 502, and a search ranking model training module 503, wherein:

[0152] The search information acquisition module 501 is used to acquire multiple historical input items entered by the user in the search box, a list of historical suggestion words corresponding to each historical input item, and historical search items determined by the user in the list of historical suggestion words;

[0153] The position deviation elimination sample data generation module 502 is used to generate position deviation elimination sample data based on historical input items, historical prompt word list, and historical search items;

[0154] The search ranking model training module 503 is used to train a deep learning model based on the sample data for eliminating positional bias, with the ranking loss function as the optimization objective, to obtain the search ranking model.

[0155] Optionally, the sample data generation module 502 for eliminating positional bias includes:

[0156] The suggestion word tag generation unit is used to set tags for each suggestion word in the historical suggestion word list based on historical search terms, so as to obtain the suggestion word tags of the historical suggestion word list;

[0157] The position deviation elimination sample data generation unit is used to generate position deviation elimination sample data based on the historical input items, the historical prompt word list, and the prompt word label, as well as the correspondence between the three.

[0158] Optionally, the sample data generation module 502 for eliminating positional bias includes:

[0159] The input item vector generation unit is used to obtain multiple associated input items corresponding to each historical input item, and input the historical input items and their corresponding multiple associated input items into the preset word vector model in sequence to obtain the corresponding input item vectors.

[0160] The similarity generation unit is used to determine the similarity between each historical input item and its corresponding multiple associated input items based on each input item vector;

[0161] The context semantic feature generation unit is used to use similarity as a context semantic feature and add the context semantic feature to the sample data to eliminate positional bias.

[0162] Optionally, the device may also include:

[0163] The context information splitting module is used to obtain the context information corresponding to each historical input item before obtaining multiple associated input items corresponding to each historical input item and inputting the historical input items and their corresponding multiple associated input items into the preset word vector model in sequence. It determines the time interval information between each search item in the context information and splits the context information according to the time interval information to obtain multiple corpus information.

[0164] The corpus information splitting module is used to determine the time interval information between each search item in the corpus information if the corpus information does not meet the preset corpus limitation conditions, and to split the corpus information according to the time interval information until all the obtained corpus information meets the preset corpus limitation conditions.

[0165] The pre-built word vector model generation module is used to train and generate a pre-built word vector model based on each piece of corpus information if all the corpus information meets the pre-defined corpus constraints.

[0166] Optionally, the sample data generation module 502 for eliminating positional bias includes:

[0167] The data length determination unit is used to determine the data length corresponding to each historical input item and each prompt word in the historical prompt word list;

[0168] The click volume acquisition unit is used to acquire the number of clicks a user makes on each suggestion word in the historical suggestion word list within a preset time period.

[0169] The search length feature generation unit is used to generate search length features based on data length and click volume, and add the search length features to the sample data to eliminate positional bias.

[0170] Optional, the search ranking model training module 503 includes:

[0171] The loss function determination unit is used to determine the ranking loss function based on the number of samples corresponding to the sample data for eliminating positional bias, sample ranking information, sample position information, prompt word labels, sample weight information, and the ranking score output by the search ranking model for each sample.

[0172] The search ranking model training unit is used to train a deep learning model based on sample data with the positional bias eliminated, with the ranking loss function as the optimization objective, to obtain the search ranking model.

[0173] The technical solution of this embodiment obtains multiple historical input items entered by the user in the search box, a list of historical suggestion words corresponding to each historical input item, and historical search items determined by the user in the list of historical suggestion words. Based on the historical input items, the list of historical suggestion words, and the historical search items, sample data for eliminating positional bias is formed. Based on the sample data for eliminating positional bias, a deep learning model is trained with a ranking loss function as the optimization objective to obtain a search ranking model. This method, by focusing on the positional bias information in the sample data, can generate a search ranking model that better conforms to relevance ranking, thereby facilitating the obtaining of suggestion word ranking results that better meet user needs through this search ranking model.

[0174] The search ranking model generation device of the present invention can execute the search ranking model generation method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0175] Example 6

[0176] Figure 6 This is a schematic diagram of a search sorting and display device according to Embodiment Six of the present invention. Figure 6 As shown, the device includes: a target prompt word acquisition module 601, a target prompt word sorting module, and a prompt word list display module 603, wherein:

[0177] The target suggestion word determination module 601 is used to obtain the target input item entered by the user in the search box; and determine one or more target suggestion words that match the target input item in the search engine;

[0178] The target prompt word list determination module 602 is used to input the target input item and the corresponding target prompt words into the search ranking model of any embodiment of the present invention to obtain a target prompt word list generated by sorting the target prompt words;

[0179] The prompt word list display module 603 is used to search and display based on the target prompt word list.

[0180] The technical solution of this embodiment involves obtaining the target input item entered by the user in the search box; determining one or more target suggestion words matching the target input item in the search engine; inputting the target input item and the corresponding target suggestion words into the search ranking model of any embodiment of the present invention to obtain a target suggestion word list generated by ranking the target suggestion words; and displaying the search results based on the target suggestion word list. This method, through the search ranking model in any of the above embodiments, provides users with a list of suggestion words that are more relevant to the user's search content and conform to the relevance ranking.

[0181] The search sorting and display device provided in this embodiment of the invention can execute the search sorting and display method provided in Embodiment 3 of the invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0182] Example 7

[0183] Figure 7 This is a schematic diagram of an electronic device provided for an embodiment of the present invention. The device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices (e.g., helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the embodiments of the invention described and / or claimed herein.

[0184] like Figure 7 As shown, the electronic device 10 includes at least one processor 11 and a memory, such as a read-only memory (ROM) 12 or a random access memory (RAM) 13, communicatively connected to the at least one processor 11. The memory stores computer programs executable by the at least one processor. The processor 11 can perform various appropriate actions and processes based on the computer program stored in the ROM 12 or loaded from storage unit 18 into the RAM 13. The RAM 13 may also store various programs and data required for the operation of the electronic device 10. The processor 11, ROM 12, and RAM 13 are interconnected via a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0185] Multiple components in electronic device 10 are connected to I / O interface 15, including: input unit 16, such as keyboard, mouse, etc.; output unit 17, such as various types of displays, speakers, etc.; storage unit 18, such as disk, optical disk, etc.; and communication unit 19, such as network card, modem, wireless transceiver, etc. Communication unit 19 allows electronic device 10 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0186] Processor 11 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. Processor 11 performs the various methods and processes described above, such as search ranking model generation methods.

[0187] In some embodiments, the search ranking model generation method may be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed on electronic device 10 via ROM 12 and / or communication unit 19. When the computer program is loaded into RAM 13 and executed by processor 11, one or more steps of the search ranking model generation method described above may be performed. Alternatively, in other embodiments, processor 11 may be configured to perform the search ranking model generation method by any other suitable means (e.g., by means of firmware).

[0188] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0189] Computer programs for implementing the methods of embodiments of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The computer programs may be executed entirely on a machine, partially on a machine, or as a standalone software package, partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0190] In the context of embodiments of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0191] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0192] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or computing systems that include middleware components (e.g., application servers), or computing systems that include frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0193] A computing system can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system to address the shortcomings of traditional physical hosts and VPS services, such as high management difficulty and weak business scalability.

[0194] It should be understood that the various forms of processes shown above can be used to reorder, add, or delete steps. For example, the steps described in the embodiments of the present invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of the embodiments of the present invention can be achieved, and this document does not impose any restrictions.

[0195] The specific embodiments described above do not constitute a limitation on the scope of protection of the embodiments of the present invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the embodiments of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for generating a search ranking model, characterized in that, include: Obtain multiple historical input items entered by the user in the search box, a list of historical suggestion words corresponding to each historical input item, and historical search items determined by the user in the list of historical suggestion words; Based on the historical input items, the historical suggestion word list, and the historical search items, sample data for eliminating positional deviation is formed; Based on the sample data for eliminating positional bias, a deep learning model is trained with the ranking loss function as the optimization objective to obtain a search ranking model. The step of forming sample data to eliminate positional bias based on the historical input items, the historical suggestion word list, and the historical search items includes: Based on the historical search terms, tags are set for each suggestion in the historical suggestion list to obtain suggestion tags for the historical suggestion list; wherein, the suggestion tags for the historical suggestion list are used to distinguish suggestion words corresponding to the historical search terms selected by the user in the historical suggestion list from suggestion words that have not been selected; Based on the historical input items, the historical prompt word list, and the prompt word tags, and the correspondence among the historical input items, the historical prompt word list, and the prompt word tags, sample data for eliminating positional deviation is formed; wherein, the sample data for eliminating positional deviation is <historical input items, historical prompt word list, prompt word tags>.

2. The method according to claim 1, characterized in that, Based on the historical input items, the historical suggestion list, and the historical search items, sample data for eliminating positional bias is generated, including: Obtain multiple associated input items corresponding to each of the historical input items, and input the historical input items and the multiple associated input items into a preset word vector model in sequence to obtain the corresponding input item vectors; Based on each input item vector, determine the similarity between each historical input item and its corresponding multiple associated input items; The similarity is used as a contextual semantic feature, and the contextual semantic feature is added to the sample data with the positional deviation eliminated.

3. The method according to claim 2, characterized in that, Before acquiring multiple associated input items corresponding to each of the historical input items, and sequentially inputting the historical input items and their corresponding multiple associated input items into a preset word vector model, the method further includes: Obtain context information corresponding to each of the historical input items, determine the time interval information between each search item in the context information, and split the context information according to the time interval information to obtain multiple corpus information; If the corpus information does not meet the preset corpus limitation conditions, then the time interval information between each search item in the corpus information is determined, and the corpus information is split according to the time interval information until all the obtained corpus information meets the preset corpus limitation conditions. If all the corpus information meets the preset corpus constraints, then a preset word vector model is generated by training based on the corpus information.

4. The method according to claim 1, characterized in that, Based on the historical input items, the historical suggestion list, and the historical search items, sample data for eliminating positional bias is generated, including: Determine the data length corresponding to the historical input items and each prompt word in the historical prompt word list; Get the number of clicks a user makes on each suggestion word in the historical suggestion word list within a preset time period; Based on the data length and the number of clicks, a search length feature is formed, and the search length feature is added to the sample data for eliminating positional deviation.

5. The method according to claim 1, characterized in that, Based on the sample data for eliminating positional bias, a deep learning model is trained with the ranking loss function as the optimization objective to obtain a search ranking model, including: The ranking loss function is determined based on the number of samples, sample ranking information, sample position information, prompt word labels, sample weight information, and the ranking score output by the search ranking model for each sample in the sample data corresponding to the elimination of positional bias. Based on the sample data for eliminating positional bias, a deep learning model is trained using the ranking loss function as the optimization objective to obtain a search ranking model.

6. A search sorting and display method, characterized in that, include: Get the target input item entered by the user in the search box; And in the search engine, identify one or more target suggestion words that match the target input item; The target input item and the corresponding target prompt words are input into the search ranking model as described in any one of claims 1-5 to obtain a target prompt word list generated by sorting the target prompt words. Search results will be displayed based on the target suggestion list.

7. A search ranking model generation device, characterized in that, The device includes: The search information acquisition module is used to acquire multiple historical input items entered by the user in the search box, a list of historical prompt words corresponding to each historical input item, and historical search items determined by the user in the list of historical prompt words. The position deviation elimination sample data generation module is used to generate position deviation elimination sample data based on the historical input items, the historical prompt word list, and the historical search items. The search ranking model training module is used to train a deep learning model based on the eliminated position deviation sample data, with the ranking loss function as the optimization objective, to obtain the search ranking model. The sample data generation module for eliminating positional deviation includes: The suggestion word tag generation unit is used to set tags for each suggestion word in the historical suggestion word list based on historical search terms, thereby obtaining suggestion word tags for the historical suggestion word list; wherein, the suggestion word tags in the historical suggestion word list are used to distinguish suggestion words corresponding to the historical search terms selected by the user in the historical suggestion word list from suggestion words that have not been selected; The position deviation elimination sample data generation unit is used to generate position deviation elimination sample data based on historical input items, historical prompt word list, and prompt word tags, as well as the correspondence between the three; wherein, the position deviation elimination sample data is <historical input items, historical prompt word list, prompt word tags>.

8. An electronic device, the electronic device comprising: One or more processors; Storage device for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors execute the programs to implement the search ranking model generation method as described in any one of claims 1-5; or, the search ranking display method as described in claim 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the program implements the search ranking model generation method as described in any one of claims 1-5; or the search ranking display method as described in claim 6.

Images