Method, device, equipment and storage medium for sorting search results

Abstract

The present application discloses a method, apparatus, device and storage medium for sorting search results, belonging to the field of artificial intelligence technology. The method includes: obtaining multiple search results to be sorted corresponding to a target search term; determining the correlation feature between each search result and the target search term; determining the diversity feature corresponding to each search result based on a diversity feature extraction model, wherein the diversity feature is used to indicate how much of the corresponding search result includes the meaning corresponding to the target search term; determining the sorting indicator value corresponding to each search result based on the correlation feature and diversity feature corresponding to each search result; and sorting the multiple search results based on the sorting indicator value corresponding to each search result. The use of this application can improve the efficiency of users searching for information through search terms.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a method, apparatus, device, and storage medium for sorting search results. Background Art

[0002] With the development of Internet technology and big data, users searching for relevant information through search terms has become a very common way to obtain information.

[0003] In related technologies, after receiving a search request from a user terminal, a search platform can match corresponding search results in a database based on the search terms carried in the search request. The platform then sorts the matching search results based on their degree of match with the search terms, and finally returns the sorted search results to the user terminal.

[0004] The same search term may have different meanings in different fields. When users search for information using a search term, the search platform cannot clearly understand the specific meaning of the search term entered by the user. For example, the names of some animals and fruits may be registered by companies as their own brands. In this way, when a user uses the name of an animal or fruit registered as a brand as a search term, the search platform cannot know whether the user is searching for related brand information or information related to the animal or fruit. Therefore, if the search platform only sorts the search results based on the match between the search results and the search term, the top-ranked search results may not be the information the user is looking for, resulting in low search efficiency. Summary of the Invention

[0005] The embodiments of the present application provide a method, apparatus, device, and storage medium for sorting search results, which can solve the problem of low search efficiency. The technical solution is as follows:

[0006] In a first aspect, a method for sorting search results is provided, the method comprising:

[0007] Obtain multiple search results to be sorted corresponding to the target search term;

[0008] Determining a relevance characteristic of each search result to the target search term;

[0009] Determining a diversity feature corresponding to each search result based on a diversity feature extraction model, wherein the diversity feature is used to indicate the number of meanings corresponding to the target search term included in the corresponding search result;

[0010] Determining a ranking indicator value corresponding to each search result based on the relevance feature and the diversity feature corresponding to each search result;

[0011] The multiple search results to be sorted are sorted based on the sorting indicator value corresponding to each search result.

[0012] Optionally, determining the diversity feature corresponding to each search result based on the diversity feature extraction model includes:

[0013] Determine the data characteristics corresponding to each search result;

[0014] Establishing a first fully connected graph corresponding to the plurality of search results to be sorted, wherein a plurality of nodes in the first fully connected graph correspond one-to-one to data features of the plurality of search results to be sorted;

[0015] Inputting data features corresponding to nodes connected by each edge in the first fully connected graph into a weight calculation unit in the diversity feature extraction model, determining a weight value corresponding to each edge, and obtaining a second fully connected graph after the weight value is determined;

[0016] The second fully connected graph is input into the diversity feature extraction unit in the diversity feature extraction model to obtain the diversity feature corresponding to each search result.

[0017] Optionally, determining the ranking indicator value corresponding to each search result based on the relevance feature and the diversity feature corresponding to each search result includes:

[0018] Determine the data characteristics corresponding to each search result;

[0019] For each search result, inputting the relevance feature corresponding to the search result into a relevance score calculation module to obtain a relevance score corresponding to the relevance feature; inputting the data feature, diversity feature, and target search term corresponding to the search result into a diversity score calculation module to obtain a diversity score corresponding to the diversity feature;

[0020] The relevance score and the diversity score of each search result are weighted and summed to obtain a ranking indicator value corresponding to each search result.

[0021] Optionally, before determining the diversity feature corresponding to each search result based on the diversity feature extraction model, the method further includes:

[0022] Obtaining a benchmark search result sequence corresponding to the sample search term, the benchmark search result sequence including a plurality of search results matching the sample search term;

[0023] generating a positive sample sequence and a negative sample sequence based on the benchmark search result sequence, wherein the order of the first search results included in the positive sample sequence is the same as the order of the first search results in the benchmark search result sequence, and the order of the second search results included in the negative sample sequence is different from the order of the second search results in the benchmark search result sequence;

[0024] Determining, based on the diversity extraction model, a first diversity score corresponding to the search results included in the positive sample sequence, and determining a second diversity score corresponding to the search results included in the negative sample sequence; determining a first diversity reference value corresponding to the search results included in the positive sample sequence, and determining a second diversity reference value corresponding to the search results included in the negative sample sequence;

[0025] The diversity extraction model is trained based on the first diversity score, the second diversity score, the first diversity reference value, and the second diversity reference value.

[0026] Optionally, sorting the multiple search results to be sorted based on the sort indicator value corresponding to each search result includes:

[0027] The plurality of search results to be sorted are sorted in descending order of the sorting indicator values.

[0028] In a second aspect, a device for sorting search results is provided, the device comprising:

[0029] An acquisition module is used to obtain multiple search results to be sorted corresponding to the target search term;

[0030] a determination module configured to determine a correlation feature between each search result and the target search term; determine a diversity feature corresponding to each search result based on a diversity feature extraction model, wherein the diversity feature is used to indicate the number of meanings corresponding to the target search term included in the corresponding search result; and determine a ranking indicator value corresponding to each search result based on the correlation feature and diversity feature corresponding to each search result;

[0031] The sorting module is configured to sort the plurality of search results to be sorted based on the sorting indicator value corresponding to each search result.

[0032] Optionally, the determining module is configured to:

[0033] Determine the data characteristics corresponding to each search result;

[0034] Establishing a first fully connected graph corresponding to the plurality of search results to be sorted, wherein a plurality of nodes in the first fully connected graph correspond one-to-one to data features of the plurality of search results to be sorted;

[0035] Inputting data features corresponding to nodes connected by each edge in the first fully connected graph into a weight calculation unit in the diversity feature extraction model, determining a weight value corresponding to each edge, and obtaining a second fully connected graph after the weight value is determined;

[0036] The second fully connected graph is input into the diversity feature extraction unit in the diversity feature extraction model to obtain the diversity feature corresponding to each search result.

[0037] Optionally, the determining module is configured to:

[0038] Determine the data characteristics corresponding to each search result;

[0039] For each search result, inputting the relevance feature corresponding to the search result into a relevance score calculation module to obtain a relevance score corresponding to the relevance feature; inputting the data feature, diversity feature, and target search term corresponding to the search result into a diversity score calculation module to obtain a diversity score corresponding to the diversity feature;

[0040] The relevance score and the diversity score of each search result are weighted and summed to obtain a ranking indicator value corresponding to each search result.

[0041] Optionally, the device further includes a training module for:

[0042] Obtaining a benchmark search result sequence corresponding to the sample search term, the benchmark search result sequence including a plurality of search results matching the sample search term;

[0043] generating a positive sample sequence and a negative sample sequence based on the benchmark search result sequence, wherein the order of the first search results included in the positive sample sequence is the same as the order of the first search results in the benchmark search result sequence, and the order of the second search results included in the negative sample sequence is different from the order of the second search results in the benchmark search result sequence;

[0044] Determining, based on the diversity extraction model, a first diversity score corresponding to the search results included in the positive sample sequence, and determining a second diversity score corresponding to the search results included in the negative sample sequence; determining a first diversity reference value corresponding to the search results included in the positive sample sequence, and determining a second diversity reference value corresponding to the search results included in the negative sample sequence;

[0045] The diversity extraction model is trained based on the first diversity score, the second diversity score, the first diversity reference value, and the second diversity reference value.

[0046] Optionally, the sorting module is used to:

[0047] The plurality of search results to be sorted are sorted in descending order of the sorting indicator values.

[0048] In a third aspect, a computer device is provided, comprising a processor and a memory, wherein the memory stores at least one instruction, and the at least one instruction is loaded and executed by the processor to implement the operations performed by the method for sorting search results as described in the first aspect above.

[0049] In a fourth aspect, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores at least one instruction, and the at least one instruction is loaded and executed by a processor to implement the operations performed by the method for sorting search results as described in the first aspect above.

[0050] In a fifth aspect, a computer program product is provided, wherein the computer program product includes at least one instruction, and the at least one instruction is loaded and executed by a processor to implement the operations performed by the method for sorting search results as described in the first aspect above.

[0051] The beneficial effects of the technical solution provided by the embodiments of the present application are:

[0052] In an embodiment of the present application, by determining the correlation features corresponding to a plurality of search results to be sorted and the diversity features used to indicate how many meanings the target search terms include, the corresponding sorting indicator value is determined based on the correlation features and diversity features corresponding to each search result. In this way, when sorting the search results to be sorted based on the sorting indicator value corresponding to each search result, in addition to referring to the correlation between the search results and the query terms, the number of meanings of the query terms included in the search results is also referred to. In this way, in the search results after sorting, the search results that are ranked higher can include more meanings of the query terms, which can increase the probability of hitting the user's search intent, thereby improving search efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0054] Figure 1This is a schematic diagram of an implementation environment provided by an embodiment of the present application;

[0055] Figure 2 This is a flow chart of a method for sorting search results provided by an embodiment of the present application;

[0056] Figure 3 This is a schematic diagram of a method for sorting search results provided by an embodiment of the present application;

[0057] Figure 4 This is a schematic diagram of a method for sorting search results provided by an embodiment of the present application;

[0058] Figure 5 This is a flow chart of a method for training a diversity feature extraction model provided in an embodiment of the present application;

[0059] Figure 6 This is a schematic diagram of the structure of a device for sorting search results provided in an embodiment of the present application;

[0060] Figure 7 It is a structural diagram of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0061] In order to make the objectives, technical solutions and advantages of this application clearer, the implementation methods of this application will be further described in detail below with reference to the accompanying drawings.

[0062] The method for sorting search results provided in this application involves artificial intelligence technology, wherein artificial intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technology in computer science that attempts to understand the essence of intelligence and produce a new type of intelligent machine that can respond in a similar way to human intelligence. Artificial intelligence is also the study of the design principles and implementation methods of various intelligent machines, so that machines have the functions of perception, reasoning and decision-making.

[0063] Artificial intelligence (AI) technology is a comprehensive discipline encompassing a wide range of fields, encompassing both hardware and software technologies. Foundational AI technologies generally include sensors, specialized AI chips, cloud computing, distributed storage, big data processing, operating / interaction systems, and mechatronics. AI software technologies primarily encompass computer vision, speech processing, natural language processing, and machine learning / deep learning.

[0064] Computer vision (CV) is the science of making machines "see." Specifically, it refers to machine vision, where cameras and computers replace the human eye in identifying and measuring objects, performing further image processing to create images more suitable for human observation or transmission to instruments. As a scientific discipline, computer vision studies related theories and technologies, aiming to build artificial intelligence systems capable of extracting information from images or multidimensional data. Computer vision technologies typically include image processing, image recognition, image semantic understanding, image retrieval, optical character recognition (OCR), video processing, video semantic understanding, video content / behavior recognition, three-dimensional object reconstruction, 3D technology, virtual reality, augmented reality, simultaneous localization and mapping, and other technologies. Common biometric recognition technologies include facial recognition and fingerprint recognition.

[0065] Key technologies in speech technology include automatic speech recognition (ASR), text-to-speech (TTS), and voiceprint recognition. Enabling computers to hear, see, speak, and feel is the future direction of human-computer interaction, with speech becoming one of the most promising methods of human-computer interaction.

[0066] Natural language processing (NLP) is a key area of ​​research in computer science and artificial intelligence. It studies the theories and methods that enable effective communication between humans and computers using natural language. Natural language processing (NLP) integrates linguistics, computer science, and mathematics. Therefore, research in this field involves natural language—the language we use in everyday life—and is closely linked to the study of linguistics. Natural language processing technologies typically include text processing, semantic understanding, machine translation, robotic question answering, and knowledge graphs.

[0067] Machine Learning (ML) is a multi-disciplinary interdisciplinary subject involving probability theory, statistics, approximation theory, convex analysis, algorithmic complexity theory and other disciplines. It specializes in studying how computers simulate or implement human learning behavior to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve their own performance. Machine learning is the core of artificial intelligence and the fundamental way to make computers intelligent. Its applications are spread across all areas of artificial intelligence. Machine learning and deep learning generally include artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and learning by formula. For example, in this application, it is necessary to train the machine learning model involved in the method for sorting search results provided in this application through machine learning, so that the trained machine learning model can implement the method for sorting search results. The machine learning model involved in this application will be introduced in the following embodiments.

[0068] Autonomous driving technology usually includes high-precision maps, environmental perception, behavioral decision-making, path planning, motion control and other technologies. Autonomous driving technology has broad application prospects.

[0069] With the research and advancement of artificial intelligence technology, artificial intelligence technology has been studied and applied in many fields, such as common smart homes, smart wearable devices, virtual assistants, smart speakers, smart marketing, unmanned driving, autonomous driving, drones, robots, smart medical care, smart customer service, etc. It is believed that with the development of technology, artificial intelligence technology will be applied in more fields and play an increasingly important role.

[0070] The solutions provided in the embodiments of this application involve artificial intelligence natural language processing, machine learning and other technologies, which are specifically illustrated by the following embodiments:

[0071] Figure 1 This is a schematic diagram of an implementation environment provided by an embodiment of the present application. Figure 1 The method for eliminating background audio data provided in this application can be implemented on a server. The server can be a backend server corresponding to a search platform. The server can be connected to the Internet and can communicate with a user terminal. The user terminal can send a search term input by the user to the server. After receiving the search term sent by the user terminal, the server can query the search term and obtain search results that match the search term. The search results can then be sorted and the sorted search results can be sent to the user terminal.

[0072] Since a search term has different meanings in different fields, the specific meaning of the search term entered by the user cannot be clearly known based only on the search term sent by the user terminal. For example, the names of some animals and fruits may be registered by companies as their own brands. In this way, when a user uses the name of an animal or fruit registered as a brand as a search term, the search platform cannot know whether the user wants to search for related brand information or search for information related to the animal or fruit. However, the method for sorting search results provided by the embodiment of the present application, after sorting the search results corresponding to the search term, can make the search results with higher rankings cover more meanings of the search term. In other words, the content of the search results with higher rankings includes information about the search term in multiple fields.

[0073] Figure 2 This is a flow chart of a method for sorting search results provided in an embodiment of the present application, see Figure 2 The method flow includes:

[0074] Step 201: Obtain multiple search results to be sorted corresponding to the target search term.

[0075] In practice, after receiving any search term (hereinafter referred to as a target search term) sent from one user terminal to another, the server may perform query processing on the target search term to obtain search results matching the search term. The search results may include documents, audio, images, etc. matching the target search term. For example, after querying the search term "apple," the search results obtained may include Apple's website, the nutritional value of apples, and apple cultivation methods.

[0076] In one case, all search results matching the search term may be determined as a plurality of search results to be sorted. In another case, since a single search term may match tens of thousands of search results, a preset number of search results with the highest degree of matching with the target search term among the search results matching the target search term may be determined as the plurality of search results corresponding to the target search term to be sorted.

[0077] Step 202: Determine the correlation characteristics between each search result and the target search term.

[0078] Among them, the correlation feature is used to represent the correlation between each search result and the target search term. The correlation feature of each search result and the target search term can be obtained by calculating the search result through a correlation calculation algorithm. For example, the correlation calculation algorithm can be used to calculate the overall search result, Uniform Resource Locator (URL), anchor document, title, etc. to obtain multiple correlation features between each search result and the target search term. Among them, the correlation calculation algorithm can be a weighted technology used for information retrieval and data mining (termfrequency–inverse document frequency, TF-IDF), BM25, language modeling ininformation retrieval (LMIR), web page ranking algorithm (PageRank), etc. In addition, the number of inbound links and outbound links in the search results can also be used as the correlation feature between the search result and the target search term.

[0079] Step 203: Determine the diversity feature corresponding to each search result based on the diversity feature extraction model.

[0080] The diversity feature is used to indicate the number of search results that include the corresponding meanings of the target search term.

[0081] For multiple search results to be sorted, the data features corresponding to each search result can be extracted through a preset data feature extraction model. For example, after querying the search term "apple", the search results obtained may include Apple's website, the nutritional value of apples, apple planting methods, etc. Among them, Apple's website includes display images of its products, the nutritional value of apples is a document introducing the nutritional value of apples, and the planting method of apples is a video introducing the planting method of apples. Then, the display images of Apple's products on its website can be input into the pre-trained image feature extraction model to obtain the corresponding image features; the document introducing the nutritional value of apples can be input into the pre-trained document feature extraction model to obtain the corresponding document features; the video introducing the planting method of apples can be input into the pre-trained video feature extraction model to obtain the corresponding video features.

[0082] After obtaining the data features corresponding to each search result, the target keyword and the data features corresponding to each search result can be input into a pre-trained diversity feature extraction model, which then outputs the diversity features corresponding to each search result. The diversity feature extraction model can be a Graph Attention Network (GAT). The training process of the diversity feature extraction model will not be described here.

[0083] Further processing to determine the diversity characteristics corresponding to each search result may include:

[0084] Determine the data features corresponding to each search result; establish a first fully connected graph corresponding to multiple search results to be sorted; input the data features corresponding to the nodes connected by each edge in the first fully connected graph into the weight calculation unit in the diversity feature extraction model, determine the weight value corresponding to each edge, and obtain a second fully connected graph after the weight value is determined; input the second fully connected graph into the diversity feature extraction unit in the diversity feature extraction model to obtain the diversity features corresponding to each search result.

[0085] Among them, the number of nodes included in the first fully connected graph is equal to the number of search results to be sorted, and multiple nodes in the first fully connected graph correspond one-to-one to multiple search results, and each node is a data feature of the corresponding search result. In the first fully connected graph, there is an edge between every two nodes, and the weight value of each edge can be used to represent the correlation between the two nodes connected by the edge, that is, the correlation between the search results corresponding to the two nodes connected by the edge. After the first fully connected graph is established, the weight value corresponding to each edge in the first fully connected graph can be initialized first, for example, the weight value of each edge is set to 1. Figure 3 As shown, Figure 3 This is a schematic diagram of a first fully connected graph provided by an embodiment of the present application. Figure 3 In , the search results to be sorted include 5, corresponding to nodes d0-d4 respectively, and the weight value of the edge between any two nodes is 1.

[0086] After obtaining the first fully connected graph, the first fully connected graph can be input into a graph attention network, which outputs the diversity features corresponding to each search result. The graph attention network includes a weight calculation unit and a diversity feature extraction unit. The weight calculation unit can be a multilayer perceptron (MLP), and the diversity feature extraction unit can be composed of at least one layer of a graph neural network. The multilayer perceptron included in the graph attention network may be referred to as the first multilayer perceptron later. The number of layers of the graph neural network can be preset by technicians.

[0087] After the first fully connected graph is input into the graph attention network, the weight value of each edge can be updated by the first multi-layer perceptron in the graph attention network. For the weight value of each edge, the data features of the search results corresponding to the two nodes connected by the corresponding edge can be input into the first multi-layer perceptron, and the first multi-layer perceptron outputs the updated weight value of the edge. After determining the updated weight value corresponding to each edge, a second fully connected graph can be obtained. The second fully connected graph is the first fully connected graph after the weight value of the edge is updated. In this way, if the weight values ​​of the edges corresponding to a node and other nodes are relatively high, the search results corresponding to the node may include more meanings of the search terms. After obtaining the second fully connected graph, the second fully connected graph can be input into the diversity feature extraction unit, and the diversity feature extraction unit outputs the diversity features corresponding to each search result.

[0088] Step 204: Determine a ranking indicator value corresponding to each search result based on the relevance feature and diversity feature corresponding to each search result.

[0089] After obtaining the correlation feature and diversity feature corresponding to each search result, a ranking indicator value of each search result may be calculated based on the correlation feature and diversity feature corresponding to each search result.

[0090] Among them, for each search result, the correlation feature corresponding to the search result is input into the correlation score calculation module to obtain the correlation score corresponding to the correlation feature; the data feature corresponding to each search result is determined, and the data feature, diversity feature and target search term corresponding to the search result are input into the diversity score calculation module to obtain the diversity score corresponding to the diversity feature; the correlation score and the diversity score are weighted and summed to obtain the ranking indicator value corresponding to the search result.

[0091] The relevance score calculation module may be a multilayer perceptron, which may be referred to as a second multilayer perceptron. The diversity score calculation module may also be a multilayer perceptron, which may be referred to as a third multilayer perceptron. It should be noted that the first, second, and third multilayer perceptrons are different multilayer perceptrons.

[0092] like Figure 4 As shown, for example, 18 correlation features r1-r 18 , the 18 correlation features r1-r corresponding to each search result 18 Input to the second multi-layer perceptron, which outputs the relevance score corresponding to the search results. After obtaining the first fully connected graph corresponding to the search results to be sorted ( Figure 4(not shown), the first fully connected graph can be input into a graph attention network, which outputs a diversity feature corresponding to each search result. The data features, diversity features, and target search terms corresponding to each search result are then input into a third multi-layer perceptron, which outputs a diversity score corresponding to the search result.

[0093] exist Figure 4 In the formula, q is the target search term, d is the search result, r is the relevance feature, L is the number of layers of the graph attention network, K is the number of attention heads, and MLP is a multi-layer perceptron. The corresponding calculation formula is as follows:

[0094] First, construct an all-1 adjacency matrix A to represent the first fully connected graph, where n is the number of search results to be sorted, represents the real number field. Then use the learnable Convert the dimension of the data features of the search results, for example, convert the embedding representation of the search results from F(100) to F ′ (256) dimensions to improve the expressive power of the model. Then the weight of the edge corresponding to any two nodes is calculated according to the following formula.

[0095]

[0096] Among them, the superscript represents the number of layers of the graph neural network, is the calculation formula of attention score. i Represents the i-th node in the fully connected graph of the 0th layer input, d j Represents the jth node in the fully connected graph of the 0th layer input. E (0) (d i ) represents the feature corresponding to the i-th node in the fully connected graph of the 0th layer input, E (0) (d j ) represents the feature corresponding to the jth node in the fully connected graph of the 0th layer input; E (l) (d i ) represents the feature corresponding to the i-th node in the fully connected graph of the l-th layer input, E (l) (d j ) represents the feature corresponding to the jth node in the fully connected graph of the lth layer input. The weight calculation formula is as follows:

[0097]

[0098] MLP represents a first multi-layer perceptron, and the activation function of the first multi-layer perceptron may be a linear rectification function (ReLU). represents the i-th node in the fully connected graph of the l-th layer input, Represents the jth node in the fully connected graph of the lth layer input.

[0099] After obtaining the weight corresponding to each edge, the weight corresponding to each edge can be normalized as follows:

[0100]

[0101] in, is the weight of the edge corresponding to the i-th node and the j-th node after normalization, is the weight of the edge between the jth node and the kth node before normalization, is the weight of the edge between the j-th node and the i-th node before normalization. is the number of nodes in the first fully connected graph.

[0102] According to the above formula, the update of the input of each layer is finally obtained:

[0103]

[0104] Here K is the number of heads in the multi-head attention mechanism, which is 4 here. Represents the i-th node in the fully connected graph of the l-th layer input. The last layer output is the diversity feature corresponding to the i-th node in the fully connected graph.

[0105] For each search result, after obtaining the relevance score and diversity score corresponding to the search result, the ranking indicator value corresponding to each search result can be calculated based on the relevance score and diversity score corresponding to the search result. For example, the technician can pre-set the weight coefficients corresponding to the relevance score and diversity score respectively. After obtaining the relevance score and diversity score corresponding to the search result, the relevance score and diversity score can be weighted and summed according to the pre-set weight coefficients to obtain the ranking indicator value corresponding to the search result. Figure 4 In one possible case, the calculation of the ranking indicator value can be achieved by the following formula:

[0106] S rel (d i )×λ+(1-λ)×S diυ (d i )=S(d i )

[0107] Among them, λ is the pre-set weight coefficient, S rel (d i) is the correlation score corresponding to the search result output by the first multi-layer perceptron, S div (d i ) is the diversity score corresponding to the search results output by the second multi-layer perceptron. i ) is the ranking indicator value calculated.

[0108] Step 205: Sort the multiple search results based on the sorting indicator value corresponding to each search result.

[0109] After obtaining the ranking indicator value corresponding to each search result, the multiple search results are sorted according to the ranking indicator value corresponding to each search result. The sorted search results are then sent to the user terminal. For example, the multiple search results can be sorted in descending order of the ranking indicator value.

[0110] In this way, after receiving the multiple search results after sorting, the user terminal can display the multiple search results in the corresponding order. In the embodiment of the present application, the correlation characteristics corresponding to the multiple search results to be sorted and the diversity characteristics used to indicate how many corresponding meanings of the target search terms are included are determined, and then the corresponding sorting indicator value is determined according to the correlation characteristics and diversity characteristics corresponding to each search result. In this way, when sorting the search results to be sorted according to the sorting indicator value corresponding to each search result, in addition to referring to the correlation between the search results and the query terms, the search results also refer to how many meanings of the query terms are included. In this way, in the search results after sorting, the search results that are ranked higher can include more meanings of the query terms, which can increase the probability of hitting the user's search intention, thereby improving the search efficiency.

[0111] Figure 5 This is a method for training a diversity feature extraction model provided in an embodiment of the present application, see Figure 5 , the method comprising:

[0112] Step 501: Obtain a benchmark search result sequence corresponding to a sample search term, where the benchmark search result sequence includes multiple search results matching the sample search term.

[0113] Before training the diversity feature extraction model, a sample search term and a baseline search result sequence corresponding to the sample search term can be pre-set. The baseline search result sequence includes multiple search results that match the sample search term. Search results ranked higher in the baseline search result sequence cover more of the meaning of the sample search term, while search results ranked lower in the baseline search result sequence cover less of the meaning of the sample search term.

[0114] Step 502: Generate a positive sample sequence and a negative sample sequence based on the benchmark search result sequence.

[0115] Among them, the order of the first search results included in the positive sample sequence is the same as the order of the first search results in the benchmark search result sequence, and the order of the second search results included in the negative sample sequence is different from the order of the second search results in the benchmark search result sequence.

[0116] For a sample search term, a positive sample sequence and multiple negative sample sequences can be generated based on the benchmark search result sequence corresponding to the sample search term. The multiple search results included in the positive sample sequence are the search results in the benchmark search result sequence, and the order of the multiple search results included in the positive sample sequence is the same as that of the multiple search results in the benchmark search result sequence. The multiple search results included in the negative sample sequence are the search results in the benchmark search result sequence, and the order of the multiple search results included in the positive sample sequence is different from that of the multiple search results in the benchmark search result sequence.

[0117] Step 503: Based on the diversity extraction model, determine a first diversity score corresponding to the search results included in the positive sample sequence, and determine a second diversity score corresponding to the search results included in the negative sample sequence.

[0118] For a positive sample sequence corresponding to the sample search term, the data features corresponding to each first search result included in the positive sample sequence can be determined, and a first fully connected graph corresponding to each first search result can be determined. The first fully connected graph is then input into a diversity extraction model to be trained, and the diversity extraction model outputs a diversity feature corresponding to each first search result. After obtaining the diversity feature corresponding to each first search result, the sample search term and the diversity features and data features corresponding to the first search result can be input into a second multi-layer perceptron, which then outputs a first diversity score corresponding to each first search result.

[0119] For a negative sample sequence corresponding to the sample search term, the data features corresponding to each second search result included in the negative sample sequence can be determined, and a first fully connected graph corresponding to each second search result can be determined. The first fully connected graph is then input into a diversity extraction model to be trained, and the diversity extraction model outputs a diversity feature corresponding to each second search result. After obtaining the diversity feature corresponding to each second search result, the sample search term and the diversity features and data features corresponding to the second search result can be input into a second multi-layer perceptron, which then outputs a second diversity score corresponding to each second search result.

[0120] Step 504: Determine a first diversity reference value corresponding to the search results included in the positive sample sequence, and determine a second diversity reference value corresponding to the search results included in the negative sample sequence.

[0121] The first diversity benchmark value is the diversity indicator value corresponding to the search results included in the positive sample sequence calculated by the diversity evaluation algorithm. The second diversity benchmark value is the diversity indicator value corresponding to the search results included in the negative sample sequence calculated by the diversity evaluation algorithm. The diversity evaluation algorithm can be α-normalized discounted cumulative gain (NDCG).

[0122] Step 505: Train a diversity extraction model based on the first diversity score, the second diversity score, the first diversity reference value, and the second diversity reference value.

[0123] After obtaining the first diversity score, the second diversity score, the first diversity benchmark value, and the second diversity benchmark value, the corresponding loss value can be determined based on the first diversity score, the second diversity score, the first diversity benchmark value, and the second diversity benchmark value. The diversity extraction model is then trained based on the loss value. The corresponding loss function is as follows:

[0124]

[0125] Among them, Q represents the set of all sample search terms, q represents the sample search term, S q is the training sample set of the sample search term q, s is the training sample, r1 is the positive sequence sample, and r2 is the negative sequence sample. ΔM=M(r1)-M(r2) represents the weight of the training sample, where M(r1) is the calculated first diversity benchmark value and M(r2) is the calculated second diversity benchmark value. When ΔM>0, y s =1, otherwise y s =0, is the first diversity score calculated, is the second diversity score calculated by the model, and Represents 1-y respectively s and 1-P(r1,r2).

[0126] It should be noted that, during training, the diversity extraction model and the multilayer perceptron can be trained together using a gradient descent algorithm. When the overall model meets the training completion conditions, the training of the diversity extraction model can be determined to be completed.

[0127] As can be seen, using the method for training a diversity feature extraction model provided in this application, it is not necessary to use the meaning of each search result as a training label when training the diversity feature extraction model. This also eliminates the need to determine the meaning of each search result when constructing training samples, thereby improving the efficiency of constructing training samples and the efficiency of training the diversity feature extraction model.

[0128] In an embodiment of the present application, by determining the correlation features corresponding to a plurality of search results to be sorted and the diversity features used to indicate how many meanings the target search terms include, and then determining the corresponding sorting indicator value based on the correlation features and diversity features corresponding to each search result. In this way, when sorting the search results to be sorted based on the sorting indicator value corresponding to each search result, in addition to referring to the correlation between the search results and the query terms, the number of meanings of the query terms included in the search results is also referred to. In this way, in the search results after sorting, the search results that are ranked higher can include more meanings of the query terms, which can increase the probability of hitting the user's search intent, thereby improving search efficiency.

[0129] All the above optional technical solutions can be arbitrarily combined to form optional embodiments of the present disclosure, and will not be described in detail here.

[0130] Figure 6 An embodiment of the present application provides a device for sorting search results, which may be the server in the above embodiment. The device includes:

[0131] An acquisition module 610 is configured to acquire a plurality of search results to be sorted corresponding to a target search term;

[0132] Determination module 620 is configured to determine a relevance feature between each search result and the target search term; determine a diversity feature corresponding to each search result based on a diversity feature extraction model, wherein the diversity feature is used to indicate the number of meanings corresponding to the target search term included in the corresponding search result; and determine a ranking indicator value corresponding to each search result based on the relevance feature and diversity feature corresponding to each search result;

[0133] The sorting module 630 is configured to sort the plurality of search results to be sorted based on the sorting indicator value corresponding to each search result.

[0134] Optionally, the determining module 620 is configured to:

[0135] Determine the data characteristics corresponding to each search result;

[0136] Establishing a first fully connected graph corresponding to the plurality of search results to be sorted, wherein a plurality of nodes in the first fully connected graph correspond one-to-one to data features of the plurality of search results to be sorted;

[0137] Inputting data features corresponding to nodes connected by each edge in the first fully connected graph into a weight calculation unit in the diversity feature extraction model, determining a weight value corresponding to each edge, and obtaining a second fully connected graph after the weight value is determined;

[0138] The second fully connected graph is input into the diversity feature extraction unit in the diversity feature extraction model to obtain the diversity feature corresponding to each search result.

[0139] Optionally, the determining module 620 is configured to:

[0140] Determine the data characteristics corresponding to each search result;

[0141] For each search result, inputting the relevance feature corresponding to the search result into a relevance score calculation module to obtain a relevance score corresponding to the relevance feature; inputting the data feature, diversity feature, and target search term corresponding to the search result into a diversity score calculation module to obtain a diversity score corresponding to the diversity feature;

[0142] The relevance score and the diversity score of each search result are weighted and summed to obtain a ranking indicator value corresponding to each search result.

[0143] Optionally, the device further includes a training module for:

[0144] Obtaining a benchmark search result sequence corresponding to the sample search term, the benchmark search result sequence including a plurality of search results matching the sample search term;

[0145] generating a positive sample sequence and a negative sample sequence based on the benchmark search result sequence, wherein the order of the first search results included in the positive sample sequence is the same as the order of the first search results in the benchmark search result sequence, and the order of the second search results included in the negative sample sequence is different from the order of the second search results in the benchmark search result sequence;

[0146] Determining, based on the diversity extraction model, a first diversity score corresponding to the search results included in the positive sample sequence, and determining a second diversity score corresponding to the search results included in the negative sample sequence; determining a first diversity reference value corresponding to the search results included in the positive sample sequence, and determining a second diversity reference value corresponding to the search results included in the negative sample sequence;

[0147] The diversity extraction model is trained based on the first diversity score, the second diversity score, the first diversity reference value, and the second diversity reference value.

[0148] Optionally, the sorting module 630 is configured to:

[0149] The multiple search results are sorted in descending order of the sorting indicator values.

[0150] In an embodiment of the present application, by determining the correlation features corresponding to a plurality of search results to be sorted and the diversity features used to indicate how many meanings the target search terms include, and then determining the corresponding sorting indicator value based on the correlation features and diversity features corresponding to each search result. In this way, when sorting the search results to be sorted based on the sorting indicator value corresponding to each search result, in addition to referring to the correlation between the search results and the query terms, the number of meanings of the query terms included in the search results is also referred to. In this way, in the search results after sorting, the search results that are ranked higher can include more meanings of the query terms, which can increase the probability of hitting the user's search intent, thereby improving search efficiency.

[0151] It should be noted that the apparatus for sorting search results provided in the above embodiment only uses the division of the above functional modules as an example to illustrate the sorting of search results. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the apparatus for sorting search results provided in the above embodiment and the method embodiment for sorting search results are based on the same concept. The specific implementation process is detailed in the method embodiment and will not be repeated here.

[0152] Figure 7 : This is a schematic diagram of the structure of a computer device provided in an embodiment of the present application. The computer device may be the server in the above embodiment. The computer device 700 may have relatively large differences due to different configurations or performances, and may include one or more processors (central processing units, CPUs) 701 and one or more memories 702, wherein the memory 702 stores at least one instruction, and the at least one instruction is loaded and executed by the processor 701 to implement the methods provided in the above-mentioned various method embodiments. Of course, the server may also have components such as a wired or wireless network interface, a keyboard, and an input / output interface for input and output. The computer device 700 may also include other components for implementing device functions, which will not be described in detail here.

[0153] In an exemplary embodiment, a computer-readable storage medium is further provided, such as a memory including instructions, wherein the instructions can be executed by a processor in a terminal to perform the method for sorting search results in the above embodiment. The computer-readable storage medium can be non-transitory. For example, the computer-readable storage medium can be a read-only memory (ROM), a random access memory (RAM), a magnetic tape, a floppy disk, an optical data storage device, etc.

[0154] In an exemplary embodiment, a computer program product is further provided. The computer program product includes at least one instruction, and the at least one instruction is loaded and executed by a processor to implement the method for sorting search results in the above embodiment.

[0155] Those skilled in the art will understand that all or part of the steps to implement the above embodiments may be accomplished by hardware, or by a program to instruct the relevant hardware, and the program may be stored in a computer-readable storage medium, which may be a read-only memory, a disk, or an optical disk, etc.

[0156] It should be noted that the information (including but not limited to user device information, user personal information, etc.), data (including but not limited to data used for analysis, stored data, displayed data, etc.) and signals (including but not limited to signals transmitted between user terminals and other devices, etc.) involved in this application are all authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions. For example, the data involved in this application (such as target search terms, search results, etc.) are all obtained with full authorization.

[0157] The above description is only a preferred embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present application should be included in the scope of protection of the present application.

Claims

1. A method for sorting search results, characterized in that: The method comprises: Obtain multiple search results to be sorted corresponding to the target search term; Determining a relevance characteristic of each search result to the target search term; Determine the data characteristics corresponding to each search result; Establishing a first fully connected graph corresponding to the plurality of search results to be sorted, wherein a plurality of nodes in the first fully connected graph correspond one-to-one to data features of the plurality of search results to be sorted; Inputting the data features corresponding to the nodes connected by each edge in the first fully connected graph into the weight calculation unit in the diversity feature extraction model, determining the weight value corresponding to each edge, and obtaining a second fully connected graph after the weight value is determined, wherein the weight value is used to represent the correlation between the search results corresponding to the two nodes connected by the edge; Inputting the second fully connected graph into a diversity feature extraction unit in the diversity feature extraction model to obtain a diversity feature corresponding to each search result, wherein the diversity feature is used to represent the number of meanings corresponding to the target search term; Determining a ranking indicator value corresponding to each search result based on the relevance feature and the diversity feature corresponding to each search result; The plurality of search results to be sorted are sorted based on the sorting indicator value corresponding to each search result, wherein the more meanings the target search term corresponds to, the higher the sorting of the search result is.

2. The method according to claim 1, characterized in that The determining, based on the correlation feature and the diversity feature corresponding to each search result, a ranking indicator value corresponding to each search result, includes: Determine the data characteristics corresponding to each search result; For each search result, inputting the relevance feature corresponding to the search result into a relevance score calculation module to obtain a relevance score corresponding to the relevance feature; inputting the data feature, diversity feature, and target search term corresponding to the search result into a diversity score calculation module to obtain a diversity score corresponding to the diversity feature; The relevance score and the diversity score of each search result are weighted and summed to obtain a ranking indicator value corresponding to each search result.

3. The method according to claim 1, characterized in that The method further comprises: Obtaining a benchmark search result sequence corresponding to the sample search term, the benchmark search result sequence including a plurality of search results matching the sample search term; generating a positive sample sequence and a negative sample sequence based on the benchmark search result sequence, wherein the order of the first search results included in the positive sample sequence is the same as the order of the first search results in the benchmark search result sequence, and the order of the second search results included in the negative sample sequence is different from the order of the second search results in the benchmark search result sequence; Determining, based on the diversity feature extraction model, a first diversity score corresponding to the search results included in the positive sample sequence, and determining a second diversity score corresponding to the search results included in the negative sample sequence; determining a first diversity reference value corresponding to the search results included in the positive sample sequence, and determining a second diversity reference value corresponding to the search results included in the negative sample sequence; The diversity feature extraction model is trained based on the first diversity score, the second diversity score, the first diversity reference value, and the second diversity reference value.

4. The method according to claim 1, wherein Sorting the plurality of search results to be sorted based on the sorting indicator value corresponding to each search result includes: The plurality of search results to be sorted are sorted in descending order of the sorting indicator values.

5. A device for sorting search results, characterized in that: The device comprises: An acquisition module is used to obtain multiple search results to be sorted corresponding to the target search term; A determination module is configured to determine a data feature corresponding to each search result; establish a first fully connected graph corresponding to the plurality of search results to be sorted, wherein a plurality of nodes in the first fully connected graph correspond one-to-one to the data features of the plurality of search results to be sorted; input the data feature corresponding to each edge-connected node in the first fully connected graph into a weight calculation unit in a diversity feature extraction model, determine a weight value corresponding to each edge, and obtain a second fully connected graph after determining the weight value, wherein the weight value is used to represent the degree of association between the search results corresponding to two nodes connected by the edge; input the second fully connected graph into a diversity feature extraction unit in the diversity feature extraction model to obtain a diversity feature corresponding to each search result, wherein the diversity feature is used to represent the number of meanings corresponding to the target search term included in the corresponding search result; and determine a ranking indicator value corresponding to each search result based on the relevance feature and diversity feature corresponding to each search result; The sorting module is used to sort the multiple search results to be sorted based on the sorting indicator value corresponding to each search result, wherein the more meanings corresponding to the target search term, the higher the sorting of the search result.

6. A computer device, characterized in that: The computer device includes a processor and a memory, wherein the memory stores at least one instruction, and the at least one instruction is loaded and executed by the processor to implement the operations performed by the method for sorting search results according to any one of claims 1 to 4.

7. A computer-readable storage medium, characterized in that The computer-readable storage medium stores at least one instruction, and the at least one instruction is loaded and executed by the processor to implement the operations performed by the method for sorting search results according to any one of claims 1 to 4.

8. A computer program product, characterized in that The computer program product includes at least one instruction, and the at least one instruction is loaded and executed by a processor to implement the operations performed by the method for sorting search results according to any one of claims 1 to 4.

Images