A universal search engine result page rearrangement method based on user perspective

Abstract

The application discloses a general search engine result page rearrangement method based on a user perspective, which comprehensively considers the matching degree of the search result and the initial query under the user perspective, the relevance of the search result and the initial query under the search engine perspective, and the authority of the search result source website itself in the rearrangement and display stage of the general search engine result page, and rearranges the search engine result page.The application uses a large language model to rearrange the search result of the search engine by comprehensively considering the user perspective, the search engine perspective and the authority of the source website, so that the matching degree of the search engine result page to the user intention can be effectively improved without affecting the diversity of the traditional search engine search result, and better use experience can be brought to the user.

Description

TECHNICAL FIELD

[0001] The present application belongs to the field of information retrieval and large language models, and in particular to a user perspective-based general search engine result page rearrangement method. BACKGROUND

[0002] Traditional search engines use keyword matching to quickly recall possible relevant documents from a large number of web pages, then use a lighter model to filter out documents with higher relevance from the recalled documents in the coarse sorting stage, then select a few most relevant documents in the maximum limit of computing power in the fine sorting stage, and finally the coarse sorting stage needs to sort the few documents selected in the fine sorting stage as a whole, taking into account business needs and user experience, and display these documents to the user. However, traditional search engines have the following shortcomings:

[0003] (1) The general search engine directly presents the result page to the user after rearrangement, and there is a visual gap between the user and the search engine, which makes the result page not well meet the intent relevance. Specifically, when calculating the relevance score between the query and the document, the search engine can see many relevant features, such as document text length, document publication date, and embedding vectors calculated by neural networks. However, for the user, only the title, abstract, and website returned by the search engine can be seen. Due to this visual gap, the intent relevance of the result page is poor, making it difficult for the user to efficiently find satisfactory results from the search engine's returned results. The general search engine is different from the vertical search engine for one or several industries, and retrieves information from all web data including various industries.

[0004] (2) The keyword matching retrieval method can cause loss of the initial query intent. For an intent-specific user query, the keyword matching-based method cannot understand the intent deeply, ultimately leading to a retrieval result with high relevance but low intent matching degree.

[0005] In addition, with the rapid development of large language model technology, a new retrieval paradigm has emerged--"search + generation", that is, first use a traditional search engine to retrieve relevant information, then use the excellent language understanding ability of a large language model to extract key information and present it to the user in an easy-to-understand way. The defects of this new paradigm "retrieval + generation" include:

[0006] (1) When generating answers, large language models need to read and generate a large amount of web page content, making the reasoning cost too high.

[0007] (2) Large language models inevitably have hallucination problems, which refer to the generation of information that does not conform to the facts or is meaningless when large language models generate text. There are three main sources of hallucination problems. One is that the model has learned outdated or even incorrect knowledge. The second is that the model's incorrect understanding leads to omissions or biases in the extraction of key web page content. The third is that the model has conflicts in organizing and generating answers. Traditional search engines cannot deeply understand the true intent of user queries, and directly using the "search + generation" paradigm for search result sorting has high costs and inevitably has hallucination problems, which makes search results unable to meet the needs of users. SUMMARY

[0008] To solve the problems existing in the traditional retrieval method, the application realizes a general search engine result page rearrangement method based on user perspective, which comprehensively considers the intention matching degree of the retrieval result and the initial query under the user perspective, the relevance of the retrieval result under the search engine perspective and the authority of the website itself.

[0009] To achieve the above purpose, the technical scheme adopted by the application is:

[0010] In a first aspect, the application discloses a general search engine result page rearrangement method based on user perspective, comprising the following steps:

[0011] S1: For the initial query input into the search engine, an intention clear coefficient is obtained by using a large language model, which describes the clear degree of the query intention indicated by the initial query;

[0012] S2: An ordered retrieval result set is obtained from the search engine result page; the ordered retrieval result set contains all retrieval results, which are sorted according to the order of retrieval results in the search engine result page;

[0013] S3: The sorting sequence number of each retrieval result in the ordered retrieval result set is obtained, and the reciprocal of the sorting sequence number is taken as the search engine perspective relevance score of the retrieval result;

[0014] S4: For each retrieval result, a large language model is called to evaluate the relevance of the title, abstract and source website of the retrieval result with respect to the initial query, and the obtained relevance is weighted and summed to obtain the user perspective relevance score of the retrieval result;

[0015] S5: The authority score of the source website of each retrieval result is evaluated by using machine learning technology;

[0016] S6: Combine the intent explicitness coefficient, the search engine perspective relevance score, the user perspective relevance score, and the authority score of the source website to calculate the final relevance of each search result, and arrange the search results in the search engine results page in descending order based on the final relevance.

[0017] Further, the evaluation method for obtaining the intent explicitness coefficient by using the large language model in step S1 is:

[0018] Input the initial query into the large language model, and use the large language model to score the initial query according to the explicitness of the query intent. The scoring requirements include: the larger the score, the more explicit the query intent, the minimum score is 0, indicating that the initial query is a single noun, and the maximum score is 1, indicating that the initial query has no ambiguity; the score output by the large language model is the intent explicitness coefficient α.

[0019] Further, the specific steps for obtaining the user perspective relevance score r u in step S4 are as follows:

[0020] The titles of all search results are used as a title set, the abstracts of all search results are used as an abstract set, and the source websites of all search results are used as a website set. The title set, the abstract set, and the website set are reordered from high to low according to the relevance of the title, the abstract, and the source website to the initial query by using the large language model from the perspective of the large language model, to obtain a title rearranged set, an abstract rearranged set, and a website rearranged set. The inverse of the ordering number of the title rearranged set, the abstract rearranged set, and the website rearranged set is obtained, to obtain the title relevance of each title in the title rearranged set, the abstract relevance of each abstract in the abstract rearranged set, and the website relevance of each source website in the website rearranged set with respect to the initial query. The weighted results of the title relevance, the abstract relevance, and the website relevance are used as the user perspective relevance score.

[0021] Further, 1) a website authority evaluation data set is constructed, which includes several features of the website and the authority score of the website;

[0022] 2) the LightGBM model is trained using the website authority evaluation data set;

[0023] 3) the authority score of the source website of each search result is predicted using the trained LightGBM model.

[0024] Further, in step 3), the authority score of the source website of each search result is predicted using the trained LightGBM model, specifically: the features of the source website of each search result are input into the trained LightGBM model, and the output of the model is the authority score of the source website of the search result.

[0025] Further, the features of the website include: website domain name, mobile terminal ranking, computer terminal ranking, whether it is an educational website, whether it is an organization website, domain name age, website type, website nature, authentication time, registration location, whether there is a record number, the number of competitive websites, website ranking, update frequency, verticality and credibility.

[0026] Further, the step S6 combines the intent explicitness coefficient, the search engine perspective relevance score, the user perspective relevance score and the authority score of the source website to calculate the final relevance of each search result, which is specifically:

[0027] ro=α*(ru+γ*aw)+(1-α)*rse

[0028] Wherein, r o is the final relevance, alpha is the intent explicitness coefficient, r se is the search engine perspective relevance score, r u is the user perspective relevance score, a w is the authority score of the source website, and gamma is a hyperparameter.

[0029] In a second aspect, the present application discloses a general search engine result page rearrangement system based on a user perspective, which is used to realize the general search engine result page rearrangement method based on a user perspective.

[0030] In a third aspect, the present application discloses an electronic device, which includes a processor and a memory, the memory stores machine executable instructions capable of being executed by the processor, and the processor executes the machine executable instructions to realize the general search engine result page rearrangement method based on a user perspective.

[0031] In a fourth aspect, the present application discloses a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to realize the general search engine result page rearrangement method based on a user perspective.

[0032] The present application proposes a general search engine result page rearrangement method based on a user perspective. The method is based on the search results of a general search engine, and comprehensively considers the intent matching degree of the search results and the initial query under the user perspective, the relevance of the search results under the search engine perspective and the authority of the website itself, which significantly improves the intent matching degree of the search engine result page and overcomes the reasoning cost and illusion problem of the large language model under the "search + generation" paradigm. BRIEF DESCRIPTION OF DRAWINGS

[0033] Figure 1A comparison example diagram of search results of a traditional general search engine and the method of the present application;

[0034] Figure 2 A flowchart of the present application;

[0035] Figure 3 An architecture diagram of the present application;

[0036] Figure 4 A user pre-sorting architecture diagram of the present application. DETAILED DESCRIPTION

[0037] In order to make the above objectives, features and advantages of the present application more obvious and easy to understand, the specific embodiments of the present application will be described in detail below with reference to the accompanying drawings. In the following description, many specific details are set forth in order to provide a thorough understanding of the present application. However, the present application can be implemented in many other ways different from those described herein, and those skilled in the art can make similar improvements without departing from the scope of the present application. Therefore, the present application is not limited to the specific embodiments disclosed below. The technical features in each embodiment of the present application can be combined accordingly without conflict.

[0038] In the description of the present application, it should be understood that the terms "first", "second" are only used for distinguishing description purposes and cannot be understood as indicating or implying relative importance or implicitly indicating the number of the technical features indicated. Therefore, the features defined with "first", "second" can explicitly or implicitly include at least one of the features.

[0039] As shown in Figure 2 In a preferred implementation of the present application, the above-mentioned general search engine result page rearrangement method based on user perspective includes the following S1-S6 steps. The specific implementation process will be described in detail below.

[0040] S1: For the initial query input by the user, the excellent language understanding ability of the large language model (LLM) is used to evaluate to what extent the current initial query accurately indicates the user's intention, and the intention clear coefficient a of the initial query is obtained. The larger the intention clear coefficient a, the more explicit the intention of the initial query.

[0041] The initial query is the text input by the user when using the general search engine to search, and the large language model includes ChatGPT, Qwen, Llama, etc.

[0042] The specific sub-steps include:

[0043] (1)Judge whether the initial query is a simple noun. If so, the query intent score is 0, because at this time the specific intent of the user is not known, that is, it means that the user may want to know any sub-topic under the current query noun. For example, when the initial query is "apple", it may represent the Apple company, may represent the Apple computer, may represent the fruit or other intent, at this time the query intent is completely unclear, and the traditional general search method is used for retrieval.

[0044] (2) If the initial query is not a simple noun, a carefully designed prompt word is needed to prompt the large language model to determine to what extent the user query clearly expresses the intent. Specifically, the large language model used in this embodiment is Qwen, and the prompt word designed in this embodiment is:

[0045] "User: You are a search algorithm expert, and your task is to evaluate the intent clear coefficient a of the given query of the user.

[0046] Query:

[0047] {query}.

[0048] Evaluation method:

[0049] - The intent clear coefficient a is divided into 5 levels according to the degree of query intent, and the numerical value is divided from small to large

[0050] It is represented as 0, 0.25, 0.5, 0.75 and 1. The larger the value, the more clear the query intent;

[0051] - 0 means that the query is a noun or a simple combination of multiple nouns. For example, "apple", the query intent may be edible apple, may be Apple company, may be Apple computer or other, the query intent is very unclear, so r is 0;

[0052] - 1 means that the query is a complete and clear expression. For example, "Will migraine be inherited?", The query intent is very clear, so r is 1.

[0053] Please output the intent clear coefficient a of the given query. Only the final intent clear coefficient a should be output, without other text content or explanation."

[0054] It should be noted that {.} indicates that the brackets and their contents in the brackets will be replaced with the corresponding content in actual implementation, and the brackets indicate what should be replaced; For example, {query} will be replaced with the initial query input by the user in actual implementation.

[0055]

[0056] ​S2: input the initial query of the user into a general search engine to obtain a high-quality ordered search result set. It should be noted that in this step S2, the retrieval method and process of the traditional search engine are strictly followed, and the ordered search result set of the search engine is obtained.

[0057] S3: for the ordered result set in S2, obtain the ranking sequence number of each search result, and the reciprocal of the ranking sequence number is the relevance score of the result and the initial query, i.e. the search engine perspective relevance score r se .

[0058] It should be noted that in the perspective of the search engine, the position of the search result in the ordered search result set returned by the search engine is more forward, and the overall relevance is higher, and the probability of the user adopting the result is greater. In order to calculate the overall relevance of the search engine perspective search result, the present application designs a simple and easy-to-use calculation method, which includes the following sub-steps:

[0059] (1) determine the position of each document in the ordered search result set, i.e. the ranking sequence number i;

[0060] (2) take the reciprocal of the ranking sequence number of each document, i.e. the reciprocal of the ranking sequence number represents the overall relevance of the current document in the perspective of the search engine r se , i.e.

[0061]

[0062] The overall relevance of the document can be easily obtained by taking the reciprocal of the ranking sequence number, so that the document ranking is more backward and the overall relevance of the document is lower.

[0063] S4: for the ordered result set in S2, carefully design the prompt words to prompt the large language model to evaluate the relevance of each title, abstract and source website of the result set to the initial query from the perspective of the user, and then weightedly sum to obtain the relevance of each search result to the initial query, i.e. the user perspective relevance score r u . The specific sub-steps include:

[0064] (1) obtain the title of each search result in the search result returned by the search engine to obtain the search result title set corresponding to the current query;

[0065] (2) design an efficient prompt template for the large language model to store the initial query of the current user and the title set, and clearly state the task details. Since the search engine aims to adjust the position of the search result, the present application designs the task as a ranking task. Specifically, the prompt words used by the present application are:

[0066] User: I will provide you with {num} sentences, each represented by [i]. Please rank them according to their relevance to {query}.

[0067] Sentences:

[0068] [1] {Sentence 1}

[0069] [2] {Sentence 2} ...

[0071] [{num}] {Sentence {num}}

[0072] Query:

[0073] {query}.

[0074] Please rank the sentences according to their relevance to {query}, and all sentences must be ranked and represented by [i]. The output format should be [i]>[j], such as [4]>[2]>[6]>.... Only the final ranking result needs to be output, without other text content or explanation."

[0075] Note that {num} will be replaced by the number of sentences in the actual implementation; {Sentence 1}, {Sentence 2},..., {Sentence {num}} will be replaced by the titles in the title set, the abstracts in the abstract set, and the website names in the source website set in the actual implementation.

[0076] Designing appropriate prompt words to prompt the large language model to give evaluation scores for input data or to rank the input data set according to relevance is a common means in the art, and those skilled in the art can modify the prompt words described in the examples without departing from the spirit of the invention.

[0077] In the present invention, the large language models used in steps S1 and S4 can be different large language models.

[0078] (3) Input the prompt word into the large language model to prompt it to reorder the title set according to the relevance of each title in the title set to the intent of the current initial query, and obtain a reordered title set;

[0079] (4) Taking the inverse of the ranking number of each title, the title intent relevance r of each title to the initial query can be obtained t .

[0080] (5) The calculation method of abstract relevance r s and website relevance r w is consistent with title relevance r t .

[0081] (6)Referring to Figure 4 , the user perspective relevance score r u is calculated by the following formula

[0082] r u = r t + r s + β*r w

[0083] wherein β is a hyperparameter for controlling the weight of the website relevance r w , which is set to 0.2 in this embodiment.

[0084] S5: For the ordered result set in S2, the source websites of each result in the result set are evaluated using data mining and machine learning techniques to obtain the authority score a w of each source website. The specific steps are as follows:

[0085] (1) Build an authority score evaluation dataset D by data mining techniques, which contains the features and authority scores of a number of websites, including: website domain name, mobile ranking, computer ranking, whether it is an educational website, whether it is an organization website, domain age, website type, website nature, authentication time, registration location, whether there is a record number, the number of competing websites, website ranking, update frequency, verticality and credibility. The above website feature data can be obtained by querying and processing third-party websites, or sorted or classified by other means.

[0086] (2) Train the LightGBM model with the authority score evaluation dataset D;

[0087] (3) Obtain the features of each source website and input them into the trained LightGBM model to predict the authority score a w of the source website.

[0088] S6: Referring to Figure 3 , the search engine perspective relevance r se , the user perspective relevance r u and the website authority score a w are weighted and summed by the intent clarity coefficient α and the hyperparameter γ to obtain the final relevance r o of each search result, and the formula is as follows:

[0089] r o = α*(r o + γ*a w ) + (1-α)*r se

[0090] Wherein, γ is a super parameter, used to control the weight of the website authority, which is set to 0.1 in the embodiment.

[0091] Further, according to the final relevance r of each search result o The search results are ranked from high to low, and the result page that matches the user's initial query intent more is obtained.

[0092] Taking the query "which non-normal colleges have normal majors" as an example, the query is obviously intent-specific, and the general search engine result page before and after rearrangement is shown in Figure 1 The left side of the figure is the search result of the general search engine result page before rearrangement, and the right side is the search result after rearrangement using the application, which is consistent with the current intent. The search result that matches the current intent is ranked in the front, and the search result that does not match the current intent is ranked in the back. It can be seen that the application effectively rearranges the result page from the user's perspective.

[0093] Based on the same inventive concept, in the embodiment, a general search engine result page rearrangement system based on the user's perspective is also provided, which comprises:

[0094] A query intent evaluation module is used to obtain an intent-specific coefficient of the initial query;

[0095] A search engine module is used to obtain an ordered search result set from the search engine result page;

[0096] A search engine ranking module is used to obtain a search engine perspective relevance score of each search result in the ordered search result set;

[0097] A user pre-ranking module is used to evaluate a user perspective relevance score of the search result;

[0098] A website authority evaluation module is used to evaluate an authority score of the website where the search result comes from;

[0099] A result page comprehensive ranking module is used to calculate a final relevance of the search result by comprehensively considering the intent-specific coefficient, the search engine perspective relevance score, the user perspective relevance score, and the authority score of the website where the search result comes from, and to rearrange the search engine result page based on the final relevance.

[0100] For the system embodiment, since it basically corresponds to the method embodiment, the relevant part is described in the method embodiment, and the implementation method of the remaining modules is not described here. The system embodiment described above is only illustrative, wherein the units described as separate components can or can not be physically separated, and the components displayed as units can or can not be physical units, that is, they can be located in one place or distributed on multiple network units. Part or all of the modules can be selected to achieve the purpose of the present application according to actual needs. Those skilled in the art can understand and implement without creative labor.

[0101] The system embodiment of the present application can be applied to any device with data processing capability, which can be a device or apparatus such as a computer. The system embodiment can be implemented by software, hardware, or a combination of software and hardware. Taking software implementation as an example, as a logical device, it is formed by reading the corresponding computer program instructions in the non-volatile memory into the memory and running by the processor of the device with data processing capability.

[0102] The embodiment of the present application also provides an electronic device, comprising a processor and a memory:

[0103] The memory is used for storing a computer program;

[0104] The processor is used for implementing the user perspective-based general search engine result page rearrangement method described above when executing the computer program.

[0105] The embodiment of the present application also provides a computer readable storage medium, which stores a computer program, and the computer program is executed by the processor to implement the user perspective-based general search engine result page rearrangement method described above.

[0106] The computer readable storage medium can be an internal storage unit of any device with data processing capability, such as a hard disk or a memory. The computer readable storage medium can also be an external storage device of any device with data processing capability, such as a plug-in hard disk, a smart media card (SMC), an SD card, a flash card, etc. Further, the computer readable storage medium can include both the internal storage unit and the external storage device of any device with data processing capability. The computer readable storage medium is used for storing the computer program and other programs and data required by the device with data processing capability, and can also be used for temporarily storing data that has been output or will be output.

[0107] The above examples are only used for illustrating the present application and not for limiting the scope of the present application. For those skilled in the art, other various corresponding changes and modifications can be made according to the technical solutions and concepts described above, and all these changes and modifications should belong to the protection scope of the claims of the present application.

Claims

1. A method for rearranging a general search engine results page based on a user's perspective, the method comprising: The method comprises the following steps: S1: for an initial query input into a search engine, a large language model is used to evaluate an intention clear coefficient, which describes the degree of clarity of the initial query indicating the query intention; S2: an ordered search result set is obtained from a search engine results page; the ordered search result set contains all search results, sorted by the order of the search results in the search engine results page; S3: the ranking number of each search result in the ordered search result set is obtained, and the reciprocal of the ranking number is taken as the search engine perspective relevance score of the search result; S4: for each search result, a large language model is called to evaluate the relevance of the title, abstract and source website of the search result to the initial query respectively, and the weighted sum of the relevance obtained by evaluation is taken as the user perspective relevance score of the search result; S5: the authority score of the source website of each search result is evaluated using machine learning technology; S6: the final relevance of each search result is calculated by combining the intention clear coefficient, the search engine perspective relevance score, the user perspective relevance score and the authority score of the source website, and the search results in the search engine results page are ranked in descending order based on the final relevance.

2. A method for rearranging the results of a general search engine based on the user's perspective as claimed in claim 1, wherein, The evaluation method for obtaining the intention clear coefficient by using the large language model in step S1 is as follows: The initial query is input into the large language model, and the large language model is used to score the initial query according to the degree of clarity of the query intention. The scoring requirements include: the higher the score, the more clear the query intention, the lowest score is 0, indicating that the initial query is a single noun, and the highest score is 1, indicating that there is no ambiguity in the initial query; the score output by the large language model is the intention clear coefficient.

3. The method of claim 1, wherein the method is based on a user's perspective. The specific steps for obtaining the user perspective relevance score in step S4 are as follows: The titles of all search results are taken as a title set, the abstracts of all search results are taken as an abstract set, and the source websites of all search results are taken as a website set; the title set, the abstract set and the website set are reordered from high to low in terms of the relevance of the title, the abstract and the source website to the initial query according to the large language model perspective, to obtain a title rearranged set, an abstract rearranged set and a website rearranged set; the reciprocal of the ranking number of the title rearranged set, the abstract rearranged set and the website rearranged set is taken as the title relevance of each title in the title rearranged set, the abstract relevance of each abstract in the abstract rearranged set and the website relevance of each source website in the website rearranged set; the weighted results of the title relevance, the abstract relevance and the website relevance are taken as the user perspective relevance score.

4. The method of claim 1, wherein the method is based on a user's perspective. The specific steps for calculating the website authority of each search result in step S5 are as follows: ​ 1) a website authority evaluation data set is constructed, which includes several features of the website and the authority score of the website; 2) the LightGBM model is trained using the website authority evaluation data set; 3) the authority score of the source website of each search result is predicted using the trained LightGBM model.

5. A method for rearranging the results of a general search engine based on the user's perspective as recited in claim 4, wherein, In step 3), the LightGBM model trained is used to predict the authority score of the source website of each search result, specifically: the features of the source website of each search result are input into the trained LightGBM model, and the output of the model is the authority score of the source website of the search result.

6. A method for rearranging the results of a general search engine based on the user's perspective according to claim 4 or 5, characterized in that, The features of the website include: website domain name, mobile ranking, computer ranking, whether it is an educational website, whether it is an organization website, domain age, website type, website nature, authentication time, registration location, whether there is a record number, the number of competitive websites, website ranking, update frequency, verticality and credibility.

7. A method for rearranging the results of a general search engine based on the user's perspective as recited in claim 1, wherein, The final relevance of each search result is calculated by combining the intent clarity coefficient, search engine perspective relevance score, user perspective relevance score and authority score of the source website, as described in step S6. r o = a * (r u + g * a w ) + (1 - a) * r se where r o is the final relevance, a is the intent explicit coefficient, r se is the search engine perspective relevance score, r u is the user perspective relevance score, a w is the authority score of the source website, and g is a hyperparameter.

8. A user perspective based general search engine results page reordering system, comprising: The system for implementing the rearrangement method of any one of claims 1-7 comprises: An intent evaluation module for obtaining the intent clarity coefficient of the initial query; A search engine module for obtaining an ordered search result set from a search engine results page; A search engine ranking module for obtaining the search engine perspective relevance score of each search result in the ordered search result set; A user pre-ranking module for evaluating the user perspective relevance score of the search result; A website authority evaluation module for evaluating the authority score of the source website of the search result; A result page comprehensive ranking module for combining the intent clarity coefficient, search engine perspective relevance score, user perspective relevance score and authority score of the source website to calculate the final relevance of the search result, and rearranging the search engine results page based on the final relevance.

9. An electronic device, comprising: A processor and a memory, the memory storing machine executable instructions executable by the processor, the processor executing the machine executable instructions to implement the user perspective based general search engine results page rearrangement method of any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, A computer program is stored thereon, and the computer program is executed by a processor to implement the user perspective based general search engine results page rearrangement method of any one of claims 1-7.

Images