Search method, system, device, equipment and storage medium

Abstract

The present disclosure provides a search method, system, apparatus, device, and storage medium, relating to the fields of computer technology, particularly search, neural networks, large language models, and other technical fields. A specific implementation scheme comprises: inputting a search statement and multiple search results into a distribution module, which distributes the search statement and multiple search results to a first model; wherein the first model is a large language model suitable for the search scenario corresponding to the search statement and multiple search results; and, using the first model, generating search results based on the search statement and multiple search results. The present disclosure can optimize search results.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technology, and in particular to technical fields such as search, neural networks, and large language models. Background Art

[0002] A search engine is a retrieval technology that uses specific strategies based on user needs and algorithms to retrieve specific information from the internet and provide it back to users. The development of Large Language Model (LLM) technology has led to a rapid increase in the natural language understanding and processing capabilities of artificial intelligence systems. Integrating LLM technology into search engines to enhance their search capabilities is a technical challenge that needs to be addressed. Summary of the Invention

[0003] The present disclosure provides a search method, system, apparatus, device, and storage medium.

[0004] According to one aspect of the present disclosure, a search method is provided, comprising:

[0005] Inputting a search statement and a plurality of search contents into a distribution module, and the distribution module distributing the search statement and the plurality of search contents to a first model; wherein the first model is a large language model applicable to the search scenario corresponding to the search statement and the plurality of search contents; and

[0006] Generate search results based on the search statement and multiple search contents through the first model.

[0007] According to another aspect of the present disclosure, a search system is provided, including a distribution module and multiple large language models, each of which corresponds to a different search scenario:

[0008] The distribution module is configured to receive a search statement and a plurality of search contents, and distribute the search statement and the plurality of search contents to a first model; wherein the first model is a large language model applicable to the search scenario corresponding to the search statement and the plurality of search contents;

[0009] The large language model is used to receive a search statement and multiple search contents from the distribution module, and generate search results based on the search statement and multiple search contents.

[0010] According to another aspect of the present disclosure, there is provided a distribution device, comprising:

[0011] A receiving unit, configured to receive a search statement and multiple search contents;

[0012] A distribution unit is used to determine a first model corresponding to the search statement and multiple search contents, and distribute the search statement and multiple search contents to the first model, so that the first model generates search results based on the search statement and multiple search contents; wherein the first model is a large language model suitable for the search scenario corresponding to the search statement and multiple search contents.

[0013] According to another aspect of the present disclosure, there is provided an electronic device, comprising:

[0014] at least one processor; and

[0015] a memory communicatively connected to the at least one processor; wherein,

[0016] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform any method in the embodiments of the present disclosure.

[0017] According to another aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to enable the computer to execute any method according to the embodiments of the present disclosure.

[0018] According to another aspect of the present disclosure, a computer program product is provided, including a computer program. When the computer program is executed by a processor, the computer program implements any one of the methods according to the embodiments of the present disclosure.

[0019] The present disclosure uses a distribution module to input the search statement and multiple search results into a large language model that is "excellent" at handling the search scenario. This large language model is then adapted to the search scenario corresponding to the search statement and multiple search results. This large language model then generates the final search results based on the search statement and multiple search results. Therefore, the present disclosure embodiment can fully leverage the advantages of each large language model to optimize search results.

[0020] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present disclosure, nor are they intended to limit the scope of the present disclosure. Other features of the present disclosure will become readily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] The accompanying drawings are used to better understand the present invention and do not constitute a limitation of the present invention.

[0022] Figure 1 is a schematic diagram of an application scenario according to an embodiment of the present disclosure;

[0023] Figure 2 is a flowchart of an implementation of a search method according to an embodiment of the present disclosure;

[0024] Figure 3 is a flowchart of an implementation of a search method according to another embodiment of the present disclosure;

[0025] Figure 4 is a flowchart for determining a large language model applicable to each search scenario from multiple large language models in a search method according to another embodiment of the present disclosure;

[0026] Figure 5 is a schematic diagram of a search method according to another embodiment of the present disclosure;

[0027] Figure 6 is a schematic structural diagram of a search system 600 according to an embodiment of the present disclosure;

[0028] Figure 7 is a structural diagram of a search system 700 according to another embodiment of the present disclosure

[0029] Figure 8 A schematic block diagram of an example electronic device 800 is shown, which may be used to implement embodiments of the present disclosure. DETAILED DESCRIPTION

[0030] The following description of exemplary embodiments of the present disclosure is made in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be appreciated by those skilled in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0031] The “and / or” in the embodiments of the present disclosure indicates that there may be three relationships. For example, A and / or B may indicate three situations: A exists alone, A and B exist at the same time, and B exists alone. The term “at least one” herein indicates any combination of at least two of any one or more of a plurality of. For example, at least one of A, B, and C may indicate any one or more elements selected from the set consisting of A, B, and C. The terms “first” and “second” herein refer to and distinguish between multiple similar technical terms, and do not mean to limit the order or to limit the meaning to only two. For example, the first feature and the second feature refer to two categories / two features. The first feature may be one or more, and the second feature may also be one or more.

[0032] A search engine is a retrieval technology that uses specific strategies based on user needs and algorithms to retrieve specific information from the internet and provide it back to the user. Traditional search engine technology indexes content on the internet and provides keyword-based web search services. However, while traditional search engines can quickly return a large amount of relevant information, they often lack deep understanding and the ability to adapt and adjust to specific questions. If the original webpage does not meet the user's query requirements or the webpage content is low in knowledge density and contains irrelevant arguments, it will increase the user's query and search costs and reduce the user's search experience.

[0033] A large language model is a deep learning-based natural language processing (NLP) model used to process and generate human language text. It can learn language patterns from large corpora, understand and generate complex language structures, and possess contextual understanding, memory, and reasoning capabilities. The development of large model technology has led to a rapid increase in the natural language understanding and processing capabilities of artificial intelligence systems. Incorporating the capabilities of large language models into search engines, enabling them to deeply understand user needs and improve the quality of search results, is currently an area of ​​active exploration.

[0034] The embodiment of the present disclosure aims to propose a search method that introduces large language model technology into search engine technology to solve users' search needs.

[0035] Figure 1 is a schematic diagram of an application scenario according to an embodiment of the present disclosure, such as Figure 1 As shown, the application scenario diagram of the embodiment of the present disclosure may include a search terminal 110 and a search system 120, and the search terminal 110 and the search system 120 can communicate with each other through any type of wired or wireless network. Specifically, the user inputs a search request through the search terminal 110, and the search terminal 110 sends the search request to the search system 120 through a wired or wireless network; the search system 120 receives the search request, searches based on the search request, and obtains search results. Among them, the search terminal 110 proposed in the embodiment of the present disclosure includes but is not limited to electronic devices such as mobile phones, computers, intelligent voice interaction devices, smart home appliances, car terminals, game consoles, e-book readers, multimedia playback devices, wearable devices, etc.; the search system 120 may include electronic devices or servers for providing search services for the search terminal 110. In addition, the embodiment of the present disclosure does not specifically limit the number of search terminals 110. For example, the application scenario diagram of the embodiment of the present disclosure may include one or more search terminals 110.

[0036] Figure 21 is a flowchart of an implementation of a search method according to an embodiment of the present disclosure, including:

[0037] S210: Input a search statement and a plurality of search contents into a distribution module, and the distribution module distributes the search statement and the plurality of search contents to a first model; wherein the first model is a large language model applicable to the search scenario corresponding to the search statement and the plurality of search contents; and

[0038] S220: Generate search results based on the search statement and multiple search contents using the first model.

[0039] As can be seen, the search method proposed in the embodiment of the present disclosure summarizes the search scenarios that the large language model is "excellent at". When the distribution model receives a search statement and multiple search contents, it inputs the search statement and multiple search contents into the large language model that is "excellent at processing" the search scenario, that is, the large language model that is applicable to the search scenario corresponding to the search statement and multiple search contents. The large language model then generates the final search results based on the search statement and multiple search contents. Therefore, the embodiment of the present disclosure can fully utilize the advantages of each large language model and optimize the search effect.

[0040] Figure 3 This is a flowchart of a search method according to another embodiment of the present disclosure. In some implementations, before the search statement and multiple search contents are input into the distribution module, the following steps may also be included:

[0041] S310, inputting a search statement into a search engine;

[0042] S320: Determine the plurality of search contents according to the output of the search engine; the plurality of search contents match the search statement.

[0043] For example, the search statement is a search query input by the user, and the search engine outputs the K search contents (TopK search contents) that have the highest matching degree with the user's search query; the search query and TopK search contents are distributed by the distribution model to the large language model applicable to the search scenario. The large language model can be used as a large language model problem processing module built into the search engine. The search engine can have multiple large language model problem processing modules (i.e., large language models) built into it, and each large language model is adapted to different fields or search scenarios, such as medical, legal, life knowledge, education, encyclopedia, etc. The large language model can summarize and refine the web page content (i.e., multiple search contents) searched by the search engine based on its text understanding and organization generation capabilities, and use the internalized knowledge obtained from model pre-training in this process to further enrich the summary content, thereby obtaining richer and more accurate answers than the original web page, thereby improving the efficiency of users in acquiring knowledge.

[0044] It can be seen that the retrieval method proposed in the embodiment of the present disclosure can comprehensively utilize a system of multiple large language models, analyze the capabilities of multiple large language models from the dimensions of cost, performance, field, demand, etc., and combine the characteristics of the search scenario to select the large language model that best suits the scenario to solve the user's search needs.

[0045] In some implementations, after the large language model generates search results, the search results may be returned to a search engine, and the search engine may render and display the search results.

[0046] It can be seen that the embodiment of the present disclosure can use multiple large language models as built-in modules of the search engine. After the large language model generates search results, the search engine is used to render and display the search results. Therefore, it can fully utilize the existing functions of the search engine and make relatively small changes to the existing search engine.

[0047] In some implementations, the search method further includes: pre-training a plurality of large language models; and determining, from the plurality of large language models, a large language model applicable to each search scenario.

[0048] Each large language model is a model that answers questions. Its input is the query and a TopK web page index, and its output is the answer to the question. For example, if the query is "What is the significance of Qingming Festival?", the input TopK web page index is a collection of web pages related to this question, retrieved from a search engine. These pages include content describing the origins, customs, and cultural significance of Qingming Festival. The query and web page text are then distributed to a large language model appropriate for that search scenario. The model then summarizes the web page content and outputs the answer to "What is the significance of Qingming Festival?" In this process, the large language model primarily relies on its internalized knowledge and content summarization capabilities, referencing the web page content to answer the query.

[0049] The training set for a large language model includes multiple queries and the top K web pages, potentially containing thousands to tens of thousands of such training data. Then, through distillation using other more powerful large models or manual answer annotation, high-quality answers to the corresponding queries are obtained. This results in a training set consisting of the query, the top K web page content, and the answers. The "query + top K web page content" in this training set is the training sample, and the corresponding answer is the sample label. This training set is used to train the large language model, enabling it to acquire the answer summarization capabilities desired in this solution.

[0050] The different training sets used by different large language models result in different performances of different large language models in different scenarios. For example, model A is good at historical questions, while model B is good at common sense questions.

[0051] Since there is currently no large language model that can perfectly address all search needs and domain-specific problems, pre-trained models with smaller parameters can achieve good performance for simple knowledge question-answering tasks after a few rounds of fine-tuning. However, such models have limited capabilities for complex tasks. Conversely, using larger models for simple tasks wastes computing power and increases costs. For example, simple search requirements include key data and encyclopedia queries, such as "a person's hometown" or "the location of a certain tourist attraction." Complex search requirements, such as summary-based requirements, include "the nutritional value of a certain food and precautions for consumption" or "analysis of the causes of a certain phenomenon or event." The retrieval method proposed in the disclosed embodiments can distribute search statements and multiple search results for simple requirements to a smaller large language model, using this smaller large language model to solve the simple requirements; and distribute search statements and multiple search results for complex requirements to a larger large language model, using this larger large language model to solve the complex requirements. This achieves good retrieval performance without wasting computing power. Search engines that introduce a distribution module have the ability to rationally dispatch large language models based on specific needs, achieving good results in various scenarios at the most reasonable cost. The embodiment of the present disclosure pre-trains multiple large language models suitable for different search scenarios. Therefore, the set of search scenarios to which the multiple large language models are applicable can be equivalent to the entire field. Therefore, the retrieval method proposed in the embodiment of the present disclosure can achieve better retrieval effects in the entire field.

[0052] In addition, since there is currently no large language model that can perfectly respond to all search needs and problems in the field, in order to achieve the best capabilities in multiple dimensions, the enhanced training of the large language model often brings about the "seesaw" problem, that is, after improving the ability of a certain search scenario, the ability of another search scenario will decrease. The retrieval method proposed in the embodiment of the present disclosure can solve this type of problem. The embodiment of the present disclosure pre-trains multiple large language models, and each large language model is applicable to (i.e., "good at") different search scenarios. Therefore, when training a large language model, it is only necessary to focus on the search scenario to which the large language model is applicable, and improve the ability of the large language model in the search scenario in which it is used, without paying attention to its ability in other search scenarios, and the "seesaw" problem will not occur.

[0053] After training multiple large language models, we can conduct a comprehensive evaluation of their capabilities to determine the types of questions each model excels at. This comprehensive evaluation can be performed using a query set that covers a more comprehensive range of fields. By comparing the quality of the output answers of different large models, we can rank their capabilities.

[0054] like Figure 4As shown, in some embodiments, determining a large language model applicable to each search scenario from multiple large language models includes:

[0055] S410: Obtain a verification data set covering multiple search scenarios, wherein the verification data set includes M groups of data, each group of data including a search statement and multiple search contents, and each group of data corresponds to a search scenario; M is a positive integer;

[0056] S420: For each large language model, input the data in the verification dataset into the large language model, and score the search results output by the large language model to obtain M scoring results for the large language model;

[0057] S430: Based on the M scoring results of each large language model, rank the effects of multiple large language models in each search scenario;

[0058] S440: Determine the large language model applicable to each search scenario based on the ranking results.

[0059] For example, in a common-sense search scenario, the performance of multiple large language models in this scenario is ranked as Model A, Model B, Model C, and so on. That is, Model A is the most effective large language model in this scenario. Therefore, the appropriate large language model for this scenario can be determined to be Model A. Using a validation dataset covering a variety of search scenarios allows for a comprehensive evaluation of multiple successfully trained large language models, thereby improving the accuracy of determining the appropriate large language models for different search scenarios.

[0060] In addition, the embodiments of the present disclosure can further fine-tune each large language model to improve the ability of the large language model in its applicable scenario. For example, for each large language model applicable to each search scenario, the large language model is fine-tuned, and the fine-tuning methods include:

[0061] Determining a first training set for fine-tuning, where the first training set includes a search statement and search content corresponding to the search scenario;

[0062] The first training set is used to fine-tune the large language model applicable to the search scenario.

[0063] For example, after training multiple large language models, we can continue to enrich training data in different fields to form specialized training sets, such as historical training sets and common sense training sets. Then, the historical training sets are only used to fine-tune model A, and the common sense training sets are only used to fine-tune model B. This will further enhance the capabilities of each model in its own area of ​​expertise and form specialized capabilities.

[0064] The distribution module proposed in the embodiment of the present disclosure may include a pre-trained distribution model, which is used to determine the corresponding search scenario and a large language model applicable to the search scenario based on the search statement and multiple search contents.

[0065] After obtaining the performance ranking of different large language models, we can use the query + web page -> the model that best answers the query as a training feature to train the distribution model.

[0066] In some embodiments, the training method includes:

[0067] Determine a second training set, the second training set including multiple training data and labels corresponding to each training data; the training data includes a search statement and multiple search contents, and the labels corresponding to the training data include search scenarios and a large language model corresponding to the search statement and the multiple search contents;

[0068] The distribution model is trained using the second training set.

[0069] By training the distribution model with search statements and search content corresponding to different search scenarios, the successfully trained distribution model can be equipped with the ability to identify the search scenarios corresponding to the search statements and search content, and then identify the appropriate large language model. This enables the distribution model to act as a central node, distributing query traffic and allowing each large model downstream to perform its duties.

[0070] Figure 5 FIG. 1 is a schematic diagram of a search method according to another embodiment of the present disclosure. Figure 5 As shown, the method includes:

[0071] S501, the search engine receives a search query input by a user;

[0072] S502: The search engine searches based on the query to obtain multiple related web page contents, and sorts the multiple related web page contents by keywords to obtain the top K search results with the highest relevance (TopK), where K is a positive integer.

[0073] S503: The search engine inputs the query input by the user and the TopK search results of the query into the distribution module;

[0074] S504: The distribution module determines a large language model applicable to the query and the TopK search results based on the query and the TopK search results; for example, the distribution module may include a distribution model;

[0075] S505: Organize a large model prompt using the query and TopK search results, and input the prompt into a large language module determined by the distribution module. The large language model generates search results based on the query and TopK search results, and returns the search results to the search engine.

[0076] S506: The search engine performs front-end rendering and display on the search results, and presents them to the user.

[0077] In summary, the search method proposed in the embodiments of this disclosure analyzes each large language model from the perspectives of cost, domain, demand, and performance, summarizing the search scenarios in which the model excels. Based on the user's search query and the TopK indexes with the highest matching degree with the query provided by the search engine, these are identified as features and input into the large language model. By adjusting the prompt, the ability of each model to solve the problem is determined. The model's ability to rank in each scenario is then evaluated and ranked, serving as the distribution benchmark for the distribution module.

[0078] During retrieval, the query and TopK search content are used as input, and the language model selected by the distribution benchmark is used as output. The distribution decision logic is designed, which includes the decision code and the distribution model. The distribution decision code performs simple distribution at the basic feature level, and the distribution model is used for complex feature extraction and distribution decisions based on complex features.

[0079] Based on the distribution benchmark, we collect fine-tuning training data for each model in its area of ​​expertise and train it in its own specialized scenarios, thereby enhancing its problem-solving capabilities in its own search scenarios.

[0080] The distribution module (or distribution model) involved in the embodiments of this disclosure serves as the post-processing logic for search engine search results and as the central node for each large language model. It distributes query traffic, enabling each downstream large model to perform its respective functions. After the large model obtains search results, the search engine collects the output of the large language model and performs post-processing such as structuring and front-end display.

[0081] The present disclosure also provides a search system. Figure 6 6 is a schematic diagram of the structure of a search system 600 according to an embodiment of the present disclosure, including a distribution module 601 and multiple large language models 602, each of which corresponds to a different search scenario:

[0082] The distribution module 601 is configured to receive a search statement and multiple search contents, and distribute the search statement and multiple search contents to a first model; wherein the first model is a large language model applicable to the search scenario corresponding to the search statement and multiple search contents;

[0083] The large language model 602 is configured to receive a search statement and multiple search contents from the distribution module, and generate search results based on the search statement and multiple search contents.

[0084] Figure 7 FIG. 7 is a schematic diagram of a search system 700 according to another embodiment of the present disclosure. In some implementations, the system further includes:

[0085] The search engine 703 is configured to receive a search statement, determine multiple search contents matching the search statement, and send the search statement and the multiple search contents to the distribution module.

[0086] In some implementations, the search engine 703 is further configured to:

[0087] receiving search results from the large language model;

[0088] The search results are rendered and displayed.

[0089] The search system proposed in the embodiment of the present disclosure changes the system architecture from a single upstream and downstream serial structure model to a star structure, that is, the distribution module is used as the distribution center of the peripheral model (large language model). From a macro perspective, the distribution module (such as the distribution model) increases the time consumption of the entire system, but the number of parameters in the distribution model is much smaller than that of the large language model. In addition, because the distribution model can distinguish between peripheral large language models of different sizes, large language models with smaller parameters, such as the 1B model, share part of the processing, and large language models with longer parameters, such as the 20B and 70B models, are no longer used to handle all search needs. Therefore, the average total time consumption of the system is reduced compared to a single large parameter model.

[0090] This application can adaptively select the most appropriate large model to address specific problems, implementing a "learn from many" strategy. In this way, this application addresses the limitations of a single large model in search engine applications. In various fields or user needs, this system can provide users with high-quality search results at a reasonable cost, thereby meeting their needs and improving the search experience.

[0091] In some implementations, the multiple large language models are pre-trained, and the multiple large language models are applicable to different search scenarios.

[0092] In some implementations, a method for determining a large language model applicable to each search scenario includes:

[0093] Obtain a verification data set covering multiple search scenarios, wherein the verification data set includes M groups of data, each group of data includes a search statement and multiple search contents, and each group of data corresponds to a search scenario; wherein M is a positive integer;

[0094] For each of the large language models, input the data in the verification dataset into the large language model, and score the search results output by the large language model to obtain M scoring results for the large language model;

[0095] Based on the M scoring results of each of the large language models, ranking the effects of the multiple large language models in each of the search scenarios;

[0096] Based on the ranking results, determine the large language model applicable to each search scenario.

[0097] In some implementations, after the multiple large language models are trained, they are fine-tuned, and the fine-tuning method includes:

[0098] Determining a first training set for fine-tuning the large language model, where the first training set includes search statements and search content corresponding to a search scenario applicable to the large language model;

[0099] The first training set is used to fine-tune the large language model applicable to the search scenario.

[0100] In some embodiments, the distribution module 601 includes a pre-trained distribution model, which is used to determine a corresponding search scenario based on a search statement and multiple search contents, as well as a large language model applicable to the search scenario.

[0101] In some embodiments, the distribution model is pre-trained, and the method of training the distribution model includes:

[0102] Determine a second training set, the second training set including a plurality of training data and labels corresponding to each training data; the training data including a search statement and a plurality of search contents, and the labels corresponding to the training data including search scenarios and a large language model corresponding to the search statement and the plurality of search contents;

[0103] The distribution model is trained using the second training set.

[0104] The present disclosure also provides a distribution device, including:

[0105] A receiving unit, configured to receive a search statement and multiple search contents;

[0106] A distribution unit is used to determine a first model corresponding to the search statement and multiple search contents, and distribute the search statement and multiple search contents to the first model, so that the first model generates search results based on the search statement and multiple search contents; wherein the first model is a large language model suitable for the search scenario corresponding to the search statement and multiple search contents.

[0107] For the description of specific functions and examples of each module and submodule of the device in the embodiment of the present disclosure, please refer to the relevant description of the corresponding steps in the above method embodiment, which will not be repeated here.

[0108] In the technical solution disclosed herein, the acquisition, storage and application of personal information of users involved are in compliance with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0109] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0110] Figure 8 A schematic block diagram of an example electronic device 800 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0111] like Figure 8 As shown, the electronic device 800 includes a computing unit 801, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 802 or a computer program loaded from a storage unit 808 into a random access memory (RAM) 803. In the RAM 803, various programs and data required for the operation of the electronic device 800 can also be stored. The computing unit 801, the ROM 802, and the RAM 803 are connected to each other via a bus 804. An input / output (I / O) interface 805 is also connected to the bus 804.

[0112] Multiple components in the electronic device 800 are connected to the I / O interface 805, including: an input unit 806, such as a keyboard, a mouse, etc.; an output unit 807, such as various types of displays, speakers, etc.; a storage unit 808, such as a magnetic disk, an optical disk, etc.; and a communication unit 809, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 809 allows the electronic device 800 to exchange data with other devices via a computer network such as the Internet and / or various telecommunication networks.

[0113] The computing unit 801 can be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 801 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units that run machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 801 performs the various methods and processes described above, such as the search method. For example, in some embodiments, the search method can be implemented as a computer software program that is tangibly contained in a machine-readable medium, such as the storage unit 808. In some embodiments, part or all of the computer program can be loaded and / or installed on the electronic device 800 via the ROM 802 and / or the communication unit 809. When the computer program is loaded into the RAM 803 and executed by the computing unit 801, one or more steps of the search method described above can be performed. Alternatively, in other embodiments, the computing unit 801 can be configured to perform the search method by any other appropriate means (e.g., by means of firmware).

[0114] Various embodiments of the systems and techniques described herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), system-on-chip systems (SOCs), programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include being implemented in one or more computer programs that are executable and / or interpreted on a programmable system comprising at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0115] The program code for implementing the method of the present disclosure can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device so that when the program code is executed by the processor or controller, the functions / operations specified in the flow chart and / or block diagram are implemented. The program code can be executed entirely on the machine, partially on the machine, as a stand-alone software package, partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0116] In the context of the present disclosure, a machine-readable medium can be a tangible medium that can contain or store a program for use by or in conjunction with an instruction execution system, device or equipment. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium can include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0117] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0118] The systems and techniques described herein can be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network (LAN), a wide area network (WAN), and the Internet.

[0119] A computer system may include a client and a server. The client and server are generally remote from each other and typically interact through a communication network. The client-server relationship arises through computer programs running on the respective computers and having a client-server relationship with each other. The server may be a cloud server, a server in a distributed system, or a server integrated with a blockchain.

[0120] It should be understood that the various forms of the processes shown above can be used to reorder, add, or delete steps. For example, the steps described in this disclosure can be performed in parallel, sequentially, or in a different order, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved. This is not limited herein.

[0121] The above specific embodiments do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art will appreciate that various modifications, combinations, sub-combinations, and substitutions may be made based on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the principles of this disclosure shall be included within the scope of protection of this disclosure.

Claims

1. A search method, comprising: Pre-training multiple large language models; obtaining a verification dataset covering multiple search scenarios, wherein the verification dataset includes M groups of data, each group of data corresponding to a search scenario; wherein M is a positive integer; for each of the large language models, inputting the data in the verification dataset into the large language model, and scoring the search results output by the large language model to obtain M scoring results for the large language model; and determining the large language model applicable to each search scenario based on the M scoring results of each of the large language models; Inputting a search statement and a plurality of search contents into a distribution module, and the distribution module distributing the search statement and the plurality of search contents to a first model; wherein the first model is a large language model applicable to the search scenario corresponding to the search statement and the plurality of search contents; and Generate search results based on the search statement and multiple search contents through the first model.

2. The method according to claim 1, before inputting the search statement and the plurality of search contents into the distribution module, further comprising: Entering the search statement into a search engine; Determining the plurality of search contents according to the output content of the search engine; The multiple search contents match the search statement.

3. The method according to claim 2, further comprising: Returning the search results to the search engine; The search results are rendered and displayed through the search engine.

4. The method according to any one of claims 1 to 3, wherein: Each set of data includes a search statement and multiple search contents; The determining of the large language model applicable to each search scenario based on the M scoring results of each of the large language models includes: Based on the M scoring results of each of the large language models, ranking the effects of the multiple large language models in each of the search scenarios; Based on the ranking results, determine the large language model applicable to each search scenario.

5. The method according to claim 4, further comprising: Fine-tune the large language model applicable to each search scenario. The fine-tuning method includes: Determining a first training set for fine-tuning, where the first training set includes a search statement and search content corresponding to the search scenario; The first training set is used to fine-tune the large language model applicable to the search scenario.

6. The method according to any one of claims 1 to 3, wherein: The distribution module includes a pre-trained distribution model, which is used to determine a corresponding search scenario based on a search statement and multiple search contents, and a large language model suitable for the search scenario.

7. The method according to claim 6, further comprising pre-training the distribution model, wherein the training method comprises: Determine a second training set, where the second training set includes a plurality of training data and a label corresponding to each training data; The training data includes a search statement and multiple search contents, and the labels corresponding to the training data include search scenarios and large language models corresponding to the search statement and multiple search contents; The distribution model is trained using the second training set.

8. A search system comprising a distribution module and multiple large language models, each of which corresponds to a different search scenario: The distribution module is used to receive a search statement and multiple search contents, and distribute the search statement and multiple search contents to the first model; wherein, The first model is a large language model applicable to the search scenario corresponding to the search statement and the multiple search contents; The large language model is configured to receive a search statement and a plurality of search contents from the distribution module, and generate search results based on the search statement and the plurality of search contents; The multiple large language models are pre-trained and are applicable to different search scenarios. The method for determining the large language model applicable to each search scenario includes: obtaining a verification data set covering multiple search scenarios, the verification data set including M groups of data, each group of data corresponding to a search scenario; M is a positive integer; for each of the large language models, the data in the verification data set is input into the large language model respectively, and the search results output by the large language model are scored to obtain M scoring results of the large language model; based on the M scoring results of each of the large language models, the large language model applicable to each search scenario is determined.

9. The system according to claim 8, wherein: The system further comprises: The search engine is configured to receive a search statement, determine a plurality of search contents matching the search statement, and send the search statement and the plurality of search contents to the distribution module.

10. The system according to claim 9, wherein: The search engine is also used to: receiving search results from the large language model; The search results are rendered and displayed.

11. The system according to any one of claims 8 to 10, wherein: Each set of data includes a search statement and multiple search contents; Determining the large language model applicable to each search scenario based on the M scoring results of each of the large language models includes: ranking the effects of multiple large language models in each of the search scenarios based on the M scoring results of each of the large language models; Based on the ranking results, determine the large language model applicable to each search scenario.

12. The system according to claim 11, wherein After the training of the multiple large language models is completed, they are fine-tuned. The fine-tuning method includes: Determining a first training set for fine-tuning the large language model, where the first training set includes search statements and search content corresponding to a search scenario applicable to the large language model; The first training set is used to fine-tune the large language model applicable to the search scenario.

13. The system according to any one of claims 8 to 10, wherein: The distribution module includes a pre-trained distribution model, which is used to determine a corresponding search scenario based on a search statement and multiple search contents, and a large language model suitable for the search scenario.

14. The system according to claim 13, wherein: The distribution model is pre-trained, and the method of training the distribution model includes: Determine a second training set, the second training set including a plurality of training data and labels corresponding to each training data; the training data including a search statement and a plurality of search contents, and the labels corresponding to the training data including search scenarios and a large language model corresponding to the search statement and the plurality of search contents; The distribution model is trained using the second training set.

15. A dispensing device comprising: A receiving unit, configured to receive a search statement and multiple search contents; a distribution unit, configured to determine a first model corresponding to the search statement and the plurality of search contents, and distribute the search statement and the plurality of search contents to the first model, so that the first model generates search results based on the search statement and the plurality of search contents; wherein the first model is a large language model applicable to the search scenario corresponding to the search statement and the plurality of search contents; The large language model is pre-trained, and different large language models are suitable for different search scenarios; The method for determining the large language model applicable to each search scenario includes: obtaining a verification data set covering multiple search scenarios, the verification data set including M groups of data, each group of data corresponding to a search scenario; M is a positive integer; for each of the large language models, the data in the verification data set is input into the large language model respectively, and the search results output by the large language model are scored to obtain M scoring results of the large language model; based on the M scoring results of each of the large language models, the large language model applicable to each search scenario is determined.

16. An electronic device comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 7.

17. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to cause the computer to execute the method according to any one of claims 1 to 7.

18. A computer program product comprising a computer program, which, when executed by a processor, implements the method according to any one of claims 1 to 7.

Images