Search system updating method and device based on large language model, equipment and storage medium

Abstract

The application provides a large language model-based search system updating method and device, equipment and a storage medium. The large language model is used to predict the category, identify the entity, correct the word and infer the synonym of a search term with low accuracy, and the rule base of the search term is updated based on the large model prediction result, so that the rule base is formally updated when the accuracy of the search result of the search term after the gray update is improved. Through the method of the application, the operation efficiency of the search term in the search engine can be improved without modifying the existing prediction logic of the search system, the operation cost is reduced compared with manual operation of the search term, and the accuracy of the search engine operation is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of search engines, in particular to a search system updating method and device based on a large language model, equipment and a storage medium. BACKGROUND

[0002] A search engine is a core portal for users to obtain information and has been widely used in various scenarios such as e-commerce and content platforms. Taking the e-commerce scenario as an example, the search engine can perform category prediction, error correction and synonym search on the search term input by the user based on a rule library, obtain the processed search term, and then perform retrieval in the product library based on the processed search term to obtain and recall the products related to the search term input by the user.

[0003] In order to improve the search accuracy of the search engine, the rule library needs to be operated and maintained. In the prior art, the rule library is generally manually operated and maintained, including correcting the category prediction results and synonyms of the search term, and performing error correction on the search term.

[0004] However, the manual operation of the rule library is inefficient, and there is a high operation cost in some scenarios due to the technical threshold. SUMMARY

[0005] The present application aims to solve the problem of manual operation of the rule library of the search engine in the prior art by providing a search system updating method and device based on a large language model, equipment and a storage medium.

[0006] To achieve the above-mentioned purpose, the technical solution adopted by the present application is as follows:

[0007] In a first aspect, the present application provides a search system updating method based on a large language model, which comprises:

[0008] Obtaining a search term to be operated, and determining a query understanding (QU) result of the search term to be operated based on a rule library by a search system, wherein the QU result comprises at least one of the following: an initial category prediction result, an initial entity recognition result, an initial word correction result and an initial synonym result;

[0009] Performing QU prediction on the search term to be operated based on a large language model to obtain a large model prediction result, wherein the large model prediction result comprises at least one of the following: a model category prediction result, a model entity recognition result, a model word correction result and a model synonym result;

[0010] According to the large model prediction result, the rule library in the search system is modified to obtain a new rule library, the new rule library is pushed to a preset internal test account, and the search system of the internal test account is used as a gray engine system.

[0011] Based on the search results of the grayscale engine system for the search terms to be operated, the rule base in the search system is officially updated.

[0012] Optionally, the step of formally updating the rule base in the search system based on the search results of the grayscale engine system for the search term to be operated includes:

[0013] The search terms to be operated are input into the grayscale engine system to obtain search results;

[0014] The accuracy of the search results is determined based on the large language model.

[0015] The accuracy rate will determine whether to formally update the rule base.

[0016] If so, the rule base in the search system will be officially updated based on the prediction results of the large model.

[0017] Optionally, the process of performing QU prediction on the search terms to be operated based on a large language model to obtain the model category prediction result includes:

[0018] The search term to be operated, the initial category prediction result, and the first relevance judgment criterion are filled into the first prompt term template to obtain the first prompt statement. The first prompt statement is then input into the large language model to delete the category prediction results that are not related to the search term to be operated from the initial category prediction results, thereby obtaining the first category prediction result.

[0019] The search terms to be operated are input into the large language model to obtain the second category prediction results of the search terms to be operated;

[0020] The search term to be operated, the second category prediction result, and the second relevance judgment criterion are filled into the second prompt term template to obtain the second prompt statement. The first prompt statement is then input into the large language model to filter the category prediction results related to the search term to be operated from the second category prediction results to obtain the third category prediction result.

[0021] The prediction results of the first category and the prediction results of the third category are combined into the prediction results of the model category.

[0022] Optionally, the process of performing QU prediction on the search term to be operated based on a large language model to obtain the model entity recognition result includes:

[0023] The search terms to be operated and the initial entity recognition results are filled into the third prompt word template to obtain the third prompt statement. The third prompt statement is then input into the large language model to correct the initial entity recognition results and obtain the model entity recognition results.

[0024] Optionally, the process of performing QU prediction on the search terms to be operated based on a large language model to obtain model word correction results includes:

[0025] If the accuracy of the search term to be operated is less than the preset accuracy correction threshold, the search term to be operated is filled into the fourth prompt word template to obtain the fourth prompt statement. The fourth prompt statement is then input into the large language model to perform error correction processing on the search term to be operated, and the model word error correction result is obtained.

[0026] Optionally, the process of performing QU prediction on the search terms to be operated based on a large language model to obtain model synonym results includes:

[0027] The search terms to be operated and the initial synonym results are filled into the fifth prompt word template to obtain the fifth prompt statement. The fifth prompt statement is then input into the large language model to obtain the model synonym results.

[0028] Optionally, the step of performing grayscale updates on the rule base in the search system based on the prediction results of the large model to obtain a grayscale engine system includes:

[0029] Based on the prediction results of the large model, the rule base in the search system is modified to obtain a new rule base. The new rule base is then pushed to a preset internal test account, and the search system of the internal test account is used as the grayscale engine system.

[0030] Optionally, the step of formally updating the rule base in the search system based on the prediction results of the large model includes:

[0031] Based on the category prediction results of the model, the category information corresponding to the search terms to be operated in the rule base is officially updated;

[0032] Based on the entity recognition results of the model, the entity information corresponding to the search term to be operated in the rule base is officially updated;

[0033] Based on the error correction results of the model words, the error correction words corresponding to the search terms to be operated in the rule base are officially updated.

[0034] Based on the synonym results of the model, the synonym information corresponding to the search term to be operated in the rule base is officially updated.

[0035] Secondly, embodiments of this application also provide a search system update apparatus based on a large language model, comprising:

[0036] The initial prediction module is used to obtain the search terms to be operated, and the search system determines the query understanding (QU) results of the search terms to be operated based on the rule base. The QU results include at least one of the following: initial category prediction results, initial entity recognition results, initial word correction results, and initial synonym results.

[0037] The model prediction module is used to perform QU prediction on the search terms to be operated based on a large language model to obtain the large model prediction results. The large model prediction results include at least one of the following: model category prediction results, model entity recognition results, model word correction results, and model synonym results.

[0038] The gray-scale update module is used to modify the rule base in the search system according to the prediction results of the large model, obtain a new rule base, push the new rule base to the preset internal test account, and use the search system of the internal test account as the gray-scale engine system.

[0039] The formal update module is used to formally update the rule base in the search system based on the search results of the grayscale engine system for the search terms to be operated.

[0040] Thirdly, embodiments of this application also provide an electronic device, including: a processor, a storage medium, and a bus, wherein the storage medium stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor communicates with the storage medium via the bus, and the processor executes the machine-readable instructions to perform the steps of a search system update method based on a large language model as described in any one of the first aspects.

[0041] Fourthly, embodiments of this application also provide a computer-readable storage medium storing a computer program, which, when executed by a processor, performs the steps of a search system update method based on a large language model as described in any one of the first aspects.

[0042] The beneficial effects of this application are: by using a large language model to correct and optimize the QU results of the search terms to be operated, the automated operation of the rule base can be achieved without changing the existing architecture of the search system. Furthermore, compared with manual operation, automated operation of the rule base using a large language model has the advantages of low technical threshold, low cost, and strong versatility, which not only reduces the operating cost of the rule base but also improves its operating efficiency. By performing a gray-scale update of the rule base before a formal update, it is possible to avoid affecting normal user experience and further improve the stability of updates to the search system based on the large language model.

[0043] To make the above-mentioned objectives, features and advantages of this application more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings. Attached Figure Description

[0044] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0045] Figure 1 A flowchart of a search system update method based on a large language model provided in an embodiment of this application is shown;

[0046] Figure 2 This document illustrates a flowchart of a formal update of a search system according to an embodiment of this application.

[0047] Figure 3 This document illustrates a flowchart of an embodiment of the present application for obtaining model category prediction results;

[0048] Figure 4 This document illustrates a flowchart of a formal update of a rule base, as provided in an embodiment of this application.

[0049] Figure 5 This paper illustrates an overall flowchart of a search system update method based on a large language model, as provided in an embodiment of this application.

[0050] Figure 6 This illustration shows a schematic diagram of the structure of a search system update device based on a large language model according to an embodiment of this application;

[0051] Figure 7 A schematic diagram of the structure of an electronic device provided in an embodiment of this application is shown. Detailed Implementation

[0052] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. The components of the embodiments of this application described and shown in the accompanying drawings can generally be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely represents selected embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.

[0053] It should be noted that the term "comprising" will be used in the embodiments of this application to indicate the presence of the features declared thereafter, but does not exclude the addition of other features.

[0054] Maintaining the search engine's rule base is crucial for ensuring search accuracy. The rule base in a search engine can be understood as a collection that stores judgment logic, execution standards, and relationships in a structured form. After a search term is entered into the search engine, the search engine can perform a retrieval based on the rule base and then determine the search results corresponding to the search term.

[0055] Taking e-commerce as an example, the rule base in a search engine can store a structured set of clear logic, such as how to determine the intent of search terms, how to filter products, and how to sort results. After a user enters a search term, the search engine can retrieve products from the product database according to the search logic stored in the rule base, and then obtain multiple products related to the search term and return them to the user.

[0056] The search logic in the rule base is not static. As business scenarios expand, user search habits change, or existing terms change, the search logic stored in the rule base also needs to be modified accordingly. Therefore, to improve the search accuracy of the search engine, the rule base needs to be operated and maintained, including correcting the category prediction results and synonyms for search terms, and performing error correction on search terms.

[0057] Taking e-commerce platforms as an example, if an e-commerce platform adds a product category called "camping equipment", it needs to add the category of "camping equipment" to the rule base, add related synonyms, and collect common incorrect words entered by users during the operation of the search engine to form error correction rules.

[0058] In existing technologies, this is generally achieved through manual operation of the rule base. When a new search term appears, the operations staff adds the new search term to the rule base, along with corresponding information such as categories and synonyms, and formulates error correction rules based on commonly mis-inputted words from historical operations.

[0059] However, search engines typically handle hundreds of millions of search terms, making it impossible for manual intervention to cover all incorrect words, synonyms, and category misclassification scenarios. Furthermore, new search terms emerge rapidly, leading to delays in manual maintenance. Operating a rule base is not simply about data entry; it requires understanding industry characteristics and user intent. For highly specialized fields, such as healthcare, rule base operation and maintenance require medical professionals, presenting a high technical barrier and necessitating the formation of dedicated teams, resulting in significant labor costs.

[0060] Based on this, this application proposes a search system update method based on a Large Language Model (LLM). The method uses a LLM to perform category prediction, entity recognition, word correction, and synonym reasoning on search terms with low accuracy. The rule base for the search terms is then updated in a gray-scale manner based on the LLM prediction results. A formal update of the rule base is then performed when the accuracy of the search results for the search terms improves after the gray-scale update. This method improves the efficiency of search term management in search engines without modifying the existing prediction logic of the search system. Compared to manual search term management, it reduces operating costs and improves accuracy.

[0061] Next, combine Figure 1 This application describes the update method for a search system based on a large language model. The execution entity of this method can be an electronic device deployed with a large language model. The search system and the large language model can be deployed on the same device or on different devices with communication connections; no limitation is imposed here. Figure 1 As shown, the method includes:

[0062] S101. Obtain the search terms to be operated, and the search system determines the query understanding (QU) results of the search terms to be operated based on the rule base.

[0063] Optionally, the search terms in the search system can be sorted by search frequency or popularity to obtain the top N search terms. The accuracy of each search term can be calculated based on the top N search terms and the search results of each search term. Search terms with an accuracy rate lower than a preset accuracy threshold can be identified as search terms to be operated.

[0064] Alternatively, the top N third-party search terms can be obtained from third-party search software, compared with the search terms in the search system, and the third-party search terms not included in the search system can be used as search terms to be operated.

[0065] In one possible implementation, the search term can be input into the search system to obtain the search results output by the search system. The search term and the search results for the search term can then be input into a large language model, which will evaluate the search results and obtain the accuracy of the search term and the search results. If the accuracy of the search term is lower than a preset accuracy threshold, the search term will be considered as a search term to be operated.

[0066] The rule base can be the structured search logic used in the search system, including category prediction logic, entity recognition logic, word correction logic, and synonym reasoning logic. The QU result includes at least one of the following: initial category prediction result, initial entity recognition result, initial word correction result, and initial synonym result.

[0067] Optionally, the initial category prediction result indicates the category to which the search term to be operated belongs. For example, the initial category prediction result for the search term "solid wood desktop computer desk for home study and writing" could be "computer desk". The initial entity recognition result indicates the named entities with specific meanings included in the search term to be operated, such as brand, category, model, etc. For example, the entity recognition result for "desktop Apple computer" could be "Apple - brand term, computer - category term". The initial word correction result indicates the result of error correction processing on the search term to be operated. For example, the initial word correction result for "pan-shaped computer" is "Apple computer". The initial synonym result includes synonyms or near-synonyms of the search term to be operated. For example, the initial synonym result for "Apple computer" could be "laptop".

[0068] In one possible implementation, the search logic in the search system can be implemented through models. For example, the search system includes a category prediction model, an entity recognition model, a word correction model, and a synonym reasoning model. Based on a rule base, the query understanding (QU) results of the search terms to be operated are determined. This can be achieved by inputting the search terms to be operated into the various models of the search system to obtain the initial category prediction results output by the category prediction model, the initial entity recognition results output by the entity recognition model, the initial word correction results output by the word correction model, and the initial synonym results output by the synonym reasoning model.

[0069] S102. Based on the large language model, perform QU prediction on the search terms to be operated to obtain the prediction results of the large model.

[0070] QU prediction based on a large language model can be achieved by inputting the search terms to be operated into the large language model, which then performs category prediction, entity recognition, word correction, and synonym reasoning on the search terms.

[0071] In one possible implementation, multiple prompt word templates can be pre-defined in the large language model. Each prompt word template is used for category prediction, entity recognition, word correction, and synonym reasoning, respectively. The search term to be operated and its QU results are filled into different prompt word templates to obtain multiple prompt word statements. These prompt word statements are then input into the large language model, which optimizes the QU results to obtain the large model's prediction results.

[0072] The large-scale model prediction results include at least one of the following: model category prediction results, model entity recognition results, model word correction results, and model synonym results. The model category prediction results indicate the category to which the search term to be operated belongs after the large language model optimizes the initial category prediction results. The model entity recognition results indicate the entity types included in the search term to be operated after the large language model optimizes the initial entity recognition results. The model word correction results represent the result of error correction processing on the search term to be operated. The model synonym results represent synonyms or near-synonyms of the search term to be operated obtained after the large language model optimizes the initial synonym results.

[0073] It should be noted that when the large language model and search system perform QU prediction, the order in which category prediction, entity recognition, word correction, and synonym reasoning are performed for the search terms to be operated is not limited. For example, the order can be word correction, entity recognition, category prediction, and synonym reasoning, or other orders. Similarly, the order in which the large language model performs QU prediction can be the same as or different from the order in which the search system determines the QU results based on the rule base; no restrictions are placed here.

[0074] S103. Modify the rule base in the search system based on the prediction results of the large model to obtain a new rule base. Push the new rule base to the preset internal test account and use the search system of the internal test account as the grayscale engine system.

[0075] Optionally, the rule base in the search system can be modified based on the prediction results of the large model, including adding, deleting, or modifying search logic. For example, if a new category "tablet computer" is added in the category prediction results of the large model, then a new search rule can be added: "When searching for Apple computers, display content related to tablet computers."

[0076] The users in the search system include regular users and beta testers. The search system's rule base can be updated gradually through beta testers within the search engine. Beta testers can be accounts used for gradual updates and can access the large language model. To avoid impacting other users currently using the search system during rule base updates, after obtaining the new version of the beta engine system, the search system can be accessed through the beta tester account, and the aforementioned step S104 can be executed.

[0077] Optionally, the rule base in the search system can be updated in a phased manner, which can be done by optimizing the QU results in the search system based on the prediction results of the large model. For example, the prediction results of the large model are not pushed to all users, but are first pushed to users who are conducting internal testing.

[0078] The grayscale engine system can be the search system used by beta testers. It's important to note that the grayscale engine system and the search system can be the same system in a real-world scenario. The difference lies in the rule base: the search system's rule base is publicly available to all users, while the grayscale engine system's rule base is an optimized version, only accessible to beta testers.

[0079] S104. Based on the search results of the grayscale engine system for the search terms to be operated, the rule base in the search system is officially updated.

[0080] The grayscale engine system can test the effect of the optimized rule base. By comparing the search results of the grayscale engine system for the search terms to be operated with the search system for the same search terms, it can be determined whether the optimized rule base has improved the accuracy of the search terms to be operated. If the accuracy of the search terms to be operated has improved, the rule base in the search system can be officially updated.

[0081] In one possible implementation, after obtaining the model category prediction result based on the large language model, the above steps S103-S104 can be executed to optimize the operation of the search logic related to category prediction in the search system. After obtaining the model entity recognition result, model word correction result, and model synonym result, the above steps S103-S104 can be executed in sequence to optimize the operation of the search logic related to entity recognition, word correction, and synonym reasoning in the search system, respectively.

[0082] In this embodiment, the QU results for the search terms to be operated are corrected and optimized using a large language model, enabling automated operation of the rule base without altering the existing architecture of the search system. Compared to manual operation, automated rule base operation using a large language model has advantages such as lower technical barriers, lower costs, and greater versatility, reducing both the operational costs and efficiency of the rule base. By performing a phased update of the rule base before a formal update, disruption to normal user experience can be avoided, further enhancing the stability of updates to the search system based on the large language model.

[0083] The following explains the steps for formally updating the rule base in the search system based on the search results from the gray-scale engine system for the search terms to be operated. Figure 2 As shown, step S104 above includes:

[0084] S201. Input the search terms to be operated into the grayscale engine system to obtain search results.

[0085] The grayscale engine system includes a rule base that is updated and optimized based on the prediction results of large models. For example, the search logic for the search terms to be operated is optimized based on the model category prediction results, model entity recognition results, model word correction results, and model synonym results to obtain the optimized rule base.

[0086] Input the search terms to be operated into the grayscale engine system, and the grayscale engine system can perform retrieval based on the search logic in the updated and optimized rule base to obtain the search results for the search terms to be operated.

[0087] S202. Determine the accuracy of search results based on a large language model.

[0088] Optionally, the search terms to be operated and their search results can be filled into a preset suggestion term template to obtain a suggestion term statement for accuracy judgment. The suggestion term statement can then be input into a large language model, which will calculate the accuracy of the search terms to be operated according to a preset strategy.

[0089] Search results for search terms to be optimized include various types, such as text search results and image search results. Taking an e-commerce scenario as an example, the search terms to be optimized are the search terms entered by the user, and the search results can be product titles and product images.

[0090] S203. Determine whether to formally update the rule base based on the accuracy rate.

[0091] If the accuracy of the search results from the grayscale engine system is greater than that of the search results from the search system, it indicates that the accuracy of the search system has improved after the rule base optimization and update, and it can be determined to formally update the rule base.

[0092] In another possible implementation, if the accuracy of the search results from the grayscale engine system is greater than a preset accuracy threshold, it can also be determined to formally update the rule base.

[0093] S204. If so, the rule base in the search system will be officially updated based on the prediction results of the large model.

[0094] Optionally, a formal update to the rule base in the search system can be made by retaining the rule base in the grayscale engine system and making it effective for all users.

[0095] In one possible implementation, the search logic related to category prediction in the search system can first be updated in a grayscale manner based on the model's category prediction results, resulting in a grayscale engine system. In this grayscale engine system, only the search logic for category prediction is optimized. Then, steps S201-S204 are executed to formally update the search logic for category prediction in the search system. Similarly, grayscale and formal updates are performed sequentially based on the model entity recognition results, model word correction results, and model synonym results to achieve a comprehensive formal update of the search terms to be operated.

[0096] In this embodiment of the application, by calculating the accuracy rate of the search results of the grayscale engine system and determining whether to make a formal update based on the accuracy rate, the automated operation of the search logic related to the search terms to be operated in the search system can be realized.

[0097] The following explains the process of using a large language model to predict the search terms for operation based on QU (Queries and Categories) and obtain the model category prediction results. Figure 3 As shown, the above step S102 includes:

[0098] S301. Fill the first prompt word template with the search term to be operated, the initial category prediction result and the first relevance judgment criterion to obtain the first prompt statement, and input the first prompt statement into the large language model to delete the category prediction results that are not related to the search term to be operated from the initial category prediction results, and obtain the first category prediction result.

[0099] The first relevance criterion is used to determine whether the relevance to the search term to be operated is relevant or irrelevant. This criterion can be pre-filled into the first suggestion term template. When the relevance to the search term to be operated is considered relevant, it can mean that the relevance is related to the category of the search term.

[0100] Optionally, the first category prediction result can be the remaining category prediction result obtained after deleting category prediction results that are irrelevant to the search terms to be operated from the initial category prediction result.

[0101] For example, the first prompt word template could be "Please delete the category prediction results of %search term to be operated% that are not related to %search term to be operated% according to the first relevance judgment criteria. The category prediction results include: %initial category prediction results%".

[0102] For example, if the search term to be operated is "Apple computer", the initial category prediction results include categories such as "Apple", "Tablet", and "Desktop computer". It can be seen that the category "Apple" is an irrelevant category prediction result. After filling the search term to be operated and the initial category prediction results into the first prompt term template, the first prompt statement is obtained: "Please delete the category prediction results that are not related to %Apple computer% in the initial category prediction results of %Apple computer% according to the first relevance judgment standard, and output the remaining relevant category prediction results. The initial category prediction results of %Apple computer% include: %Apple, Tablet, Desktop computer%". The first prompt statement is then input into the large language model, which can then identify and delete the irrelevant category prediction result "Apple", and take "Tablet" and "Desktop computer" as the first category prediction results.

[0103] S302. Input the search terms to be operated into the large language model to obtain the second category prediction results of the search terms to be operated.

[0104] Among them, the second category prediction result can be the category prediction result obtained by the large language model to predict the category of the search terms to be operated.

[0105] In one possible implementation, the search terms to be operated can be filled into the category prediction prompt template to obtain the category prediction prompt statement. The category prediction prompt statement is then input into a large language model, which performs category prediction to obtain the second category prediction result of the search terms to be operated.

[0106] S303. Fill the second prompt word template with the search term to be operated, the second category prediction result and the second relevance judgment criterion to obtain the second prompt statement, and input the first prompt statement into the large language model to filter the category prediction results related to the search term to be operated from the second category prediction results to obtain the third category prediction result.

[0107] The second relevance criterion is used to determine whether the category prediction results are related to the search term to be operated. This second relevance criterion can be pre-filled into the second suggestion term template. If the second category prediction result is related to the search term to be operated, then the second category prediction result is used as the third category prediction result. For example, if the search term to be operated is "Apple computer," and the second category prediction results include the category "laptop," which is related to the search term, then "laptop" can be used as the third category prediction result.

[0108] For example, the second prompt word template could be "Please filter the category prediction results related to the %search term to be operated% in the category prediction results according to the second relevance judgment criteria. The category prediction results include: %second category prediction results%".

[0109] S304. Combine the prediction results of the first category and the prediction results of the third category into the prediction results of the model category.

[0110] After obtaining the first category prediction result by deleting category prediction results that are irrelevant to the search term to be operated from the initial category prediction results through the large language model, and supplementing the third category prediction result that is relevant to the search term to be operated, the first category prediction result and the third category prediction result can be merged. For example, the third category prediction result can be added to the first category prediction result to obtain the model category prediction result.

[0111] In this embodiment of the application, the initial category prediction results are corrected by using a large language model, irrelevant category prediction results are deleted, and relevant category prediction results are added, which can realize the automated operation of category prediction for search terms to be operated.

[0112] The process described above, which uses a large language model to predict the target search term and obtain the model entity recognition result, includes:

[0113] The search terms to be operated and the initial entity recognition results are filled into the third prompt word template to obtain the third prompt statement. The third prompt statement is then input into the large language model to correct the initial entity recognition results and obtain the model entity recognition results.

[0114] The third prompt statement is used to instruct the large language model to perform entity recognition on the search terms to be operated, and to correct the initial entity recognition results based on the entity recognition results of the large language model.

[0115] For example, the third prompt statement could be "Please perform entity recognition on %the search term to be operated% and correct the initial entity recognition result based on the entity recognition result. The initial entity recognition result includes: %initial entity recognition result%".

[0116] For example, the initial entity recognition result for the search term "Apple computer" is "Apple - category term, computer - category term". The initial entity recognition result and the search term to be operated are filled into the third prompt term template to obtain the third prompt statement. The third prompt statement is then input into the large language model. The large language model can correct the initial entity recognition result to obtain the model entity recognition result "Apple - brand term, computer - category term".

[0117] The process described above, which uses a large language model to perform QU prediction on the search terms to be operated and obtains the model word correction results, includes:

[0118] If the accuracy of the search term to be operated is less than the preset accuracy correction threshold, the search term to be operated is filled into the fourth prompt word template to obtain the fourth prompt statement. The fourth prompt statement is then input into the large language model to perform error correction processing on the search term to be operated, and the model word error correction result is obtained.

[0119] Optionally, the accuracy of the search term to be operated can be the degree of relevance between the search results of the search term to be operated and the search term to be operated.

[0120] It is worth noting that, generally speaking, if the search terms to be operated contain spelling errors or typos, the accuracy of the search terms will be very low. Therefore, an accuracy correction threshold can be set in advance, and word correction processing can be performed when the accuracy of the search terms to be operated is less than the accuracy correction threshold.

[0121] The fourth prompt statement is used to instruct the large language model to perform word correction processing on the search terms to be operated, and to obtain the word correction results of the model.

[0122] For example, the fourth prompt could be "Please infer the user's true search intent based on the search term %search term to be operated% and correct %search term to be operated% based on the user's true search intent".

[0123] For example, if the accuracy of the search results for the search term "pan computer" is still lower than the accuracy correction threshold after word correction processing, the search term can be filled into the fourth suggestion word template to obtain the fourth suggestion statement. The fourth suggestion statement is then input into the large language model, which can perform word correction processing on the search term to obtain the model word correction result "Apple computer".

[0124] The process described above, which uses a large language model to predict the search terms to be operated and obtain the model's synonym results, includes:

[0125] The search terms to be operated and the initial synonym results are filled into the fifth prompt word template to obtain the fifth prompt statement. The fifth prompt statement is then input into the large language model to obtain the model's synonym results.

[0126] For example, the fifth prompt statement could be: "Please expand the synonyms based on the search term %search term to be operated% and the initial synonym results %initial synonym results% to determine more synonyms or near-synonyms of the search term to be operated, and supplement %initial synonym results% to output the supplemented initial synonym results."

[0127] For example, the initial synonym results for the search term "Apple computer" include "tablet computer". The search term and the initial synonym results can be filled into the fifth suggestion word template to obtain the fifth suggestion statement. The fifth suggestion statement is then input into the large language model, which can expand the search term with synonyms to obtain the model's synonym results "desktop computer" and "laptop computer".

[0128] The following explains the process of formally updating the rule base in the search system based on the prediction results of the large model. Figure 4 As shown, step S104 above includes:

[0129] S401. Based on the model category prediction results, formally update the category information corresponding to the search terms to be operated in the rule base.

[0130] The category information can be the search logic for the search terms to be operated, as predicted by the category prediction in the rule base. A formal update to the category information can either use the search logic from the model's category prediction results as the search logic for the rule base's category predictions, or modify the search logic for the rule base's category predictions based on the model's category prediction results, and then apply the changes globally to make them visible to all users.

[0131] For example, if the category "desktop computer" is added to the category prediction results in the large model prediction results, then a new search rule will be added: "When searching for Apple computers, display content related to desktop computers".

[0132] S402. Based on the model entity recognition results, formally update the entity information corresponding to the search terms to be operated in the rule base.

[0133] Optionally, entity information can be the search logic for entity identification in the rule base of the search terms to be operated.

[0134] For example, if the type of "apple" in the entity recognition result of the large model prediction is changed from "category word" to "brand word", then a new search rule is added: "When searching for Apple computers, display content related to Apple brand computers".

[0135] S403. Based on the model word correction results, formally update the corresponding correction words in the rule base for the search terms to be operated.

[0136] S404. Based on the model's synonym results, formally update the synonym information corresponding to the search terms to be operated in the rule base.

[0137] Among them, synonym information can be the search logic of expanding synonyms in the rule base for the search terms to be operated.

[0138] For example, if the word correction result in the large model prediction result is changed from "flat-pan computer" to "Apple computer", then a new search rule is added: "When searching for flat-pan computer, display content related to Apple computer".

[0139] In another example, if the model's synonym results for "Apple computer" in the large model's prediction results are "laptop" and "tablet", then a new search rule is added: "When searching for Apple computer, display content related to laptop and tablet".

[0140] Next, combine Figure 5 The flowchart shown illustrates the overall process of the search system update method based on a large language model in this application, using an e-commerce scenario as an example.

[0141] The top N most frequently searched terms and their search results can be input into a large language model. The large language model will then evaluate the accuracy of each search term and identify those with an accuracy rate below a preset accuracy threshold as search terms to be implemented.

[0142] For each search term to be operated, category prediction, entity recognition, word error correction, and synonym expansion can be performed sequentially.

[0143] Taking the search term "flat-bottomed computer" as an example, the accuracy rate of the search term output by the large language model is 40%. The QU result of the search term is "word correction: flat-bottomed fruit --> apple, category prediction result - fresh fruit, vegetables and fruits, entity recognition result - apple, synonyms - laptop".

[0144] The category prediction results are optimized using a large language model to obtain the model category prediction result "Category Prediction Result - Digital Products, Computers". A gray-scale update is then performed based on the model category prediction result to obtain the gray-scale engine system. The search is then re-performed in the gray-scale engine system based on the model category prediction result. The accuracy of the search results calculated by the large language model is 60%, which is an improvement over the initial accuracy. Therefore, the search system can be officially updated based on the model category prediction result.

[0145] By performing word correction processing on the search terms to be operated using a large language model, the corrected word result "Apple computer" is obtained. Then, a gray-scale update is performed based on the corrected word result to obtain a gray-scale engine system. The search is then re-performed in the gray-scale engine system based on the corrected word result. The accuracy of the search results is calculated to be 70% by the large language model, which is an improvement over the 60% accuracy in the previous step. Therefore, the search system can be officially updated based on the corrected word result.

[0146] By using a large language model to expand the search terms to the target audience using synonyms, the model's synonym result "laptop" is obtained. Then, a gray-scale update is performed based on the model's synonym result to obtain a gray-scale engine system. In the gray-scale engine system, a new search is performed based on the model's synonym result. The accuracy of the search results calculated by the large language model is 80%, which is an improvement over the 70% accuracy in the previous step. Therefore, the search system can be officially updated based on the model's synonym result.

[0147] The search term to be operated is identified by a large language model, and the model entity recognition result is "computer". The grayscale update is performed based on the model entity recognition result to obtain the grayscale engine system. The search is re-examined in the grayscale engine system based on the model entity recognition result. The accuracy of the search results is calculated to be 90% by the large language model, which is an improvement over the 80% accuracy in the previous step. The search system can then be officially updated based on the model entity recognition result.

[0148] It is worth noting that the order of category prediction, entity recognition, word correction, and synonym expansion can be changed, and this application does not impose any restrictions on the specific order.

[0149] Based on the same inventive concept, this application also provides a search system update device based on a large language model, which corresponds to the search system update method based on a large language model. Since the principle of the device in this application is similar to the search system update method based on a large language model described above, the implementation of the device can refer to the implementation of the method, and the repeated parts will not be described again.

[0150] Figure 6 This illustration shows a schematic diagram of the structure of a search system update device based on a large language model provided in an embodiment of this application.

[0151] The initial prediction module 601 is used to obtain the search terms to be operated, and the search system determines the query understanding (QU) results of the search terms to be operated based on the rule base. The QU results include at least one of the following: initial category prediction results, initial entity recognition results, initial word correction results, and initial synonym results.

[0152] The model prediction module 602 is used to perform QU prediction on the search terms to be operated based on the large language model to obtain the large model prediction results. The large model prediction results include at least one of the following: model category prediction results, model entity recognition results, model word correction results, and model synonym results.

[0153] The gray-scale update module 603 is used to modify the rule base in the search system based on the prediction results of the large model, obtain a new rule base, push the new rule base to the preset internal test account, and use the search system of the internal test account as the gray-scale engine system.

[0154] The formal update module 604 is used to formally update the rule base in the search system based on the search results of the grayscale engine system for the search terms to be operated.

[0155] Optionally, the formal update module 604 is specifically used for:

[0156] Input the search terms to be used into the grayscale engine system to obtain search results;

[0157] The accuracy of search results is determined based on a large language model;

[0158] Determine whether to formally update the rule base based on the accuracy rate;

[0159] If so, the rule base in the search system will be officially updated based on the prediction results of the large model.

[0160] Optionally, the model prediction module 602 is specifically used for:

[0161] The search terms to be operated, the initial category prediction results, and the first relevance judgment criteria are filled into the first prompt term template to obtain the first prompt statement. The first prompt statement is then input into the large language model to delete the category prediction results that are not related to the search terms to be operated from the initial category prediction results, and the first category prediction result is obtained.

[0162] Input the search terms to be operated into the large language model to obtain the second category prediction results of the search terms to be operated;

[0163] The search term to be operated, the second category prediction result, and the second relevance judgment criterion are filled into the second prompt term template to obtain the second prompt statement. The first prompt statement is then input into the large language model to filter the category prediction results related to the search term to be operated from the second category prediction results to obtain the third category prediction result.

[0164] The prediction results for the first category and the prediction results for the third category are combined into the prediction results for the model category.

[0165] Optionally, the model prediction module 602 is specifically used for:

[0166] The search terms to be operated and the initial entity recognition results are filled into the third prompt word template to obtain the third prompt statement. The third prompt statement is then input into the large language model to correct the initial entity recognition results and obtain the model entity recognition results.

[0167] Optionally, the model prediction module 602 is specifically used for:

[0168] If the accuracy of the search term to be operated is less than the preset accuracy correction threshold, the search term to be operated is filled into the fourth prompt word template to obtain the fourth prompt statement. The fourth prompt statement is then input into the large language model to perform error correction processing on the search term to be operated, and the model word error correction result is obtained.

[0169] Optionally, the model prediction module 602 is specifically used for:

[0170] The search terms to be operated and the initial synonym results are filled into the fifth prompt word template to obtain the fifth prompt statement. The fifth prompt statement is then input into the large language model to obtain the model's synonym results.

[0171] Optionally, the formal update module 604 is specifically used for:

[0172] Based on the model's category prediction results, the corresponding category information of the search terms to be operated in the rule base will be officially updated.

[0173] Based on the model entity recognition results, the entity information corresponding to the search terms to be operated in the rule base will be officially updated.

[0174] Based on the model word correction results, the corresponding correction words in the rule base for the search terms to be operated will be officially updated.

[0175] Based on the synonym results from the model, the synonym information corresponding to the search terms to be operated in the rule base will be officially updated.

[0176] This application's embodiments utilize a large language model to correct and optimize the QU results for the search terms to be operated, enabling automated operation of the rule base without altering the existing architecture of the search system. Furthermore, compared to manual operation, automated rule base operation using a large language model has advantages such as lower technical barriers, lower costs, and greater versatility, reducing both the operational costs and efficiency of the rule base. By performing a phased update of the rule base before a formal update, disruption to normal user experience can be avoided, further enhancing the stability of updates to the search system based on the large language model.

[0177] Figure 7 This illustration shows a schematic diagram of an electronic device provided in an embodiment of this application, including: a processor 701, a storage medium 702, and a bus 703. The storage medium 702 stores machine-readable instructions executable by the processor 701. When the electronic device runs a search system update method based on a large language model as described in the embodiment, the processor 701 communicates with the storage medium 702 via the bus 703. The processor 701 executes the machine-readable instructions. The preamble of the method item of the processor 701 executes the steps in the search system update method based on the large language model described above.

[0178] This application also provides a computer-readable storage medium storing a computer program that is executed by a processor, which performs the steps in the above-described method for updating a search system based on a large language model.

[0179] In this embodiment, the computer program, when run by the processor, can also execute other machine-readable instructions to perform other methods as described in the embodiments. For details on the specific execution steps and principles, please refer to the description of the embodiments, which will not be repeated here.

[0180] In the embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. The apparatus embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. Furthermore, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Additionally, the displayed or discussed mutual couplings, direct couplings, or communication connections may be through some communication interfaces; indirect couplings or communication connections between devices or units may be electrical, mechanical, or other forms.

[0181] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0182] In addition, the functional units in the embodiments provided in this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0183] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0184] It should be noted that similar labels and letters in the following figures indicate similar items. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures. In addition, the terms "first", "second", "third", etc. are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0185] Finally, it should be noted that the above-described embodiments are merely specific implementations of this application, used to illustrate the technical solutions of this application, and not to limit them. The protection scope of this application is not limited thereto. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that any person skilled in the art can still modify or easily conceive of changes to the technical solutions described in the foregoing embodiments, or make equivalent substitutions for some of the technical features, within the scope of the technology disclosed in this application; and these modifications, changes, or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application. All should be covered within the protection scope of this application. Therefore, the protection scope of this application should be determined by the protection scope of the claims.

Claims

1. A method for updating a search system based on a large language model, characterized in that, include: The search terms to be operated are obtained, and the search system determines the query understanding (QU) results of the search terms to be operated based on the rule base. The QU results include at least one of the following: initial category prediction results, initial entity recognition results, and initial synonym results. Based on the large language model and the QU results, QU prediction is performed on the search terms to be operated to obtain the large model prediction results. The large model prediction results include at least one of the following: model category prediction results, model entity recognition results, model word correction results, and model synonym results. Based on the prediction results of the large model, the rule base in the search system is modified to obtain a new rule base. The new rule base is then pushed to a preset internal test account, and the search system of the internal test account is used as a grayscale engine system. Based on the search results of the grayscale engine system for the search terms to be operated, the rule base in the search system is officially updated.

2. The method according to claim 1, characterized in that, The step of formally updating the rule base in the search system based on the search results of the grayscale engine system for the search terms to be operated includes: The search terms to be operated are input into the grayscale engine system to obtain search results; The accuracy of the search results is determined based on the large language model. The accuracy rate will determine whether to formally update the rule base. If so, the rule base in the search system will be officially updated based on the prediction results of the large model.

3. The method according to claim 1, characterized in that, The process of performing QU prediction on the search terms to be operated based on the large language model and the QU results to obtain the model category prediction results includes: The search term to be operated, the initial category prediction result, and the first relevance judgment criterion are filled into the first prompt term template to obtain the first prompt statement. The first prompt statement is then input into the large language model to delete the category prediction results that are not related to the search term to be operated from the initial category prediction results, thereby obtaining the first category prediction result. The search terms to be operated are input into the large language model to obtain the second category prediction results of the search terms to be operated; The search term to be operated, the second category prediction result, and the second relevance judgment criterion are filled into the second prompt term template to obtain the second prompt statement. The second prompt statement is then input into the large language model to filter the category prediction results related to the search term to be operated from the second category prediction results to obtain the third category prediction result. The prediction results of the first category and the prediction results of the third category are combined into the prediction results of the model category.

4. The method according to claim 1, characterized in that, The process of performing QU prediction on the search term to be operated based on the large language model and the QU results to obtain the model entity recognition results includes: The search terms to be operated and the initial entity recognition results are filled into the third prompt word template to obtain the third prompt statement. The third prompt statement is then input into the large language model to correct the initial entity recognition results and obtain the model entity recognition results.

5. The method according to claim 1, characterized in that, The process of performing QU prediction on the search terms to be operated based on the large language model and the QU results, and obtaining the model word correction results, includes: If the accuracy of the search term to be operated is less than the preset accuracy correction threshold, the search term to be operated is filled into the fourth prompt word template to obtain the fourth prompt statement. The fourth prompt statement is then input into the large language model to perform error correction processing on the search term to be operated, and the model word error correction result is obtained.

6. The method according to claim 1, characterized in that, The process of performing QU prediction on the search terms to be operated based on the large language model and the QU results to obtain the model synonym results includes: The search terms to be operated and the initial synonym results are filled into the fifth prompt word template to obtain the fifth prompt statement. The fifth prompt statement is then input into the large language model to obtain the model synonym results.

7. The method according to claim 2, characterized in that, The formal update of the rule base in the search system based on the prediction results of the large model includes: Based on the category prediction results of the model, the category information corresponding to the search terms to be operated in the rule base is officially updated; Based on the entity recognition results of the model, the entity information corresponding to the search term to be operated in the rule base is officially updated; Based on the error correction results of the model words, the error correction words corresponding to the search terms to be operated in the rule base are officially updated. Based on the synonym results of the model, the synonym information corresponding to the search term to be operated in the rule base is officially updated.

8. A search system update device based on a large language model, characterized in that, include: The initial prediction module is used to obtain the search terms to be operated, and the search system determines the query understanding (QU) results of the search terms to be operated based on the rule base. The QU results include at least one of the following: initial category prediction results, initial entity recognition results, and initial synonym results. The model prediction module is used to perform QU prediction on the search term to be operated based on the large language model and the QU results, and obtain the large model prediction results. The large model prediction results include at least one of the following: model category prediction results, model entity recognition results, model word correction results, and model synonym results. The gray-scale update module is used to modify the rule base in the search system according to the prediction results of the large model, obtain a new rule base, push the new rule base to the preset internal test account, and use the search system of the internal test account as the gray-scale engine system. The formal update module is used to formally update the rule base in the search system based on the search results of the grayscale engine system for the search terms to be operated.

9. An electronic device, characterized in that, include: The device includes a processor, a storage medium, and a bus, wherein the storage medium stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor communicates with the storage medium via the bus, and the processor executes the machine-readable instructions to perform the steps of a search system update method based on a large language model as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, performs the steps of the search system update method based on a large language model as described in any one of claims 1 to 7.

Images