Generative search data processing method and device, equipment and storage medium

Abstract

The invention provides a generative search data processing method and device, equipment and a storage medium. The invention relates to the technical field of computers, in particular to the technical field of search technologies and large models, and can be used for application scenes such as generative search, intelligent questions and answers, intelligent recommendation and the like. The method comprises the following steps: collecting feedback data of a target object to a search answer which is firstly displayed on a search page; analyzing the feedback data to identify key factors influencing the satisfaction degree of the search answers; mining a first sample set and a second sample set from the feedback data in combination with the key factors; based on the first sample set and / or the second sample set, optimizing the first large language model to obtain a second large language model; and deploying the second large language model into the search system to replace the first large language model. According to the scheme disclosed by the invention, the high-confidence sample can be automatically mined, the self-optimization loop of the generative large language model is realized, and the quality of the search answer is improved.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technology, and in particular to the fields of search technology and large model technology. The present disclosure can be used in application scenarios such as generative search, intelligent question and answer, and intelligent recommendation, and specifically relates to a method, apparatus, device, and storage medium for processing generative search data. Background Art

[0002] In related technologies, the optimization of generative search systems usually relies on manually annotated data, which is time-consuming and economically expensive to obtain, and it is difficult to ensure that the optimization direction is consistent with the actual preferences. In addition, traditional user feedback signals are complex and noisy, making them difficult to be directly used for the optimization of generative large language models. Summary of the invention

[0003] The present disclosure provides a method, device, equipment and storage medium for processing generative search data.

[0004] According to a first aspect of the present disclosure, a method for processing generative search data is provided, the method comprising: collecting feedback data of a target object on a search answer displayed at the first position on a search page, wherein the search answer is an answer generated by a search system using a first language model; analyzing the feedback data to identify key factors affecting satisfaction with the search answer; mining a first sample set and a second sample set from the feedback data in combination with the key factors, the first sample set including samples with satisfaction greater than a first threshold, and the second sample set including samples with satisfaction less than a second threshold; optimizing the first language model based on the first sample set and / or the second sample set to obtain a second language model, wherein the search answer generated by the second language model has a better match with the target object's preference than the first language model; and deploying the second language model into the search system to replace the first language model.

[0005] According to a second aspect of the present disclosure, a search processing method is provided, the method comprising: receiving a question input by a target object on a search page; using a pre-trained large language model to generate a search answer that matches the question; displaying the search answer at the top of the search page; monitoring and recording feedback data from the target object regarding the search answer; wherein the feedback data is used to optimize the large language model.

[0006] According to a third aspect of the present disclosure, a device for processing generative search data is provided, the device comprising: a collection module, for collecting feedback data of a target object on a search answer displayed at the first position on a search page, wherein the search answer is an answer generated by a search system using a first language model; an analysis module, for analyzing the feedback data to identify key factors affecting satisfaction with the search answer; a sample mining module, for mining a first sample set and a second sample set from the feedback data in combination with the key factors, the first sample set including samples with satisfaction greater than a first threshold, and the second sample set including samples with satisfaction less than a second threshold; an optimization and adjustment module, for optimizing the first language model based on the first sample set and / or the second sample set to obtain a second language model, wherein the search answer generated by the second language model has a better match with the target object preference than the first language model; and a deployment module, for deploying the second language model into the search system to replace the first language model.

[0007] According to a fourth aspect of the present disclosure, a search processing device is provided, which includes: a receiving module for receiving a question input by a target object on a search page; a generating module for generating a search answer matching the question using a pre-trained large language model; a display module for displaying the search answer at the top of the search page; a monitoring and recording module for monitoring and recording feedback data of the target object for the search answer; wherein the feedback data is used to optimize the large language model.

[0008] According to a fifth aspect of the present disclosure, there is provided an electronic device, including:

[0009] at least one processor; and

[0010] a memory communicatively connected to the at least one processor; wherein,

[0011] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute any method in the embodiments of the present disclosure.

[0012] According to a sixth aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to cause the computer to execute any method according to the embodiments of the present disclosure.

[0013] According to a seventh aspect of the present disclosure, a computer program product is provided, comprising a computer program, which implements any method according to the embodiments of the present disclosure when executed by a processor.

[0014] By adopting the solution disclosed in the present invention, by collecting multivariate feedback data and automatically mining high-confidence samples, a self-optimization cycle of a generative large language model is realized, the quality of the search answer displayed first is improved, and the efficiency of the target object in obtaining the answer is improved.

[0015] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] The accompanying drawings are used to better understand the present solution and do not constitute a limitation of the present disclosure.

[0017] Figure 1 is a flowchart of a method for processing generative search data according to an embodiment of the present disclosure;

[0018] Figure 2 is an architectural diagram of a generative search data flywheel system according to an embodiment of the present disclosure;

[0019] Figure 3 is a schematic diagram of generating multiple versions of answers based on the same query word according to an embodiment of the present disclosure;

[0020] Figure 4 is a comparative schematic diagram of generating search answers according to different versions of large language models in an embodiment of the present disclosure;

[0021] Figure 5 is a schematic diagram of an architecture for accumulating feedback data according to an embodiment of the present disclosure;

[0022] Figure 6 is a schematic diagram of large language model optimization according to an embodiment of the present disclosure;

[0023] Figure 7 is a flowchart of a search processing method according to an embodiment of the present disclosure;

[0024] Figure 8 is an architectural diagram for optimizing search answers according to the disclosed search system;

[0025] Fig. 9 is a schematic diagram of the structure of a processing device for generative search data according to an embodiment of the present disclosure;

[0026] Fig.10 is a schematic diagram of the structure of a search processing device according to an embodiment of the present disclosure;

[0027] Fig.11 is a scenario schematic diagram of a search processing method according to an embodiment of the present disclosure;

[0028] Fig.12It is a structural diagram of an electronic device used to implement the generative search data processing method and / or the search processing method of the embodiments of the present disclosure. DETAILED DESCRIPTION

[0029] The following is a description of exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope of the present disclosure. Similarly, for the sake of clarity and conciseness, the description of well-known functions and structures is omitted in the following description.

[0030] The term "and / or" in this article is only a description of the association relationship of associated objects, indicating that there may be three relationships. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. The term "at least one" in this article means any combination of at least two of any one or more of a plurality of. For example, including at least one of A, B, and C can mean including any one or more elements selected from the set consisting of A, B, and C. The terms "first" and "second" in this article refer to multiple similar technical terms and distinguish them. They do not mean to limit the order or to limit them to only two. For example, the first feature and the second feature refer to two types / two features. The first feature can be one or more, and the second feature can also be one or more.

[0031] In addition, in order to better illustrate the present disclosure, numerous specific details are given in the following specific embodiments. It should be understood by those skilled in the art that the present disclosure can also be implemented without certain specific details. In some examples, methods, means, components and circuits well known to those skilled in the art are not described in detail in order to highlight the subject matter of the present disclosure.

[0032] Before introducing the technical solutions of the embodiments of the present disclosure, the technical terms that may be used in the present disclosure are further explained:

[0033] Large Language Model (LLM): refers to a deep learning model trained using a large amount of text data. It has strong language understanding and generation capabilities and can handle a variety of natural language tasks.

[0034] Generative search: It is a search technology based on a large language model. Different from traditional search engines based on keyword matching, generative search generates answers in natural language form by understanding and analyzing the user's query intent, rather than just providing a list of links. It can use the power of a large language model to integrate multi-source information to create a more structured and complete answer that better meets user needs.

[0035] Alignment algorithm: Alignment algorithm is used to adjust the output of the model to be consistent with human preferences. During the training process of the generative model, by using the alignment algorithm, the model can learn to better meet the expectations or needs of the target object, improving the relevance and satisfaction of the search results.

[0036] Target object preference: refers to the target object's inclination and preference for a certain type of result, answer or information in a specific situation or when using a system. It reflects the content form, structure, style or type that the target object is more satisfied with or more willing to accept when faced with multiple choices or answers.

[0037] Supervised Fine-Tuning (SFT): is a technique for fine-tuning a pre-trained model through supervised learning. In this process, the model learns how to improve its prediction ability on a specific task based on given labeled data.

[0038] In the related technologies, the optimization of generative search still relies heavily on a large amount of manually annotated data, which has high time and economic costs for data acquisition, low optimization efficiency, and the consistency between the optimization direction and the actual preference cannot be guaranteed. Generative search has the following problems: 1. Generative search feedback signals are scarce; 2. Feedback signals lack actual semantics and are difficult to optimize directly: the lack of semantics in signals will cause large models to be unable to understand the meaning of signals, such as skip clicks, browsing time and other signals, so they cannot be directly used as optimization targets; 3. Traditional sorting feedback signals are complex and noisy, making them difficult to use.

[0039] Therefore, data flywheels in generative search scenarios have been difficult to apply on a large scale so far.

[0040] In order to at least partially solve one or more of the above problems and other potential problems, the present disclosure proposes a method for processing generative search data, which realizes the self-optimization cycle of the generative large language model by collecting multivariate feedback data and automatically mining high-confidence samples, and effectively solves the problems of high data acquisition cost, inconsistency between optimization direction and preference, etc. It not only improves the sample mining efficiency, but also improves the quality of the search answer displayed first, and improves the efficiency of the target object in obtaining the answer.

[0041] The present disclosure provides a method for processing generative search data. Figure 1It is a flow chart of a method for processing generative search data according to an embodiment of the present disclosure. The method for processing generative search data can be applied to a device for processing generative search data. The device for processing generative search data is located in an electronic device. The electronic device includes but is not limited to fixed devices and / or mobile devices. For example, fixed devices include but are not limited to servers, and servers can be cloud servers or ordinary servers. For example, mobile devices include but are not limited to: mobile phones, tablet computers. In some possible implementations, the method for processing generative search data can also be implemented by a processor calling computer-readable instructions stored in a memory. For example Figure 1 As shown, the method for processing the generative search data includes:

[0042] S101, collecting feedback data of the target object on the search answer displayed first on the search page, wherein the search answer is an answer generated by the search system using the first language model;

[0043] S102, analyzing the feedback data to identify key factors that affect satisfaction with search answers;

[0044] S103. Mining a first sample set and a second sample set from the feedback data in combination with key factors, wherein the first sample set includes samples whose satisfaction is greater than a first threshold, and the second sample set includes samples whose satisfaction is less than a second threshold;

[0045] S104, based on the first sample set and / or the second sample set, optimizing the first largest language model to obtain a second largest language model, wherein the matching degree between the search answer generated by the second largest language model and the target object preference is better than that of the first largest language model;

[0046] S105: deploy the second largest language model into the search system to replace the first largest language model.

[0047] In the disclosed embodiment, the first largest language model is the model used by the search system to generate the search answer at the first moment. The second largest language model is the model used by the search system to generate the search answer at the second moment. The second moment is later than the first moment. The second largest language model is usually more accurate in generating the search answer that meets the preferences of the target object than the first largest language model, with a higher matching degree and better performance.

[0048] In the disclosed embodiment, the first threshold and the second threshold can be set according to actual needs. For example, according to the analysis results, the first threshold (such as satisfaction score ≥ 4, the full score is 5 points) and the second threshold (such as satisfaction score ≤ 2, the full score is 5 points) are set to distinguish high-quality and low-quality search answer samples.

[0049] In some implementations, the feedback data is analyzed to identify key factors that affect the satisfaction of the search answers, including: using statistical analysis and machine learning techniques (such as logistic regression, decision trees, etc.) to conduct in-depth analysis on the collected feedback data to identify key factors that affect the satisfaction of the search answers. These factors may include the relevance, accuracy, completeness, readability, timeliness, etc. of the answer.

[0050] In some embodiments, based on the first sample set and / or the second sample set, the first language model is optimized to obtain the second language model, including: using high-quality samples in the first sample set to supervise and fine-tune the first language model so that it is better at generating search answers that meet the preferences of the target object. At the same time, it is possible to consider using low-quality samples in the second sample set as negative samples to improve the model's ability to recognize wrong answers through comparative learning and other methods. After multiple iterative training, the optimized second language model is obtained.

[0051] In some implementations, the optimized second largest language model is deployed to the search system to replace the original first largest language model. The performance of the new model is monitored, including indicators such as satisfaction with search answers, click-through rate of target objects, and dwell time, and the model is continuously optimized based on new feedback data.

[0052] Figure 2 The architecture diagram of the generative search data flywheel system is shown as Figure 2 As shown in the figure, the system architecture includes three parts: online feedback signal collection, data mining and alignment training.

[0053] During the online feedback signal collection phase, the large language model obtains resources from multiple information sources based on the query and generates search answers; it collects multiple feedback signals online and stores them in logs.

[0054] In the data mining stage, if K answers are generated based on the same query word and the same reference content, scores are generated for each of the K answers, and the highest-scoring answer and the lowest-scoring answer for the query word are selected from the K scores to form a score pair sample. If 1 answer is generated based on the same query word and the same reference content, a score is generated for this answer. If the score is high, the single sample is stored in a single-point sample set with high satisfaction; if the score is low, the single sample is stored in a single-point sample set with low satisfaction. Figure 3 The schematic diagram shows the generation of multiple versions of answers based on the same query word. Based on the same query word "what are the three checks and eight pairs", two versions of search answers are generated. According to the feedback data, Figure 3 The search answers shown on the right are rated higher than Figure 3The left figure shows the search answer score. Based on these feedback data, the first language model is optimized to obtain the second language model. After deploying the new model, the answers generated by the search system are more in line with the expectations and needs of the target objects, thereby improving the efficiency of the target objects in obtaining answers and improving satisfaction.

[0055] In the alignment training phase, manual review and annotation are performed on paired sample sets, single-point sample sets with high satisfaction, and single-point sample sets with low satisfaction to generate high-quality sample sets. Select any algorithm from a variety of alignment algorithms, select appropriate samples from the sample sets mined from data, optimize the large language model, and generate an optimized large language model. Figure 4 A comparison diagram of different versions of large language models generating search answers is shown, Figure 4 As shown, the search answer generated by the first language model is as follows Figure 4 As shown in the left figure, the search answer generated by the second largest language model is as follows Figure 4 As shown in the figure on the right, it is obvious that the search answers generated by the optimized large language model are more accurate and richer.

[0056] In this way, the preferences of the target objects can be identified by analyzing the feedback data, and the identified preference data can be used to continuously optimize the large language model, so that the search system can self-learn and evolve, improve the accuracy and relevance of search answers, and make the search results more in line with the expectations and needs of the target objects.

[0057] In some embodiments, the feedback data includes: a first type of feedback data and a second type of feedback data; wherein the first type of feedback data includes historical behavior data during the target object search process; the second type of feedback data includes survey data that the system actively collects from the target object; wherein the first type of feedback data and the second type of feedback data are data generated by a single round of generative question and answering.

[0058] In the disclosed embodiment, single-round generative question-answering refers to a one-time question-answering interaction between the target object and the system. The target object inputs a query term, and the system generates and returns an answer. There is no subsequent interaction in the whole process, and the answer does not rely on contextual information or involve continuous dialogue. The system only needs to generate a one-time result based on the single input of the target object.

[0059] In the disclosed embodiments, multi-round generative question-answering refers to a question-answering process in which the target object and the system interact multiple times. In this process, the system's answer is based on the current question and previous context information. The target object can ask follow-up questions or further clarify, and the system will generate a coherent answer. Multi-round generative question-answering simulates natural conversation, and the answer can be gradually deepened or clarified as the conversation progresses.

[0060] In some embodiments, the first type of feedback data is historical behavior data during the target object search process. This type of data mainly refers to the behavior records generated by the target object during the search process, including but not limited to search keywords, clicked search results, dwell time, scrolling page behavior, returning to search or re-entering new keywords, etc. Among them, search keywords: the original question or query word entered by the target object, such as "today's weather". Click behavior: which search result the target object clicked, such as clicking the first link about "today's weather". Dwelling time: the length of time the target object stays on a search result page, which can reflect the degree to which the result satisfies the target object's needs. Scrolling page: whether the target object scrolls the page to view more search results, and the speed and depth of the scrolling. Re-search: If the target object is not satisfied with the first search result, it may return to the search page, modify the query word or add more conditions to search. By analyzing the historical behavior of the target object, the search engine can learn the preferences and search habits of the target object, thereby providing more personalized search results in the future.

[0061] In the disclosed embodiment, the first type of feedback data can indirectly reflect the target object's satisfaction with the current answer, including multiple implicit features. Here, implicit features include but are not limited to:

[0062]

[0063]

[0064] Table 1

[0065] In Table 1, the result page loading of a search behavior is regarded as one PV. For example, when the target object enters keywords in the search engine and clicks search, the search result page returned by the search engine is loaded and displayed to the target object. This process can be regarded as one PV.

[0066] It can be understood that the above implicit features can be supplemented or deleted according to needs.

[0067] In some embodiments, the second type of feedback data is survey data that the system actively collects from the target object. This type of data is usually the subjective feelings and opinions actively collected by the system from the target object through questionnaires, target object interviews, satisfaction ratings, etc. Among them, satisfaction survey: a satisfaction questionnaire pops up on the search results page or in the application, asking the target object about the satisfaction with the search results, page layout, advertising interference, etc. Target object interview: by inviting some target objects to participate in the interview, we can gain an in-depth understanding of their search habits, needs, and views on existing functions. Feedback portal: set up a feedback portal in the application (Application, APP) to encourage the target object to actively submit feedback on search experience, function suggestions, etc. The second type of feedback data provides the search engine with the direct voice and opinions of the target object, which helps to discover potential problems and improvement points. These data are of great significance for product iteration, function optimization, and improving the target object experience.

[0068] In the disclosed embodiment, the second type of feedback data can directly reflect the target object's satisfaction with the current answer, including a variety of explicit features.

[0069] Here, implicit features include but are not limited to:

[0070]

[0071]

[0072] Table 2

[0073] It can be understood that the above explicit features can be supplemented or deleted according to needs.

[0074] In this way, by combining historical behavior data and survey data, we can fully capture the implicit and explicit feedback of the target object on the search results, reduce the deviation that may be caused by a single data source, and improve the comprehensiveness and accuracy of feedback data. Based on richer feedback data, we can more accurately identify the key factors that affect the satisfaction of search answers, thereby optimizing the language model and effectively improving the matching degree between the generated answers and the preferences of the target objects.

[0075] In some embodiments, the feedback data also includes: a third category of feedback data and a fourth category of feedback data; wherein the third category of feedback data includes data generated during multiple rounds of interactive dialogues; the fourth category of feedback data includes data on the depth of multiple rounds of interactive dialogues; wherein the third category of feedback data and the fourth category of feedback data are data generated by multiple rounds of generative question and answering.

[0076] In the disclosed embodiment, a multi-round interactive dialogue refers to a series of continuous question-and-answer processes between a target object and a search system or a dialogue assistant, and each round of dialogue is conducted based on the context of the previous round.

[0077] In the disclosed embodiments, the depth of conversation refers to the complexity and depth of the interaction between the target object and the system in a multi-round interactive conversation, which is usually related to the number of conversation rounds, the richness of information, and the degree to which the needs of the target object are met.

[0078] In the disclosed embodiment, the third type of feedback data includes data records generated during multiple rounds of interactive dialogues that can reflect the needs and reactions of the target object.

[0079] In the disclosed embodiment, the fourth type of feedback data is data used to measure the depth of multi-round interactive dialogues, including indicators such as the number of dialogue rounds, complexity, and the degree to which the needs of the target object are met.

[0080] In some embodiments, the third type of feedback data is collected, including: when the target object has multiple rounds of interactive dialogue with the search system, the system needs to record the content of each round of dialogue, the query input by the target object, the answer generated by the system, and the reaction of the target object (such as clicks, replies, dwell time, etc.). The process of multiple rounds of dialogue is deeply analyzed to identify the changes in the needs of the target object at different stages of the dialogue, the transfer of interests, and the impact of the system answer on the subsequent behavior of the target object.

[0081] In some embodiments, collecting the fourth type of feedback data includes: designing a method to measure the depth of multi-round interactive dialogues, which may include the number of dialogue rounds, the complexity of the query input by the target object, the amount of information generated by the system in the answer, and whether the target object actively asks more in-depth questions, etc. The collected dialogue depth data is counted and analyzed to understand the target object's acceptance and satisfaction with dialogues of different depths, and how these deep dialogues affect the quality of search answers and the target object's final decision.

[0082] For example, the target object first inputs "What is artificial intelligence?" After the system gives a brief definition, the target object further asks "What are the applications of artificial intelligence in the medical field?" The system generates answers to multiple application scenarios, but the target object modifies the question to "How can artificial intelligence help diagnose diseases?" The system generates a more specific description of the diagnosis scenario and continues to follow up on the target object's questions in multiple rounds of dialogue. The target object's behavioral data such as the duration of stay, question modification, and continuous clicks on answers in each round of dialogue are recorded by the system as the third type of feedback data, while the number of interaction rounds and reaction time of the target object in multiple rounds of dialogue are recorded as the fourth type of feedback data. These feedback data help the system understand the changes in the target object's needs and optimize the model to generate answers that better meet the needs of the target object.

[0083] In this way, by collecting behavioral data and in-depth feedback from multiple rounds of interactions, we can dynamically capture the changes in the target object's needs in complex conversations, thereby more accurately adjusting the answers generated by the model. The optimized model can adapt to multi-round question-and-answer scenarios, provide more coherent and in-depth answers to the target object, and significantly improve the target object's satisfaction in multiple rounds of conversations.

[0084] In some embodiments, collecting feedback data of the target object on the search answer displayed first on the search page also includes: selecting a target answer generation algorithm that matches the current task requirements from a plurality of answer generation algorithms; for a given set of query terms, using the target answer generation algorithm to generate multiple versions of search answers for each query term; and displaying the multiple versions of the search answers to the actual terminal or the simulated terminal in turn to collect feedback data of the target object on each version of the search answer.

[0085] In the disclosed embodiments, a variety of answer generation algorithms are provided, including but not limited to random generation (Random Generation), diversity generation (Diverse Beam Search Generation) and constraint-guided generation (Constraint-Guided Generation).

[0086] In the disclosed embodiment, random generation refers to increasing the randomness of the generated results by increasing the parameters of the SFT model.

[0087] In the disclosed embodiment, diversity generation is intended to improve the diversity of generated content. By balancing accuracy and diversity, it is suitable for use when you want to obtain answers from multiple different angles, which can improve the practicality and effectiveness of the model in dealing with open questions.

[0088] In the disclosed embodiment, constraint-guided generation is to add soft constraints to the prompt template of the model to guide the model to generate answers of a specific style or structure. For example, in the system, by fixing the query and reference, guiding instructions are added to the style part, such as "the answer should be more structured" or "the answer should be concise and to the point". This generation method can control the form and style of the answer while keeping the answer logic of the model unchanged, and is suitable for scenarios that require a specific format or style.

[0089] In some embodiments, a target answer generation algorithm that matches the current task requirements is selected from a plurality of answer generation algorithms, including: when generating search answers, the system can use a variety of different generation algorithms (such as random generation, diversity generation, constraint-guided generation, etc.). Different algorithms may have different focuses when generating answers. For example, some algorithms emphasize the diversity of generated content, while others focus on the accuracy and conciseness of answers. According to the specific requirements of the task, the system will select a most suitable algorithm to generate an answer. For example: suppose the target object enters the query "What is the highest mountain in the world?" The system can use an "accuracy first" algorithm to ensure that the returned answer is very accurate and concise - that is, "Mount Everest". Another "diversity first" algorithm can also be used to generate multiple different versions of answers, which may include relevant background information, such as the height and location of Mount Everest, and comparison with other peaks.

[0090] In the disclosed embodiments, a given query word set refers to a combination of multiple related query words or questions that may be raised by a target object in a specific scenario. These query word sets may be a set of keywords predefined by the system, or a set consisting of multiple queries input by the target object in a specific task or context. In a generative search or question-answering system, the system generates multiple versions of answers based on these query word sets using an answer generation algorithm. For example, the target object may query a series of words related to "how to improve work efficiency", such as "efficiency improvement tools", "time management methods", "concentration improvement techniques", etc. The system will generate corresponding answers based on these query word sets to meet the needs of the target object from different angles.

[0091] In some embodiments, for a given set of query terms, a target answer generation algorithm is used to generate multiple versions of search answers for each query term, including: after the target object enters the query term, the system will generate multiple versions of answers for the query based on the selected target generation algorithm. Each version of the answer may answer the target object's question from a different angle. The purpose of this is to find out which version best meets the needs of the target object. For example: The target object enters the query "How to improve work efficiency?". The target generation algorithm can generate the following three versions of answers: Version 1: Explain the concept of work efficiency in detail and propose some common time management methods. Version 2: Concisely list several specific tools (such as schedule management applications) to improve work efficiency. Version 3: Combined with psychological factors, it is recommended how to stay focused and motivated to improve efficiency.

[0092] In some embodiments, multiple versions of search answers are displayed to actual terminals or simulated terminals in turn to collect feedback data from the target object on each version of the search answer, including: after generating multiple versions of answers, the system will display these answers to the target object in turn, or use a simulated terminal (automated tool) for testing to collect the target object's feedback on each answer. Through the target object's click behavior, likes, sharing and other feedback signals, the system can determine which answer version best meets the needs of the target object. For example: the system may first display the answer of version 1 to monitor whether the target object clicks on it or continues to search for other information. If the target object does not interact or skips, the system can infer that version 1 does not meet the needs of the target object. Then display version 2 and version 3, and collect the target object's click, reading time, and further interaction (such as likes or comments) data to determine which version is more popular.

[0093] Figure 5 A schematic diagram of the architecture for accumulating feedback data is shown, Figure 5 As shown, the accumulated feedback data consists of two parts: the online system and offline training. Among them, the processing of the online system includes: the large language model selects the target generation algorithm from multiple answer generation algorithms, generates multiple versions of answers for a given query word set, and displays multiple versions of search answers to the terminal in turn to collect the target object's feedback data on each version of the search answer. Among them, the offline training includes: data mining on the collected feedback data to generate training samples containing preference features; based on the training samples, the current large language model is optimized, and the optimized large language model is put online, so as to achieve continuous accumulation of feedback data.

[0094] For example, the target object enters "How to improve sleep quality?" in the search box. The system first selects the "diversity generation algorithm" from a variety of answer generation algorithms because this question is relatively open and the target object may be interested in different solutions. The system generates three different versions of answers: the first version introduces lifestyle habits to improve sleep, the second version recommends several application tools that help sleep, and the third version combines psychology to suggest how to reduce anxiety to improve sleep quality. Based on its algorithm, the system displays the first version, which is the answer about lifestyle habits. The target object browsed this answer, but stayed for a short time and did not interact further. Later, another target object asked the same query, and the system displayed the second version, which is the answer that recommends application tools. This target object stayed on this answer for a long time and liked the recommended application tool. Based on these target object feedback, the system can judge that the second version of the answer is more in line with the needs of the target object. Through these implicit and explicit feedback data, the system records which answer versions are more popular and inputs these data into the model optimization process. The system adjusts the generation model through reinforcement learning and feedback mechanisms so that it is more inclined to generate tool recommendation answers preferred by the target object when encountering similar queries in the future.

[0095] In this way, by using multiple generation algorithms to generate different versions of answers for the same query term and by taking turns displaying and collecting feedback data, the system can better understand the preferences and needs of the target objects and optimize the subsequent answer generation process.

[0096] In some embodiments, feedback data is analyzed to identify key factors that affect satisfaction with search answers, including: preprocessing the feedback data to obtain preprocessed data; extracting multiple features that affect satisfaction from the preprocessed data; and using statistical algorithms to analyze the multiple features to identify key factors related to satisfaction.

[0097] In the disclosed embodiment, multiple features refer to various information points extracted from the pre-processed target object behavior data, which are independent but interrelated, and together describe the behavior patterns and preferences of the target object when using search answers. These features may include but are not limited to the keywords of the search query, the search result location clicked by the target object, the browsing time, the target object feedback (such as likes, shares, comments, etc.), the target object's historical search records, etc.

[0098] In the disclosed embodiment, the key factors are features that are identified from these multiple features through statistical algorithm analysis and are significantly correlated with the target object's satisfaction with the search answer. In other words, the key factors are those features that can significantly affect or explain the difference in the target object's satisfaction with the search answer. These key factors may not be all the features, but only part of them, but they have greater weight or influence in explaining the difference in satisfaction.

[0099] In the disclosed embodiment, the multiple features are extensive and comprehensive descriptions of the target object's behavior, and the key factors are a few features selected from these features that have a significant impact on satisfaction. These key factors are not only related to the characteristics of the search answer, but also reflect the preferences and needs of the target object.

[0100] In some embodiments, the feedback data is preprocessed, including: the system first collects the behavior data log of the target object in the search scenario. These data include the target object's click, display, jump and other behavior records. In order to ensure the quality of the data, the preprocessing process will clean and filter these raw data to remove noise data and outliers. Specifically, the preprocessing will remove unreasonable interaction data, such as abnormally short stay time or repeated invalid click records. The data finally generated will ensure cleanliness and analyzability, laying the foundation for subsequent feature extraction.

[0101] In some embodiments, extracting multiple features that affect satisfaction from the preprocessed data includes: in the cleaned data, the system extracts key features of the target object's behavior. These features include click-through rate, browsing time, click depth, skip behavior, dwell time, etc., which are used to represent the interaction pattern between the target object and the search answer. In order to further explore the target object's preferences, the system will also combine these features to generate pairwise samples and pointwise samples. Pairwise samples are used to compare the relative performance of different search answers in the target object feedback, and pointwise samples are used to evaluate the absolute satisfaction of a single answer.

[0102] In some embodiments, a statistical algorithm is used to analyze multiple features, including: the system uses machine learning and statistical algorithms (such as descriptive statistical analysis, correlation analysis, and regression analysis) to model the extracted features. By analyzing a large amount of target object behavior data, the system can identify key factors that are strongly correlated with the target object's satisfaction. For example, the target object's click behavior and browsing time may be key features of positive feedback, while skipping behavior may be a feature of negative feedback. Through satisfaction modeling, the system can associate behavioral features with satisfaction, build a satisfaction model, and identify high-confidence satisfied and dissatisfied samples.

[0103] For example, when the target object enters the query "How to improve work efficiency?", the system first collects the target object's behavior data on the search page, including clicks, browsing time, skipping behavior, etc. Then, the system preprocesses the data to remove abnormally short dwell time and repeated click records to ensure data quality. After data cleaning, the system extracts key features of multiple target object interactions, such as click-through rate, browsing time, skipping behavior, etc. Based on these features, the system generates pairwise samples for comparing the relative performance of two different search results, and pointwise samples for evaluating the satisfaction of a single result. In the process of generating pairwise samples, the system compares the target object's clicks and dwell time on different answers. For example, the target object stays longer on an answer that recommends an efficiency tool, but stays shorter on another answer about a psychological method. Through such a comparison, the system can determine which answer better meets the needs of the target object. At the same time, the system uses pointwise samples to evaluate the absolute satisfaction of each individual answer. For example, if the target object likes the answer that recommends the tool, it indicates that the answer is highly satisfactory. Next, the system uses machine learning and statistical algorithms to model these extracted features and analyze which behavioral features (such as click counts, skipping behaviors) have a strong correlation with target object satisfaction. Through modeling, the system identified that browsing time and click-through rate are key factors affecting satisfaction, while skipping behaviors are negative feedback. Ultimately, the system optimizes the search model based on these analysis results, making it more inclined to generate answers with high satisfaction.

[0104] In this way, by screening and verifying the output samples, and by preprocessing and extracting features from the feedback data, the quality of the analyzed data and the effectiveness of the features can be ensured, thereby improving the accuracy of the analysis results. By using statistical algorithms to analyze multiple features, the key factors that affect the satisfaction of search answers can be systematically identified, ensuring that the screened samples have high confidence and representativeness, and providing accurate data support for subsequent preference alignment training.

[0105] In some embodiments, combining key factors to mine a first sample set and a second sample set from feedback data also includes: inputting the feedback data into a pre-trained satisfaction model; wherein the satisfaction model is used to predict the target object's satisfaction with the search answer in combination with the key factors; and obtaining the first sample set and the second sample set output by the satisfaction model.

[0106] In the disclosed embodiment, the satisfaction model is a prediction model based on machine learning or statistical analysis, which is used to evaluate or predict the target object's satisfaction with the search answer. It outputs a prediction score of the target object's satisfaction by inputting the target object's behavior data (such as click-through rate, browsing time, skipping behavior, etc.) and combining it with predefined key factors.

[0107] In some embodiments, the feedback data is input into a pre-trained satisfaction model, including: collecting and pre-processing the feedback data of the target object (such as clicks, browsing time, skipping behavior, etc.). These data will be passed as input to the pre-trained satisfaction model. The model has been trained through large-scale target object behavior data and preference alignment training, and can combine multiple key factors (such as click-through rate, dwell time, skipping behavior, etc.) to predict the target object's satisfaction with each search answer. By inputting the feedback data into the satisfaction model, the system can use these key factors to evaluate the satisfaction of each search result. For example, a higher click-through rate and a longer browsing time are usually identified by the model as high satisfaction signals, while frequent skipping behaviors are identified as low satisfaction signals. By combining these key factors, the model can output a satisfaction prediction score for each search answer.

[0108] In some embodiments, obtaining the first sample set and the second sample set output by the satisfaction model includes: after completing the satisfaction prediction, the system divides the feedback data into two sample sets according to the scores output by the model. The first sample set includes samples whose predicted satisfaction is higher than the first threshold, and these samples represent that the target object is satisfied with the search results; the second sample set includes samples whose predicted satisfaction is lower than the second threshold, and these samples represent that the target object is dissatisfied with the search results. Through this classification, the system can extract typical samples with high satisfaction and low satisfaction, providing a basis for subsequent model optimization.

[0109] For example, the target object enters the query "How to improve work efficiency?", and the system generates multiple answers, including recommended efficiency tools, time management skills, and suggestions for improving concentration. The target object clicks on the answer for the recommended tool, stays for 5 minutes and likes it, and stays on the answer for the time management skill for 30 seconds before skipping and clicking on other results. After the system collects these target object behavior data, it pre-processes them and inputs them into a pre-trained satisfaction model. The model combines key factors such as click-through rate, dwell time, and skip behavior to make predictions, and judges that the answer for the recommended tool is high satisfaction (9 / 10), while the answer for the time management skill is low satisfaction (3 / 10). Subsequently, the system divides these data into two sample sets based on the scores: the answer for the recommended tool enters the first sample set (high satisfaction), and the answer for the time management skill enters the second sample set (low satisfaction). Through this type of data, the system further optimizes the generation model to improve the matching degree of search results and the target object experience.

[0110] In this way, through the pre-trained satisfaction model, the first sample set and the second sample set can be automatically mined from a large amount of feedback data, greatly improving the efficiency and accuracy of data processing. Combining key factors for prediction can more accurately identify the target object's satisfaction with the search answer, providing strong support for subsequent optimization work. Based on the mined sample set, the model can be optimized in a targeted manner to improve the quality of the search answer and the satisfaction of the target object.

[0111] In some embodiments, based on the first sample set and / or the second sample set, the first large language model is optimized to obtain the second large language model, including: selecting a target alignment algorithm that matches the current task from a plurality of alignment algorithms; based on the target alignment algorithm, screening out training samples that meet the target alignment algorithm from the first sample data set and the second sample data set; using the target alignment algorithm as a training guide, and using the training samples to optimize the first large language model.

[0112] In the disclosed embodiment, each alignment algorithm can be used to guide the first language model to align preference features. The various alignment algorithms include, but are not limited to, a reward model (RM), a direct preference optimization (DPO) algorithm, a key-term optimization (KTO) algorithm, and a proximal policy optimization (PPO) algorithm.

[0113] In the disclosed embodiment, DPO uses paired comparison data (i.e., which output the target object thinks is better) to train the model so that its output is more in line with human preferences. It usually uses a statistical model to model the comparison relationship and optimizes the winning rate of the model output in paired comparison.

[0114] In the disclosed embodiment, PPO maximizes the cumulative reward by optimizing the strategy while limiting the difference between the new strategy and the old strategy to ensure the stability of training.

[0115] In the disclosed embodiment, KTO introduces a value function and a weight function to characterize human decision-making biases when facing uncertain results (such as loss aversion and overestimation of low-probability events). KTO's loss function combines the nonlinear characteristics of the value function and the probability perception bias of the weight function to better align model output with human preferences.

[0116] In the disclosed embodiment, based on the collected feedback data, a RM model is trained to predict the reward value of any given text response. Using a reinforcement learning algorithm, the large language model is regarded as an intelligent agent, and fine-tuned through interaction with the RM model to optimize its way of generating text.

[0117] In some embodiments, the most suitable target alignment algorithm is selected from a variety of alignment algorithms according to the task requirements. For example, PPO is selected to ensure the stability of the training process, and DPO may be selected to directly optimize the target object preference. For applications in specific fields, KTO can prioritize key terms to improve the quality of professional texts. If the task goal is to optimize the overall generation quality, the RM model can be used to guide the training of the model through feedback scoring.

[0118] In some embodiments, the training samples that meet the target alignment algorithm are screened out from the first sample data set and the second sample data set, including: after the system pre-selects the target alignment algorithm, it screens out samples suitable for training from the first sample set (high satisfaction samples) and the second sample set (low satisfaction samples). For example, the system can select samples with high click-through rate and long residence time as positive feedback (first sample set), and select samples with high skip rate and short residence time as negative feedback (second sample set). By analyzing the characteristics of these samples (such as target object behavior, number of clicks, browsing time, etc.), the system constructs a representative training data set. These data will serve as the basis for model optimization and help to accurately align the preferences of the target object.

[0119] In some implementations, the first large language model is optimized using the training samples, including: after selecting the training samples and defining the optimization target, the system uses a target alignment algorithm (such as PPO or DPO) to optimize the first large language model.

[0120] Taking PPO as an example, the system gradually updates the model's generation mechanism by comparing the old model strategy with the new strategy to ensure that the generated answers are more in line with the preferences of the target object. For example, when the model receives negative feedback samples, it will reduce the generation of similar answers that do not meet the needs of the target object; when processing positive feedback samples, the system will increase the probability of generating similar high-satisfaction answers. By continuously adjusting the model parameters, the system can generate more accurate answers that meet the expectations of the target object. When using the RM model, the system will score the generated answers based on the feedback from the target object, and use these scores as reward signals to guide the training and optimization of the model. The PPO algorithm ensures the stability of model training by controlling the update amplitude, avoids excessive parameter changes in the model during training, and ensures the consistency and rationality of the generated results.

[0121] In this way, by selecting a target alignment algorithm that matches the task and optimizing the model by selecting training samples based on the algorithm, the performance of the model on specific tasks can be significantly improved, and the alignment effect between search answers and preferred features can be improved. By selecting training samples that meet the target alignment algorithm, unnecessary computational overhead can be reduced and the efficiency of model training can be improved.

[0122] In some embodiments, based on the target alignment algorithm, training samples that meet the target alignment algorithm are screened out from the first sample data set and the second sample data set, including: when the target alignment algorithm is DPO, a first answer to the target question is screened out from the first sample data set, and a second answer to the target question is screened out from the second sample data set; and a training sample is generated based on the first answer and the second answer corresponding to the target question.

[0123] In some embodiments, when the target alignment algorithm is DPO, the system directly optimizes the model parameters based on the target object feedback data without building a complex reward model. Traverse the first sample data set, and for each target question, screen out the answer that the target object is most satisfied with or has the highest preference as the first answer. Similarly, traverse the second sample data set, and for the same target question, screen out the answer with low target object satisfaction or obvious shortcomings as the second answer. These answers may cause dissatisfaction to the target object due to insufficient relevance, low accuracy or unclear expression. Pair the first answer and the second answer corresponding to each target question to form a training sample pair. Each sample pair contains a positive example (first answer) and a negative example (second answer) preferred by the target object. Annotate each training sample pair with corresponding labels, and usually use binary labels (such as 1 for positive preference and 0 for negative preference) to distinguish the first answer from the second answer. Arrange the training sample pairs and their labels in the format required for model training to ensure that they can be directly used in the model training process.

[0124] Figure 6 A schematic diagram of large language model optimization is shown, Figure 6 As shown, the SFT sample is used to fine-tune the general large language model to generate the first large language model; after the first large language model is launched, it receives questions input by the target object, obtains original data from multiple information sources, and generates search answers; displays the search answers on the search page; collects feedback data for the search answers and stores the feedback data; generates DPO samples based on the mined preference features; and performs reinforcement learning on the first large language model based on the DPO samples to generate the second large language model.

[0125] In this way, by directly selecting the first answer and the second answer from the target object's preference feedback, the generated training samples can more accurately reflect the target object's real needs and preferences, thereby improving the pertinence and effectiveness of model training. Compared with traditional random sampling or uniform sampling methods, the DPO algorithm can locate the most valuable training samples for model optimization more quickly, thereby improving the efficiency of model training. Since the training samples are generated directly based on the target object's preference feedback, the optimized model can better meet the expectations and needs of the target object when dealing with similar problems, thereby improving the generation quality.

[0126] The present disclosure provides a search processing method. Figure 7 1 is a flowchart of a search processing method according to an embodiment of the present disclosure. The search processing method can be applied to electronic devices, including but not limited to fixed devices and / or mobile devices. In some possible implementations, the search processing method can also be implemented by a processor calling a computer-readable instruction stored in a memory. Figure 7 As shown, the search processing method includes:

[0127] S701: receiving a question input by a target object on a search page;

[0128] S702: Generate a search answer matching the question using a pre-trained large language model;

[0129] S703: Display the search answer at the top of the search page;

[0130] S704: Monitor and record the target object's feedback data regarding the search answer; wherein the feedback data is used to optimize the large language model.

[0131] In the disclosed embodiment, the search answer refers to an answer or information generated for a question and closely related to the question. For example, the search answer may be a short text answer or a detailed answer containing multiple paragraphs, pictures or links.

[0132] In some implementations, receiving a question input by a target object on a search page includes: the target object inputs a query question through an input box on the search page, the system uses front-end technology to capture the input of the target object, and sends a request to a back-end server.

[0133] In some embodiments, using a pre-trained large language model to generate a search answer that matches the question includes: the large language model understands the question, retrieves relevant information from its knowledge base, and generates one or more answers that match the question. In this way, the powerful natural language processing capabilities of the large language model are used to quickly generate accurate and relevant search answers.

[0134] In some embodiments, displaying the search answer at the top of the search page includes: the backend server returns the generated search answer to the front end, and the front end uses to render the answer to the top position of the search page. At the same time, an eye-catching style can be designed to attract the attention of the target object. In this way, directly displaying the search answer at the top of the search page can reduce the search cost and time of the target object and improve the search efficiency.

[0135] In some embodiments, the feedback data of the target object for the search answer is monitored and recorded, including: the system collects the feedback data of the target object for the search answer through front-end tracking technology (such as event monitoring, page dwell time statistics, etc.) and possible explicit feedback (such as satisfaction evaluation, like / dislike buttons, etc.). These data are sent to the back-end server for storage and analysis. Subsequently, the parameters or training data of the large language model are adjusted according to the feedback data to optimize its performance. In this way, through a closed-loop feedback mechanism, the large language model is continuously iterated and optimized to make it more in line with the expectations and needs of the target object, thereby improving the accuracy and relevance of the search answer.

[0136] For example, if the target user enters "How is the weather today?" on the search page, the search system will input the question "How is the weather today?" into the big language model, use the big language model to generate an answer, such as "Today is sunny, the temperature is 20-25℃, suitable for outdoor activities.", and display the answer at the top of the search page. After seeing the answer, the target user clicks the "Satisfied" button, and the search system records this feedback and may use it to optimize the model in the future.

[0137] The technical solution of the disclosed embodiment is that the search system can quickly understand the questions input by the target object and generate accurate and relevant search answers through a pre-trained large language model. This instant feedback mechanism reduces the waiting time of the target object and improves the search efficiency. At the same time, the search answer is directly displayed at the top of the search page, so that the target object can quickly obtain the required information, further improving the experience of the target object. The system forms a closed-loop optimization mechanism by monitoring and recording the feedback data of the target object on the search answer. These feedback data provide a valuable basis for model optimization, so that the model can continuously learn and adapt to the preferences and needs of the target object. Through continuous training and adjustment, the performance of the large language model will continue to improve, thereby providing the target object with a better quality and personalized search service.

[0138] In some embodiments, a pre-trained large language model is used to generate a search answer that matches the question, including: collecting raw data related to the question from at least one information source; determining preference features corresponding to the question; inputting the raw data and the preference features into the large language model, and obtaining a search answer generated by the large language model that matches the question and conforms to the preference features.

[0139] In the disclosed embodiments, preference features refer to the personalized needs of the target object for search answers during the search process, such as language style (such as formal, humorous), answer length (such as introduction, detailed analysis), information type (such as text, picture, video), source reliability, etc.

[0140] In some embodiments, collecting raw data related to the question from at least one information source includes: according to the question raised by the target object, the system collects raw data related to the question from multiple information sources (such as databases, online documents, web pages, etc.). These data may include but are not limited to text, pictures, videos and other forms. During the collection process, the search system will consider the timeliness, accuracy and relevance of the data to ensure that the collected data can maximize the accuracy of the data source.

[0141] In some embodiments, determining the preference features corresponding to the question includes: the search system determines the target object's preference features for the search answer based on the target object's historical behavior, search habits, personal settings, etc. The preference features are expressed in an appropriate form (such as labels, weights) so as to be subsequently input into the large language model.

[0142] In some embodiments, the raw data and the preference features are input into the large language model, and the search answer generated by the large language model that matches the question and meets the preference features is obtained, including: the collected raw data and the determined preference features are input into the pre-trained large language model, and the large language model uses its powerful language understanding and generation capabilities to process and analyze the raw data, while considering the preference features of the target object, and generates a search answer that matches the question and meets the preferences of the target object. In the process of generating answers, the large language model will consider multiple factors, such as language fluency, logical rationality, information accuracy, etc., to ensure the quality of the generated answers.

[0143] For example, suppose the question asked by the target object is "How to make pizza at home?" The search system will first collect raw data related to pizza making from multiple information sources, such as recipes, step-by-step instructions, pictures, etc. Then, the search system will determine the target object's preference characteristics based on the target object's historical behavior (such as frequent browsing of food websites) and preference settings (such as liking detailed step-by-step instructions). Finally, these raw data and preference characteristics are input into the large language model, and the model will generate a detailed pizza making guide as the search answer, including the required materials, steps, precautions, etc., and the language style is consistent with the target object's preferences.

[0144] In this way, through the automated data collection and model generation process, manual intervention and waiting time are greatly reduced, search efficiency is improved, and the quality of search answers is improved. Taking into account the preference characteristics of the target object, the generated answers are more in line with the personalized needs of the target object, improving the target object's satisfaction.

[0145] In some embodiments, determining a preference feature corresponding to a question includes: obtaining scene information corresponding to the question; and selecting a preference feature that matches the scene information from a plurality of candidate preference features corresponding to the question as the preference feature of the question.

[0146] In the disclosed embodiment, the scene information refers to the specific scene where the problem occurs, which helps to more accurately understand the intention of the target object.

[0147] For example, scenario information may include but is not limited to the following aspects: Time: The time when the question is asked, such as weekdays, weekends, holidays, etc., which may affect the preferences of the target object. Location: The place where the question occurs, such as home, office, public places, etc. Different places may have different preferences. Social environment: The social environment in which the target object asks the question, such as alone, with friends, in a meeting, etc., which will affect the target object's expected answer style and level of detail. Context: The conversation before the question or the target object's previous operation behavior, which helps to understand the target object's current needs and expectations.

[0148] In some embodiments, obtaining the scene information corresponding to the question includes: receiving the question input by the target object, parsing the question through natural language processing technology, and extracting key information such as keywords and semantic roles. Further analyzing the context of the question, including the historical query records, current time, geographic location, etc. of the target object, to construct comprehensive scene information.

[0149] In the disclosed embodiment, the preference feature is an important basis for generating answers, guiding the large language model to generate answers that are more in line with the expectations of the target object and the needs of the scenario.

[0150] For example, candidate preference features may include: Language style: such as formal, informal, humorous, rigorous, etc., choose the appropriate language style according to the scene and the habits of the target object. Information volume: The target object expects the answer to be concise and clear or detailed and in-depth, which depends on the specific needs of the target object and the scene requirements. Professionalism: For some professional questions, the target object may prefer professional and accurate answers. Emotional tendency: According to the mood of the target object and the atmosphere of the scene, choose a positive, neutral or slightly emotional answer.

[0151] In some embodiments, the search system pre-builds a candidate preference feature library, including: pre-defining a set of preference feature sets according to business needs and question types, these features include but are not limited to language style, information type, etc. Mapping keywords and semantic information in the question with the pre-defined preference features to generate multiple possible candidate preference features.

[0152] In some embodiments, a preference feature that matches the scene information is selected from a plurality of candidate preference features corresponding to the question as the preference feature of the question, including: matching the scene information with the candidate preference features using a preset rule or algorithm. These rules may be based on a statistical model, a machine learning model, or an expert knowledge base. For each matching preference feature, a corresponding weight value is calculated based on its importance and relevance in the current scene. Based on the weight value, the preference feature that best matches the scene information is selected as the final result. The selected preference feature is output in a structured or easy-to-understand manner for subsequent large language model processing.

[0153] In some implementations, the large language model can automatically adjust the content and style of the answer to meet the needs of different scenarios (such as work, study, entertainment, etc.) according to the different scenarios of the target object. For example, in the work scenario, the model can generate more professional and accurate answers; while in the entertainment scenario, it can generate more relaxed and interesting answers.

[0154] In some embodiments, when generating answers, the large language model can use the summarized human preferences as part of the input, and adjust the parameters and calculation processes within the model to make the generated answers more in line with the preferences of the target object. For example, for target objects that prefer concise and clear answers, the model can generate more refined answers; and for target objects that prefer detailed explanations, it can generate answers that contain more background information and explanations. According to the language style and preferences of the target object, the large language model can adjust the language style of the answer. For example, for target objects that use a formal language style, the model can generate more rigorous and professional answers; and for target objects that use a relaxed and humorous language style, it can generate more intimate and interesting answers.

[0155] In some implementations, in addition to generating text answers, the large language model can also generate richer answers by combining multimodal information such as images, audio, and video. Based on the preferences of the target object and the needs of the scenario, the large language model can select appropriate multimodal information to present the answer, providing a more intuitive and vivid answer experience.

[0156] For example, suppose the target object asks: "Are there any restaurants suitable for family dinners tonight?" Parse the question and extract the keywords "tonight", "family dinner", and "restaurant recommendation", while considering the target object's historical query records and current geographic location and other scene information. Determine candidate preference features such as restaurant type (such as Chinese food, Western food), price range, environment atmosphere (such as quiet, lively), and dish taste. Based on the target object's location and current time (tonight), as well as the needs of family dinners, the system may match preference features such as "Chinese food", "moderately priced", and "quiet environment". Recommend several Chinese restaurants that meet these preference features to the target object.

[0157] Figure 8 The architecture diagram of the search system for optimizing search answers is shown in FIG. Figure 8 As shown, the system is divided into an offline learning system and an online system. After the answer generation model is generated based on the base model, the answer generation model is put online. The answer generation model of the online system generates multiple versions of answers or a single version of answers based on the input of the target object, collects the feedback data of the target object, and stores the feedback data in the log library. The offline learning system extracts feedback data from the log and uses the satisfaction model to determine the preference features; the answer generation model is optimized based on the preference features so that the search answers output by the answer generation model are aligned with the preference features. The model can continuously adjust and optimize its internal parameters and algorithms to better match the preferences of the target object.

[0158] In this way, by combining scenario information and preference characteristics, the target object's intention can be understood more accurately, thereby improving the accuracy and efficiency of problem handling. The service content and method are automatically adjusted according to the preference characteristics of different target objects, providing personalized services or suggestions and enhancing the user experience.

[0159] The present disclosure provides a processing device for generative search data, such as Fig. 9As shown, the device may include: a collection module 901, which is used to collect feedback data of the target object on the search answer displayed at the first position of the search page, wherein the search answer is an answer generated by the search system using the first language model; an analysis module 902, which is used to analyze the feedback data to identify key factors affecting the satisfaction of the search answer; a sample mining module 903, which is used to mine a first sample set and a second sample set from the feedback data in combination with the key factors, wherein the first sample set includes samples with a satisfaction greater than a first threshold, and the second sample set includes samples with a satisfaction less than a second threshold; an optimization and adjustment module 904, which is used to optimize the first language model based on the first sample set and / or the second sample set to obtain a second language model, wherein the search answer generated by the second language model has a better match with the target object preference than the first language model; a deployment module 905, which is used to deploy the second language model into the search system to replace the first language model.

[0160] In some embodiments, the feedback data includes: a first type of feedback data and a second type of feedback data; wherein the first type of feedback data includes historical behavior data during the target object search process; the second type of feedback data includes survey data that the system actively collects from the target object; wherein the first type of feedback data and the second type of feedback data are data generated by a single round of generative question and answering.

[0161] In some embodiments, the feedback data also includes: a third category of feedback data and a fourth category of feedback data; wherein the third category of feedback data includes data generated during multiple rounds of interactive dialogues; the fourth category of feedback data includes data on the depth of multiple rounds of interactive dialogues; wherein the third category of feedback data and the fourth category of feedback data are data generated by multiple rounds of generative question and answering.

[0162] In some embodiments, the collection module 901 includes: a first selection submodule, used to select a target answer generation algorithm that matches the current task requirements from a plurality of answer generation algorithms; a first generation submodule, used to generate multiple versions of search answers for each query word for a given set of query words using the target answer generation algorithm; and a first collection submodule, used to display multiple versions of the search answers to the actual terminal or the simulated terminal in turn to collect feedback data of the target object on each version of the search answer.

[0163] In some embodiments, the analysis module 902 includes: a preprocessing submodule, used to preprocess the feedback data to obtain preprocessed data; an extraction submodule, used to extract multiple features that affect satisfaction from the preprocessed data; and an analysis submodule, used to use statistical algorithms to analyze multiple features and identify key factors related to satisfaction.

[0164] In some embodiments, the sample mining module 903 includes: an input submodule, used to input feedback data into a pre-trained satisfaction model; wherein the satisfaction model is used to predict the target object's satisfaction with the search answer in combination with key factors; and an acquisition submodule, used to acquire a first sample set and a second sample set output by the satisfaction model.

[0165] In some embodiments, the optimization and adjustment module 904 includes: a second selection submodule, used to select a target alignment algorithm that matches the current task from a plurality of alignment algorithms; a second generation submodule, used to screen out training samples that meet the target alignment algorithm from the first sample data set and the second sample data set based on the target alignment algorithm; a training submodule, used to use the target alignment algorithm as a training guide and use the training samples to optimize the first large language model.

[0166] In some embodiments, the second generation submodule is used to: when the target alignment algorithm is the direct preference optimization (DPO) algorithm, filter out a first answer to the target question from the first sample data set, and filter out a second answer to the target question from the second sample data set; generate a training sample based on the first answer and the second answer corresponding to the target question.

[0167] For the description of specific functions and examples of each module and submodule of the device in the embodiment of the present disclosure, reference can be made to the relevant description of the corresponding steps in the above method embodiment, which will not be repeated here.

[0168] The generative search data processing device of the disclosed embodiment can collect multivariate feedback data and automatically mine high-confidence samples, thereby realizing a self-optimization cycle of the generative large language model, improving the quality of the search answer displayed first, and improving the efficiency of the target object in obtaining the answer.

[0169] The present disclosure provides a search processing device, such as Fig.10 As shown, the device may include: a receiving module 1001, used to receive a question input by a target object on a search page; a generating module 1002, used to generate a search answer matching the question using a pre-trained large language model; a display module 1003, used to display the search answer at the top of the search page; a monitoring and recording module 1004, used to monitor and record the feedback data of the target object regarding the search answer; wherein the feedback data is used to optimize the large language model.

[0170] In some embodiments, the generation module 1002 includes: a second collection submodule, used to collect raw data related to the question from at least one information source; a determination submodule, used to determine the preference features corresponding to the question; and a third generation submodule, used to input the raw data and the preference features into the large language model to obtain a search answer generated by the large language model that matches the question and conforms to the preference features.

[0171] In some embodiments, the determination submodule is used to: obtain scenario information corresponding to the question; and select a preference feature that matches the scenario information from a plurality of candidate preference features corresponding to the question as the preference feature of the question.

[0172] For the description of specific functions and examples of each module and submodule of the device in the embodiment of the present disclosure, reference can be made to the relevant description of the corresponding steps in the above method embodiment, which will not be repeated here.

[0173] The search processing device of the disclosed embodiment can improve the quality of the search answer displayed first and increase the efficiency of the target object in obtaining the answer.

[0174] The present disclosure provides a scenario diagram of a search processing method, such as Fig.11 shown.

[0175] As mentioned above, the search processing method provided by the embodiment of the present disclosure is applied to electronic devices. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smart phones, wearable devices, and other similar computing devices.

[0176] Specifically, the electronic device can perform the following operations: receive questions entered by the target object on the search page; use a pre-trained large language model to generate search answers that match the questions; display the search answers at the top of the search page; monitor and record the target object's feedback data on the search answers; wherein the feedback data is used to optimize the large language model.

[0177] It should be understood that Fig.11 The scene diagram shown is only illustrative and not restrictive. Those skilled in the art can Fig.11 Various obvious changes and / or substitutions can be made to the examples, and the resulting technical solutions still fall within the scope of the disclosure of the embodiments of the present disclosure.

[0178] In the technical solution disclosed herein, the acquisition, storage and application of user personal information involved are in compliance with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0179] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium and a computer program product.

[0180] Fig.12 A schematic block diagram of an example electronic device 1200 that can be used to implement an embodiment of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or required herein.

[0181] like Fig.12 As shown, the device 1200 includes a computing unit 1201, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 1202 or a computer program loaded from a storage unit 1208 to a random access memory (RAM) 1203. In RAM 1203, various programs and data required for the operation of the device 1200 can also be stored. The computing unit 1201, ROM 1202, and RAM 1203 are connected to each other via a bus 1204. An input / output (I / O) interface 1205 is also connected to the bus 1204.

[0182] A number of components in the device 1200 are connected to the I / O interface 1205, including: an input unit 1206, such as a keyboard, a mouse, etc.; an output unit 1207, such as various types of displays, speakers, etc.; a storage unit 1208, such as a disk, an optical disk, etc.; and a communication unit 1209, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 1209 allows the device 1200 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0183] The computing unit 1201 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 1201 include, but are not limited to, a CPU, a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The computing unit 1201 performs the various methods and processes described above, such as a processing method for generative search data and / or a search processing method. For example, in some embodiments, the processing method for generative search data and / or the search processing method may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit 1208. In some embodiments, part or all of the computer program may be loaded and / or installed on the device 1200 via the ROM 1202 and / or the communication unit 1209. When the computer program is loaded into RAM 1203 and executed by the computing unit 1201, one or more steps of the processing method for generative search data and / or the search processing method described above may be performed. Alternatively, in other embodiments, the computing unit 1201 may be configured to perform the processing method for generative search data and / or the search processing method in any other appropriate manner (e.g., by means of firmware).

[0184] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0185] The program code for implementing the method of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable generative search data processing device, so that the program code, when executed by the processor or controller, implements the functions / operations specified in the flow chart and / or block diagram. The program code may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0186] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or device, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory, a read-only memory, an erasable programmable read-only memory (EPROM), a flash memory, an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0187] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a cathode ray tube (CRT) or a liquid crystal display (LCD) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0188] The systems and techniques described herein can be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: Local Area Networks (LANs), Wide Area Networks (WANs), and the Internet.

[0189] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship of client and server is generated by computer programs running on respective computers and having a client-server relationship with each other. The server may be a cloud server, a server of a distributed system, or a server combined with a blockchain.

[0190] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and this document does not limit this.

[0191] The above specific implementations do not constitute a limitation on the protection scope of the present disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the principles of the present disclosure shall be included in the protection scope of the present disclosure.

Claims

1. A method for processing generative search data, comprising: Collecting feedback data from the target object on the search answer displayed first on the search page, wherein the search answer is an answer generated by the search system using the first language model; Analyzing the feedback data to identify key factors that affect satisfaction with the search answers; In combination with the key factors, a first sample set and a second sample set are mined from the feedback data, wherein the first sample set includes samples with a satisfaction level greater than a first threshold, and the second sample set includes samples with a satisfaction level less than a second threshold; Based on the first sample set and / or the second sample set, optimizing the first large language model to obtain a second large language model, wherein the matching degree between the search answer generated by the second large language model and the target object preference is better than that of the first large language model; The second largest language model is deployed into the search system to replace the first largest language model.

2. The method according to claim 1, wherein: The feedback data includes: first-category feedback data and second-category feedback data; wherein, the first-category feedback data includes historical behavior data during the target object search process; the second-category feedback data includes survey data actively collected by the system from the target object; wherein, the first-category feedback data and the second-category feedback data are data generated by a single round of generative question and answering.

3. The method according to claim 2, wherein: The feedback data also includes: third-category feedback data and fourth-category feedback data; wherein, the third-category feedback data includes data generated during multiple rounds of interactive dialogues; the fourth-category feedback data includes data on the depth of multiple rounds of interactive dialogues; wherein, the third-category feedback data and the fourth-category feedback data are data generated by multiple rounds of generative question and answering.

4. The method according to any one of claims 1 to 3, wherein: Collect the target audience's feedback data on the search answer displayed at the top of the search page, including: Select a target answer generation algorithm that matches the current task requirements from a variety of answer generation algorithms; For a given set of query terms, using the target answer generation algorithm to generate multiple versions of search answers for each query term; The multiple versions of the search answers are presented to the actual terminal or the simulated terminal in turn to collect the target object's feedback data on each version of the search answer.

5. The method according to claim 1, wherein: The analyzing of the feedback data to identify key factors affecting the satisfaction of the search answer includes: Preprocessing the feedback data to obtain preprocessed data; Extracting multiple features that affect satisfaction from the preprocessed data; Statistical algorithms are used to analyze the multiple features to identify key factors related to satisfaction.

6. The method according to claim 1, wherein: The step of mining a first sample set and a second sample set from the feedback data in combination with the key factors includes: Inputting the feedback data into a pre-trained satisfaction model; wherein the satisfaction model is used to predict the target object's satisfaction with the search answer in combination with the key factors; A first sample set and a second sample set output by the satisfaction model are obtained.

7. The method according to claim 1, wherein: The optimizing the first large language model based on the first sample set and / or the second sample set to obtain a second large language model includes: Select a target alignment algorithm that matches the current task from a variety of alignment algorithms; Based on the target alignment algorithm, screening out training samples that meet the target alignment algorithm from the first sample data set and the second sample data set; The target alignment algorithm is used as a training guide, and the first language model is optimized using the training samples.

8. The method according to claim 7, wherein: The step of selecting training samples that meet the target alignment algorithm from the first sample data set and the second sample data set based on the target alignment algorithm includes: In a case where the target alignment algorithm is a direct preference optimization (DPO) algorithm, a first answer to a target question is screened out from the first sample data set, and a second answer to the target question is screened out from the second sample data set; The training sample is generated based on the first answer and the second answer corresponding to the target question.

9. A search processing method, comprising: Receive questions entered by the target audience on the search page; Using a pre-trained large language model, generate search answers that match the question; Display the search answer at the first position of the search page; Monitor and record the target object's feedback data regarding the search answer; wherein the feedback data is used to optimize the large language model.

10. The method according to claim 9, wherein: The method of using a pre-trained large language model to generate a search answer matching the question includes: Collecting raw data related to the problem from at least one information source; determining a preference characteristic corresponding to the problem; The original data and the preference features are input into the large language model, and a search answer generated by the large language model that matches the question and conforms to the preference features is obtained.

11. The method according to claim 10, wherein: The determining of the preference feature corresponding to the question includes: Obtaining scenario information corresponding to the problem; A preference feature matching the scenario information is selected from a plurality of candidate preference features corresponding to the question as the preference feature of the question.

12. A processing device for generative search data, comprising: A collection module, used to collect feedback data of the target object on the search answer displayed first on the search page, wherein the search answer is an answer generated by the search system using the first language model; An analysis module, used to analyze the feedback data to identify key factors affecting the satisfaction of the search answer; A sample mining module, configured to mine a first sample set and a second sample set from the feedback data in combination with the key factors, wherein the first sample set includes samples with a satisfaction level greater than a first threshold, and the second sample set includes samples with a satisfaction level less than a second threshold; an optimization and adjustment module, configured to optimize the first large language model based on the first sample set and / or the second sample set to obtain a second large language model, wherein the matching degree between the search answer generated by the second large language model and the target object preference is better than that of the first large language model; A deployment module is used to deploy the second largest language model into the search system to replace the first largest language model.

13. The device according to claim 12, wherein: The feedback data includes: first-category feedback data and second-category feedback data; wherein, the first-category feedback data includes historical behavior data during the target object search process; the second-category feedback data includes survey data actively collected by the system from the target object; wherein, the first-category feedback data and the second-category feedback data are data generated by a single round of generative question and answering.

14. The device according to claim 12, wherein: The feedback data also includes: third-category feedback data and fourth-category feedback data; wherein, the third-category feedback data includes data generated during multiple rounds of interactive dialogues; the fourth-category feedback data includes data on the depth of multiple rounds of interactive dialogues; wherein, the third-category feedback data and the fourth-category feedback data are data generated by multiple rounds of generative question and answering.

15. The device according to any one of claims 12 to 14, wherein: The collection module comprises: The first selection submodule is used to select a target answer generation algorithm that matches the current task requirement from a plurality of answer generation algorithms; A first generation submodule is used to generate multiple versions of search answers for each query word using the target answer generation algorithm for a given query word set; The first collection submodule is used to display the multiple versions of the search answers to the actual terminal or the simulated terminal in turn, so as to collect the feedback data of the target object on each version of the search answer.

16. The device according to claim 12, wherein: The analysis module comprises: A preprocessing submodule, used to preprocess the feedback data to obtain preprocessed data; An extraction submodule, used for extracting a plurality of features affecting satisfaction from the preprocessed data; The analysis submodule is used to analyze the multiple features using statistical algorithms to identify key factors related to satisfaction.

17. The device according to claim 12, wherein: The sample mining module comprises: An input submodule, used to input the feedback data into a pre-trained satisfaction model; wherein the satisfaction model is used to predict the satisfaction of the target object with the search answer in combination with the key factors; The acquisition submodule is used to acquire the first sample set and the second sample set output by the satisfaction model.

18. The device according to claim 12, wherein: The optimization and adjustment module comprises: The second selection submodule is used to select a target alignment algorithm that matches the current task from a variety of alignment algorithms; A second generating submodule is used to select training samples that meet the target alignment algorithm from the first sample data set and the second sample data set based on the target alignment algorithm; A training submodule is used to use the target alignment algorithm as a training guide and optimize the first large language model using the training samples.

19. The device according to claim 18, wherein: The second generation submodule is used for: In a case where the target alignment algorithm is a direct preference optimization (DP0) algorithm, a first answer to a target question is selected from the first sample data set, and a second answer to the target question is selected from the second sample data set; The training sample is generated based on the first answer and the second answer corresponding to the target question.

20. A search processing device, comprising: A receiving module, used for receiving questions input by a target object on a search page; A generation module, used to generate a search answer matching the question using a pre-trained large language model; A display module, used to display the search answer at the first position of the search page; A monitoring and recording module is used to monitor and record the feedback data of the target object regarding the search answer; wherein the feedback data is used to optimize the large language model.

21. The device according to claim 20, wherein: The generating module comprises: A second collection submodule, configured to collect raw data related to the problem from at least one information source; A determination submodule, used to determine the preference feature corresponding to the question; The third generation submodule is used to input the original data and the preference feature into the large language model, and obtain a search answer generated by the large language model that matches the question and conforms to the preference feature.

22. The device according to claim 21, wherein The determining submodule is used to: Obtaining scenario information corresponding to the problem; A preference feature matching the scenario information is selected from a plurality of candidate preference features corresponding to the question as the preference feature of the question.

23. An electronic device, comprising: at least one processor; as well as a memory communicatively connected to at least one processor; wherein, The memory stores instructions that can be executed by at least one processor, and the instructions are executed by at least one processor to enable the at least one processor to perform the method of any one of claims 1-11.

24. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are for causing a computer to perform a method according to any one of claims 1-11.

25. A computer program product comprising a computer program stored on a storage medium, the computer program implementing the method according to any one of claims 1 to 11 when executed by a processor.

Images