Method, device and equipment for generating search data, and storage medium

Abstract

The present disclosure provides a processing method and device for generating search data, equipment and storage medium. The present disclosure relates to the technical field of computers, especially to the technical field of search technology and large model technology, and can be used in application scenarios such as generative search, intelligent question answering, intelligent recommendation, etc. The method comprises: collecting feedback data of a target object on a search page first displayed search answer; analyzing the feedback data to identify key factors affecting the satisfaction of the search answer; combining the key factors to mine a first sample set and a second sample set from the feedback data; based on the first sample set and / or the second sample set, optimizing the first large language model to obtain a second large language model; deploying the second large language model to the search system to replace the first large language model. According to the scheme of the present disclosure, high-confidence samples can be automatically mined, a self-optimization cycle of the generative large language model is realized, and the quality of the search answer is improved.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technology, in particular to the fields of search technology and large model technology, and can be used in application scenarios such as generative search, intelligent question-answering, and intelligent recommendation. Specifically, it relates to a method, apparatus, device, and storage medium for processing generative search data. Background Art

[0002] In related technologies, optimizing generative search systems typically relies on manually annotated data, which is time- and financially expensive to obtain, and it's difficult to ensure that the optimization direction aligns with actual preferences. Furthermore, traditional user feedback signals are complex and noisy, making them difficult to directly apply to optimizing large generative language models. Summary of the Invention

[0003] The present disclosure provides a method, apparatus, device, and storage medium for processing generative search data.

[0004] According to a first aspect of the present disclosure, a method for processing generative search data is provided, the method comprising: collecting feedback data of a target object on a search answer displayed first on a search page, wherein the search answer is an answer generated by a search system using a first language model; analyzing the feedback data to identify key factors affecting satisfaction with the search answer; mining a first sample set and a second sample set from the feedback data based on the key factors, the first sample set including samples with satisfaction greater than a first threshold, and the second sample set including samples with satisfaction less than a second threshold; optimizing the first language model based on the first sample set and / or the second sample set to obtain a second language model, wherein the search answer generated by the second language model has a better match with the target object's preference than the first language model; and deploying the second language model into the search system to replace the first language model.

[0005] According to a second aspect of the present disclosure, a search processing method is provided, which includes: receiving a question entered by a target object on a search page; using a pre-trained large language model to generate a search answer that matches the question; displaying the search answer at the top of the search page; monitoring and recording feedback data from the target object regarding the search answer; wherein the feedback data is used to optimize the large language model.

[0006] According to a third aspect of the present disclosure, a device for processing generative search data is provided, the device comprising: a collection module for collecting feedback data of a target object on a search answer displayed first on a search page, wherein the search answer is an answer generated by a search system using a first language model; an analysis module for analyzing the feedback data to identify key factors affecting satisfaction with the search answer; a sample mining module for mining a first sample set and a second sample set from the feedback data in combination with the key factors, the first sample set including samples with satisfaction greater than a first threshold, and the second sample set including samples with satisfaction less than a second threshold; an optimization and adjustment module for optimizing the first language model based on the first sample set and / or the second sample set to obtain a second language model, wherein the search answer generated by the second language model has a better match with the target object's preference than the first language model; and a deployment module for deploying the second language model into the search system to replace the first language model.

[0007] According to a fourth aspect of the present disclosure, a search processing device is provided, which includes: a receiving module for receiving a question input by a target object on a search page; a generating module for generating a search answer that matches the question using a pre-trained large language model; a display module for displaying the search answer at the top of the search page; a monitoring and recording module for monitoring and recording feedback data of the target object regarding the search answer; wherein the feedback data is used to optimize the large language model.

[0008] According to a fifth aspect of the present disclosure, there is provided an electronic device, including:

[0009] at least one processor; and

[0010] a memory communicatively connected to the at least one processor; wherein,

[0011] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform any method in the embodiments of the present disclosure.

[0012] According to a sixth aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to enable the computer to execute any method according to the embodiments of the present disclosure.

[0013] According to a seventh aspect of the present disclosure, a computer program product is provided, comprising a computer program, which implements any method according to the embodiments of the present disclosure when executed by a processor.

[0014] By adopting the solution disclosed in the present invention, by collecting multi-dimensional feedback data and automatically mining high-confidence samples, a self-optimization cycle of the generative large language model is realized, the quality of the search answer displayed first is improved, and the efficiency of the target object in obtaining the answer is improved.

[0015] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present disclosure, nor are they intended to limit the scope of the present disclosure. Other features of the present disclosure will become readily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] The accompanying drawings are provided to facilitate a better understanding of the present invention and do not constitute a limitation of the present disclosure.

[0017] Figure 1 is a flowchart of a method for processing generative search data according to an embodiment of the present disclosure;

[0018] Figure 2 is an architectural diagram of a generative search data flywheel system according to an embodiment of the present disclosure;

[0019] Figure 3 is a schematic diagram of generating multiple versions of answers based on the same query word according to an embodiment of the present disclosure;

[0020] Figure 4 2 is a comparative diagram of generating search answers using different versions of large language models according to an embodiment of the present disclosure;

[0021] Figure 5 is a schematic diagram of an architecture for accumulating feedback data according to an embodiment of the present disclosure;

[0022] Figure 6 is a schematic diagram of large language model optimization according to an embodiment of the present disclosure;

[0023] Figure 7 is a flowchart of a search processing method according to an embodiment of the present disclosure;

[0024] Figure 8 is an architectural diagram for optimizing search answers according to the disclosed search system;

[0025] Figure 9 is a structural diagram of a device for processing generative search data according to an embodiment of the present disclosure;

[0026] Figure 10 is a structural diagram of a search processing device according to an embodiment of the present disclosure;

[0027] Figure 11 is a scenario diagram of a search processing method according to an embodiment of the present disclosure;

[0028] Figure 12is a structural schematic diagram of an electronic device for implementing the processing method and / or search processing method of the generative search data of the embodiments of the present disclosure. DETAILED DESCRIPTION

[0029] Exemplary embodiments of the present disclosure are described below with reference to the accompanying drawings, which include various details of the embodiments of the present disclosure to assist in understanding, and should be considered as merely exemplary. Thus, those of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope of the present disclosure. Also, for the sake of brevity and clarity, descriptions of well-known functions and constructions are omitted from the following description.

[0030] The term "and / or" herein is merely a description of an association relationship of associated objects, and can represent three relationships, for example, A and / or B can represent three cases of A alone, A and B together, and B alone. The term "at least one" herein represents any one of a plurality or any combination of at least two of a plurality, for example, including at least one of A, B, and C can represent including any one or more elements selected from the set consisting of A, B, and C. The terms "first" and "second" herein represent referring to and distinguishing a plurality of similar technical terms, and do not mean the order or mean only two, for example, the first feature and the second feature refer to two types / two features, and the first feature can be one or more, and the second feature can also be one or more.

[0031] In addition, in order to better illustrate the present disclosure, a large number of specific details are given in the following detailed description. Those skilled in the art should understand that the present disclosure can also be implemented without certain specific details. In some examples, methods, means, elements and circuits well known to those skilled in the art are not described in detail, in order to highlight the main idea of the present disclosure.

[0032] Before introducing the technical solutions of the embodiments of the present disclosure, the technical terms that can be used in the present disclosure are further explained:

[0033] Large Language Model (LLM): refers to a deep learning model trained using a large amount of text data, with powerful language understanding and generation capabilities, capable of handling a variety of natural language tasks.

[0034] Generative search: a search technology based on large language models, different from traditional keyword matching-based search engines, generative search generates natural language form answers by understanding and analyzing user query intent, rather than just providing a list of links. It can use the power of large language models to integrate multi-source information and create more user-demand-oriented, structured and complete answers.

[0035] Alignment algorithms: Alignment algorithms are used to align model outputs with human preferences. During generative model training, alignment algorithms can help models better align with the expectations or needs of target users, improving the relevance and satisfaction of search results.

[0036] Target audience preference: This refers to the target audience's inclination and preference for a certain type of outcome, answer, or information in a specific situation or when using a system. It reflects the content form, structure, style, or type that the target audience is more satisfied with or more willing to accept when faced with multiple choices or answers.

[0037] Supervised Fine-Tuning (SFT): is a technique for fine-tuning a pre-trained model using supervised learning. In this process, the model learns how to improve its prediction ability on a specific task based on given labeled data.

[0038] In related technologies, generative search optimization still relies heavily on large amounts of manually annotated data, resulting in high time and economic costs for data acquisition, low optimization efficiency, and uncertainty about whether optimization targets align with actual preferences. Generative search suffers from the following issues: 1. Generative search feedback signals are scarce; 2. Feedback signals lack practical semantics, making them difficult to optimize directly. This lack of semantic meaning can hinder large models from understanding the signals, such as skip clicks and browsing time, making them unsuitable for direct optimization. 3. Traditional ranking feedback signals are complex and noisy, making them difficult to leverage.

[0039] Therefore, data flywheels in generative search scenarios have been difficult to apply on a large scale.

[0040] To at least partially address one or more of the aforementioned and other potential issues, this disclosure proposes a method for processing generative search data. By collecting multivariate feedback data and automatically mining high-confidence samples, this method implements a self-optimization cycle for a generative large language model, effectively addressing issues such as high data acquisition costs and inconsistencies between optimization directions and preferences. This method not only improves sample mining efficiency but also enhances the quality of the top-ranked search answers, increasing the efficiency with which target users can access answers.

[0041] The present disclosure provides a method for processing generative search data. Figure 1It is a flow chart of a method for processing generative search data according to an embodiment of the present disclosure, and the method for processing generative search data can be applied to a device for processing generative search data. The device for processing generative search data is located in an electronic device. The electronic device includes but is not limited to fixed devices and / or mobile devices. For example, fixed devices include but are not limited to servers, and servers can be cloud servers or ordinary servers. For example, mobile devices include but are not limited to: mobile phones, tablet computers. In some possible implementations, the method for processing generative search data can also be implemented by a processor calling computer-readable instructions stored in a memory. For example Figure 1 As shown, the method for processing the generative search data includes:

[0042] S101. Collecting feedback data from a target user on a search answer displayed first on a search page, where the search answer is an answer generated by a search system using a first language model.

[0043] S102. Analyze the feedback data to identify key factors that affect satisfaction with search answers;

[0044] S103. Mining a first sample set and a second sample set from the feedback data based on key factors, where the first sample set includes samples with a satisfaction level greater than a first threshold, and the second sample set includes samples with a satisfaction level less than a second threshold;

[0045] S104: Optimizing the first language model based on the first sample set and / or the second sample set to obtain a second language model, wherein the search answer generated by the second language model has a better match with the target object preference than the first language model;

[0046] S105: Deploy the second largest language model to the search system to replace the first largest language model.

[0047] In the disclosed embodiments, the first language model is the model used by the search system to generate search answers at the first moment. The second language model is the model used by the search system to generate search answers at the second moment. The second moment is later than the first moment. The second language model is typically more accurate in generating search answers that match the target object's preferences than the first language model, resulting in a higher degree of matching and better performance.

[0048] In the disclosed embodiment, the first threshold and the second threshold can be set according to actual needs. For example, based on the analysis results, the first threshold (e.g., satisfaction score ≥ 4, with a full score of 5) and the second threshold (e.g., satisfaction score ≤ 2, with a full score of 5) are set to distinguish high-quality and low-quality search answer samples.

[0049] In some embodiments, feedback data is analyzed to identify key factors that influence satisfaction with search answers, including using statistical analysis and machine learning techniques (such as logistic regression and decision trees) to conduct in-depth analysis of the collected feedback data to identify key factors that influence satisfaction with search answers. These factors may include the relevance, accuracy, completeness, readability, and timeliness of the answers.

[0050] In some embodiments, optimizing the first large language model based on the first sample set and / or the second sample set to obtain a second large language model includes: utilizing high-quality samples from the first sample set to fine-tune the first large language model through supervision, thereby enhancing its ability to generate search answers that meet the preferences of the target subject. Furthermore, low-quality samples from the second sample set may be used as negative samples to improve the model's ability to identify incorrect answers through methods such as comparative learning. After multiple iterations of training, an optimized second large language model is obtained.

[0051] In some implementations, the optimized second language model is deployed in the search system, replacing the original first language model. The performance of the new model is monitored, including metrics such as satisfaction with search answers, click-through rate of target objects, and dwell time, and the model is continuously optimized based on new feedback data.

[0052] Figure 2 The architecture diagram of the generative search data flywheel system is shown as Figure 2 As shown in the figure, the system architecture includes three parts: online feedback signal collection, data mining and alignment training.

[0053] During the online feedback signal collection phase, the large language model obtains resources from multiple information sources based on the query and generates search answers; it collects various feedback signals online and stores them in logs.

[0054] During the data mining phase, if K answers are generated based on the same query and reference content, scores are generated for each of the K answers. The highest-scoring and lowest-scoring answers for the query are selected from the K scores to form a sample of scored pairs. If a single answer is generated based on the same query and reference content, a score is generated for this answer. If the score is high, the sample is stored in a high-satisfaction single-point sample set; if the score is low, the sample is stored in a low-satisfaction single-point sample set. Figure 3 The diagram shows a schematic diagram of generating multiple versions of answers based on the same query word. Two versions of search answers are generated based on the same query word "What are the three searches and eight pairs?". According to the feedback data, Figure 3 The search answers shown on the right are rated higher than Figure 3The left image shows the search answer rating. Based on this feedback data, the first language model was optimized to produce the second language model. After deploying the new model, the answers generated by the search system better meet the expectations and needs of the target audience, thereby improving the efficiency of obtaining answers and increasing satisfaction.

[0055] During the alignment training phase, paired sample sets, high-satisfaction single-point sample sets, and low-satisfaction single-point sample sets are manually reviewed and annotated to generate high-quality sample sets. We select any of a variety of alignment algorithms, select appropriate samples from the data-mined sample sets, and optimize the large language model to generate the optimized large language model. Figure 4 A comparison diagram of different versions of large language models generating search answers is shown, as shown in Figure 4 As shown, the search answer generated by the first language model is as follows Figure 4 As shown in the left figure, the search answer generated by the second largest language model is as follows Figure 4 As shown in the figure on the right, it is obvious that the search answers generated by the optimized large language model are more accurate and richer.

[0056] In this way, by analyzing feedback data to identify the preferences of the target object, and using the identified preference data to continuously optimize the large language model, the search system can self-learn and evolve, improve the accuracy and relevance of search answers, and make the search results more in line with the expectations and needs of the target object.

[0057] In some embodiments, the feedback data includes: first-category feedback data and second-category feedback data; wherein, the first-category feedback data includes historical behavior data during the target object search process; the second-category feedback data includes survey data that the system actively collects from the target object; wherein, the first-category feedback data and the second-category feedback data are data generated by a single round of generative question and answer.

[0058] In the disclosed embodiments, single-round generative question-answering refers to a one-time question-answering interaction between a target and the system. The target enters a query, and the system generates and returns an answer. This entire process involves no subsequent interaction, and the answer does not rely on contextual information or involve a continuous conversation. The system simply generates a one-time result based on a single input from the target.

[0059] In the disclosed embodiments, multi-round generative question-answering refers to a question-answering process involving multiple interactions between the target and the system. During this process, the system's response is based on the current question and previous contextual information. The target can ask follow-up questions or seek further clarification, and the system will generate a coherent response. Multi-round generative question-answering simulates natural conversation, and responses can be gradually deepened or clarified as the conversation progresses.

[0060] In some embodiments, the first type of feedback data is historical behavior data in the target object's search process. This type of data mainly refers to the behavior records generated by the target object during the search process, including but not limited to search keywords, clicked search results, dwell time, page scrolling behavior, returning to search or re-entering new keywords, etc. Among them, the search keyword refers to the original question or query word input by the target object, such as "today's weather". The click behavior refers to which search result is clicked by the target object, such as clicking the first link about "today's weather". The dwell time refers to the length of time the target object stays on a certain search result page, which can reflect the degree of satisfaction of the target object with the result. The page scrolling refers to whether the target object scrolls the page to view more search results, as well as the speed and depth of scrolling. Re-search refers to returning to the search page, modifying the query word or adding more conditions for searching if the target object is not satisfied with the first search result. By analyzing the historical behavior of the target object, the search engine can learn the preferences and search habits of the target object, and thus provide more personalized search results in the future.

[0061] In the embodiments of the present disclosure, the first type of feedback data can indirectly reflect the satisfaction of the target object with the current answer, including various implicit features. Here, the implicit features include but are not limited to:

[0062]

[0063]

[0064] Table 1

[0065] In Table 1, the result page of a search behavior is loaded as a pv. For example, when the target object inputs a keyword in the search engine and clicks search, the search result page returned by the search engine is loaded and displayed to the target object, and this process can be regarded as a pv.

[0066] It can be understood that the above implicit features can be supplemented or deleted according to requirements.

[0067] In some embodiments, the second type of feedback data is survey data that the system actively collects from the target object. This type of data is usually the subjective feelings and opinions that the system actively collects from the target object through questionnaires, target object interviews, satisfaction ratings, etc. Among them, satisfaction survey: a satisfaction questionnaire pops up on the search results page or in the application, asking the target object about its satisfaction with the search results, page layout, advertising interference, etc. Target object interview: by inviting some target objects to participate in the interview, we can gain an in-depth understanding of their search habits, needs, and opinions on existing functions. Feedback portal: set up a feedback portal in the application (Application, APP) to encourage target objects to actively submit feedback on search experience, function suggestions, etc. The second type of feedback data provides the search engine with the direct voice and opinions of the target object, which helps to discover potential problems and improvement points. These data are of great significance for product iteration, function optimization and improving the target object experience.

[0068] In the disclosed embodiment, the second type of feedback data can directly reflect the target object's satisfaction with the current answer, including multiple explicit features.

[0069] Here, implicit features include but are not limited to:

[0070]

[0071]

[0072] Table 2

[0073] It is understandable that the above explicit features can be supplemented or deleted according to needs.

[0074] By combining historical behavioral data with survey data, we can comprehensively capture the target audience's implicit and explicit feedback on search results, reducing the potential bias introduced by a single data source and improving the comprehensiveness and accuracy of feedback data. With this richer feedback data, we can more accurately identify the key factors influencing search answer satisfaction, thereby optimizing the language model and effectively improving the match between generated answers and the target audience's preferences.

[0075] In some embodiments, the feedback data also includes: third-category feedback data and fourth-category feedback data; wherein, the third-category feedback data includes data generated during multiple rounds of interactive dialogues; the fourth-category feedback data includes data on the depth of multiple rounds of interactive dialogues; wherein, the third-category feedback data and the fourth-category feedback data are data generated by multiple rounds of generative question-answering.

[0076] In the disclosed embodiment, a multi-round interactive dialogue refers to a series of continuous question-and-answer processes between a target object and a search system or a dialogue assistant, and each round of dialogue is based on the context of the previous round.

[0077] In the embodiments of the present disclosure, the depth of the dialogue refers to the complexity and depth of the interaction between the target object and the system in the multi-round interactive dialogue, which is usually related to the round of dialogue, the richness of information, and the satisfaction of the target object's demand.

[0078] In the embodiments of the present disclosure, the third type of feedback data includes data records generated during the multi-round interactive dialogue process, which can reflect the demand and reaction of the target object.

[0079] In the embodiments of the present disclosure, the fourth type of feedback data is data for measuring the depth of the multi-round interactive dialogue, including indicators such as the round of dialogue, complexity, and satisfaction of the target object's demand.

[0080] In some embodiments, the third type of feedback data is collected, including: when the target object and the search system carry out a multi-round interactive dialogue, the system needs to record the content of each round of dialogue, the query input by the target object, the answer generated by the system, and the reaction of the target object (such as clicking, replying, dwell time, etc.). In-depth analysis of the process of multi-round dialogue identifies changes in the target object's demand, interest shift, and the influence of the system's answer on the target object's subsequent behavior at different dialogue stages.

[0081] In some embodiments, the fourth type of feedback data is collected, including: designing a method to measure the depth of the multi-round interactive dialogue, which can include the round of dialogue, the complexity of the query input by the target object, the amount of information in the answer generated by the system, and whether the target object actively asks more in-depth questions, etc. Statistical and analysis of the collected dialogue depth data helps understand the target object's acceptance and satisfaction of different depth dialogues, and how these depth dialogues affect the quality of search answers and the target object's final decision.

[0082] For example, the target object first inputs "What is artificial intelligence?" The system gives a brief definition, and the target object further asks "What are the applications of artificial intelligence in the medical field?" The system generates answers to multiple application scenarios, but the target object modifies the question to "How does artificial intelligence help diagnose diseases?" The system generates more specific descriptions of the diagnosis scenarios, and continues to follow up the target object's questions in the multi-round dialogue. The target object's dwell time, question modification, and continuous clicking on the answers in each round of dialogue are recorded by the system as the third type of feedback data, while the target object's interaction round number and reaction time in the multi-round dialogue are recorded as the fourth type of feedback data. These feedback data help the system understand the changes in the target object's demand and optimize the model to generate answers that better meet the target object's demand.

[0083] By collecting behavioral data and in-depth feedback from multiple rounds of interactions, we can dynamically capture the changing needs of the target audience in complex conversations, allowing us to more accurately adjust the answers generated by the model. The optimized model can adapt to multi-round question-and-answer scenarios, providing more coherent and insightful responses to the target audience, significantly improving their satisfaction throughout these multiple rounds of conversations.

[0084] In some embodiments, collecting the target object's feedback data on the search answer displayed first on the search page also includes: selecting a target answer generation algorithm that matches the current task requirements from a plurality of answer generation algorithms; for a given set of query terms, using the target answer generation algorithm to generate multiple versions of search answers for each query term; and displaying the multiple versions of the search answers to the actual terminal or simulated terminal in turn to collect the target object's feedback data on each version of the search answer.

[0085] In the embodiments of the present disclosure, multiple answer generation algorithms include but are not limited to random generation (Random Generation), diversity generation (Diverse Beam Search Generation) and constraint-guided generation (Constraint-Guided Generation).

[0086] In the embodiment of the present disclosure, random generation refers to increasing the randomness of the generated results by increasing the parameters of the SFT model.

[0087] In the disclosed embodiments, diversity generation aims to enhance the diversity of generated content. By balancing accuracy and diversity, it is suitable for use when seeking answers from multiple perspectives, improving the practicality and effectiveness of the model in responding to open-ended questions.

[0088] In the disclosed embodiments, constraint-guided generation involves adding soft constraints to the model's prompt template to guide the model in generating answers with a specific style or structure. For example, by fixing the query and reference, the system adds guidance in the style section, such as "Answer should be more structured" or "Answer should be concise and clear." This generation method maintains the model's answer logic while controlling the form and style of the answer, making it suitable for scenarios requiring a specific format or style.

[0089] In some embodiments, a target answer generation algorithm that matches the current task requirements is selected from a plurality of answer generation algorithms, including: when generating search answers, the system can use a plurality of different generation algorithms (such as random generation, diversity generation, constraint-guided generation, etc.). Different algorithms may have different focuses when generating answers. For example, some algorithms emphasize the diversity of generated content, while some algorithms focus on the accuracy and conciseness of the answers. Based on the specific requirements of the task, the system will select a most suitable algorithm to generate the answer. For example: suppose the target object enters the query "What is the highest mountain in the world?" The system can use an "accuracy first" algorithm to ensure that the returned answer is very accurate and concise - that is, "Mount Everest". Another "diversity first" algorithm can also be used to generate multiple different versions of answers, which may include relevant background information, such as the height and location of Mount Everest, comparison with other peaks, etc.

[0090] In the embodiment of the present disclosure, a given query word set refers to a combination of multiple related query words or questions that the target object may ask in a specific scenario. These query word sets can be a set of keywords pre-defined by the system, or they can be a set consisting of multiple queries input by the target object in a specific task or context. In a generative search or question-answering system, the system will generate multiple versions of answers based on these query word sets using an answer generation algorithm. For example, the target object may query a series of words related to "how to improve work efficiency", such as "efficiency improvement tools", "time management methods", "concentration improvement techniques", etc. The system will generate corresponding answers based on these query word sets to meet the needs of the target object from different angles.

[0091] In some embodiments, for a given set of query terms, a target answer generation algorithm is used to generate multiple versions of search answers for each query term, including: after the target object enters the query term, the system will generate multiple versions of answers for the query based on the selected target generation algorithm. Each version of the answer may answer the target object's question from a different angle. The purpose of doing this is to find out which version best meets the needs of the target object. For example: The target object enters the query "How to improve work efficiency?". The target generation algorithm can generate the following three versions of answers: Version 1: Explain the concept of work efficiency in detail and propose some general time management methods. Version 2: Concisely list several specific tools (such as calendar management applications) to improve work efficiency. Version 3: Combined with psychological factors, it is recommended how to maintain focus and motivation to improve efficiency.

[0092] In some embodiments, multiple versions of search answers are displayed in turn to actual terminals or simulated terminals to collect feedback data from the target object on each version of the search answer, including: after generating multiple versions of answers, the system will display these answers to the target object in turn, or use a simulated terminal (automated tool) for testing to collect the target object's feedback on each answer. Through the target object's click behavior, likes, sharing and other feedback signals, the system can determine which answer version best meets the needs of the target object. For example: the system may first display version 1 of the answer and monitor whether the target object clicks on it or continues to search for other information. If the target object does not interact or skips, the system can infer that version 1 does not quite meet the needs of the target object. Then display version 2 and version 3, and collect data on the target object's clicks, reading time, and further interactions (such as likes or comments) to determine which version is more popular.

[0093] Figure 5 A schematic diagram of the architecture for accumulating feedback data is shown in FIG. Figure 5 As shown, accumulating feedback data consists of two parts: an online system and offline training. The online system involves the large language model selecting a target generation algorithm from multiple answer generation algorithms, generating multiple versions of answers for a given query set, and displaying these multiple versions of search answers to the terminal in turn to collect feedback data from the target audience on each version of the search answer. Offline training involves data mining the collected feedback data to generate training samples containing preference features; optimizing the current large language model based on the training samples, and bringing the optimized large language model online to achieve continuous accumulation of feedback data.

[0094] For example, a target object inputs “how to improve sleep quality?” in the search box. The system first selects a “diversity generation algorithm” from multiple answer generation algorithms, because this question is relatively open, and the target object may be interested in different solutions. The system generates three different versions of answers: the first version introduces life habits to improve sleep, the second version recommends several application tools that help sleep, and the third version combines psychological suggestions on how to reduce anxiety to improve sleep quality. According to its algorithm, the system displays the first version, i.e., the answer about life habits. The target object browses this answer, but stays for a short time and has no further interaction behavior. Later, another target object proposes the same query, and the system displays the second version, i.e., the answer recommending application tools. This target object stays for a long time on this answer and likes the recommended application tools. Based on the feedback of these target objects, the system can judge that the second version of the answer is more in line with the needs of the target object. Through these implicit and explicit feedback data, the system records which answer version is more popular, and inputs these data into the model optimization process. The system adjusts the generation model through reinforcement learning and feedback mechanism, so that it is more inclined to generate tool recommendation type answers preferred by the target object in the future when encountering similar queries.

[0095] In this way, by generating different versions of answers for the same query word through multiple generation algorithms, and by taking turns to display and collect feedback data, the system can better understand the preferences and needs of the target object, and optimize the subsequent answer generation process.

[0096] In some embodiments, the feedback data is analyzed to identify key factors that affect the satisfaction of search answers, including: preprocessing the feedback data to obtain preprocessed data; extracting multiple features that affect satisfaction from the preprocessed data; using statistical algorithms to analyze the multiple features to identify key factors related to satisfaction.

[0097] In the embodiments of the present disclosure, the multiple features refer to various information points extracted from the preprocessed target object behavior data, which are independent of each other but are related to each other, and collectively describe the behavior patterns and preferences of the target object when using search answers. These features can include but are not limited to keywords of search queries, positions of search results clicked by target objects, browsing time, target object feedback (such as likes, shares, comments, etc.), historical search records of target objects, etc.

[0098] In the disclosed embodiments, key factors are features identified through statistical algorithm analysis from these multiple features that are significantly correlated with the target subject's satisfaction with the search answer. In other words, key factors are those features that can significantly influence or explain differences in the target subject's satisfaction with the search answer. These key factors may not be all features, but rather a subset, but they carry greater weight or influence in explaining differences in satisfaction.

[0099] In the disclosed embodiments, the multiple features provide a comprehensive description of the target user's behavior, while the key factors are a small number of features selected from these features that have a significant impact on satisfaction. These key factors are not only related to the characteristics of the search answer, but also reflect the preferences and needs of the target user.

[0100] In some embodiments, feedback data is preprocessed, including: the system first collects behavioral data logs of the target object in the search scenario. This data includes records of the target object's clicks, displays, jumps, and other behaviors. To ensure data quality, the preprocessing process cleans and filters this raw data to remove noise and outliers. Specifically, preprocessing removes unreasonable interaction data, such as abnormally short dwell times or repeated and invalid click records. The resulting data is ensured to be clean and analyzable, laying the foundation for subsequent feature extraction.

[0101] In some embodiments, extracting multiple features that affect satisfaction from the preprocessed data includes: in the cleaned data, the system extracts key features of the target object's behavior. These features include click-through rate, browsing time, click depth, skip behavior, dwell time, etc., which are used to represent the interaction pattern between the target object and the search answer. In order to further explore the target object's preferences, the system will also combine these features to generate pairwise samples and pointwise samples. Pairwise samples are used to compare the relative performance of different search answers in the target object feedback, and pointwise samples are used to evaluate the absolute satisfaction of a single answer.

[0102] In some embodiments, statistical algorithms are used to analyze multiple features, including: the system uses machine learning and statistical algorithms (such as descriptive statistical analysis, correlation analysis, and regression analysis) to model the extracted features. By analyzing a large amount of target object behavior data, the system can identify key factors that are strongly correlated with the target object's satisfaction. For example, the target object's click behavior and browsing time may be key features of positive feedback, while skipping behavior may be a feature of negative feedback. Through satisfaction modeling, the system can associate behavioral features with satisfaction, build a satisfaction model, and identify high-confidence satisfied and dissatisfied samples.

[0103] For example, the target object inputs the query "How to improve work efficiency?", the system first collects the target object's behavior data on the search page, including clicks, browsing time, skipping behavior, etc. Then, the system preprocesses these data, removes abnormal short dwell time and repeated click records, and ensures data quality. After data cleaning, the system extracts multiple key features of the target object's interaction, such as click rate, browsing time, skipping behavior, etc. The system generates pairwise samples based on these features to compare the relative performance of two different search results, and pointwise samples to evaluate the satisfaction of a single result. In the process of generating pairwise samples, the system compares the target object's clicks and dwell time on different answers. For example, the target object stays longer on an answer recommending an efficiency tool, and stays shorter on an answer about psychological methods. Through such comparison, the system can judge which answer better meets the target object's needs. At the same time, the system uses pointwise samples to evaluate the absolute satisfaction of each individual answer, such as the target object liking the answer recommending the tool, indicating that the answer has high satisfaction. Next, the system uses machine learning and statistical algorithms to model these extracted features to analyze which behavior features (such as click count, skipping behavior) have strong correlation with the target object's satisfaction. Through modeling, the system identifies that browsing time and click rate are key factors affecting satisfaction, while skipping behavior is negative feedback. Finally, the system optimizes the search model based on these analysis results, making it more inclined to generate high-satisfaction answers.

[0104] In this way, by screening and verifying the output samples, and by preprocessing and feature extraction of feedback data, the quality of the analyzed data and the effectiveness of the features can be ensured, thereby improving the accuracy of the analysis results. Using statistical algorithms to analyze multiple features can systematically identify key factors that affect the satisfaction of search answers, ensuring that the selected samples have high confidence and representativeness, providing accurate data support for subsequent preference alignment training.

[0105] In some embodiments, in combination with the key factors, mining the first sample set and the second sample set from the feedback data further includes: inputting the feedback data into a pre-trained satisfaction model; wherein the satisfaction model is used to predict the target object's satisfaction with the search answer in combination with the key factors; obtaining the first sample set and the second sample set output by the satisfaction model.

[0106] In the embodiments of the present disclosure, the satisfaction model is a prediction model based on machine learning or statistical analysis, used to evaluate or predict the target object's satisfaction with the search answer. It outputs a predicted score of the target object's satisfaction by inputting the target object's behavior data (such as click rate, browsing time, skipping behavior, etc.) in combination with pre-defined key factors.

[0107] In some embodiments, the feedback data is input into a pre-trained satisfaction model, including: collecting and pre-processing the feedback data of the target object (such as clicks, browsing time, skipping behavior, etc.). These data will be passed as input to the pre-trained satisfaction model. The model has been trained through large-scale target object behavior data and preference alignment training, and can combine multiple key factors (such as click-through rate, dwell time, skipping behavior, etc.) to predict the target object's satisfaction with each search answer. By inputting the feedback data into the satisfaction model, the system can use these key factors to evaluate the satisfaction of each search result. For example, a higher click-through rate and a longer browsing time are usually identified by the model as high satisfaction signals, while frequent skipping behaviors are identified as low satisfaction signals. By combining these key factors, the model can output a satisfaction prediction score for each search answer.

[0108] In some embodiments, obtaining the first and second sample sets output by the satisfaction model includes: after completing the satisfaction prediction, the system divides the feedback data into two sample sets based on the scores output by the model. The first sample set includes samples with predicted satisfaction scores above a first threshold, representing that the target subject is satisfied with the search results; the second sample set includes samples with predicted satisfaction scores below a second threshold, representing that the target subject is dissatisfied with the search results. Through this classification, the system can extract typical samples with high and low satisfaction scores, providing a basis for subsequent model optimization.

[0109] For example, a user enters the query "How can I improve work efficiency?" and the system generates multiple answers, including recommended productivity tools, time management techniques, and advice on improving focus. The user clicks on the answer for the recommended tools, stays on it for 5 minutes, and gives it a thumbs-up. However, the user stays on the answer for the time management techniques for 30 seconds before skipping and clicking on another result. The system collects this user behavior data, preprocesses it, and feeds it into a pre-trained satisfaction model. The model uses key factors such as click-through rate, dwell time, and skip behavior to make predictions, determining that the answer for the recommended tools receives high satisfaction (9 / 10) and the answer for the time management techniques receives low satisfaction (3 / 10). The system then divides this data into two sample sets based on the ratings: the answer for the recommended tools enters the first sample set (high satisfaction), and the answer for the time management techniques enters the second sample set (low satisfaction). Using this data, the system further optimizes the generated model to improve the matching of search results and the user experience.

[0110] In this way, by using the pre-trained satisfaction model, the first sample set and the second sample set can be automatically mined from a large amount of feedback data, greatly improving the efficiency and accuracy of data processing. In combination with the key factors for prediction, the satisfaction of the target object to the search answer can be more accurately identified, providing strong support for subsequent optimization work. Based on the mined sample set, the model is optimized in a targeted manner, which can improve the quality of the search answer and the satisfaction of the target object.

[0111] In some embodiments, based on the first sample set and / or the second sample set, the first large language model is optimized to obtain a second large language model, including: selecting a target alignment algorithm matching the current task from a plurality of alignment algorithms; based on the target alignment algorithm, screening training samples conforming to the target alignment algorithm from the first sample data set and the second sample data set; using the target alignment algorithm as a training guide, and using the training samples to optimize the first large language model.

[0112] In the embodiments of the present disclosure, each alignment algorithm can be used to guide the first large language model to align the preference feature. The plurality of alignment algorithms include but are not limited to: a reward model (RM), a direct preference optimization (DPO) algorithm, a key-term optimization (KTO) algorithm, and a proximal policy optimization (PPO) algorithm.

[0113] In the embodiments of the present disclosure, the DPO uses pairwise comparison data (i.e., which output is considered better by the target object) to train the model, so that the output of the model is more in line with human preferences. It usually uses a statistical model to model the comparison relationship and optimizes the victory rate of the model output in the pairwise comparison.

[0114] In the embodiments of the present disclosure, the PPO maximizes the cumulative reward by optimizing the strategy while limiting the difference between the new strategy and the old strategy to ensure the stability of the training.

[0115] In the embodiments of the present disclosure, the KTO describes the decision bias of humans facing uncertain results (such as loss aversion and overestimation of small probability events) by introducing a value function and a weight function. The loss function of KTO combines the nonlinear characteristics of the value function and the probability perception bias of the weight function to better align the model output with human preferences.

[0116] In this disclosed embodiment, a RM model is trained based on the collected feedback data to predict the reward value for any given text response. Using a reinforcement learning algorithm, the large language model is treated as an intelligent agent and fine-tuned through interaction with the RM model to optimize its text generation.

[0117] In some implementations, the most appropriate target alignment algorithm is selected from a variety of alignment algorithms based on the task requirements. For example, PPO might be selected to ensure training stability, while DPO might be selected to directly optimize target object preference. For applications in specific fields, KTO can prioritize key terms, thereby improving the quality of specialized text. If the task goal is to optimize overall generation quality, a RM model can be used to guide model training through feedback scoring.

[0118] In some embodiments, the system selects training samples that meet the target alignment algorithm from the first sample data set and the second sample data set, including: after pre-selecting the target alignment algorithm, the system selects samples suitable for training from the first sample set (high satisfaction samples) and the second sample set (low satisfaction samples). For example, the system can select samples with high click-through rates and long dwell times as positive feedback (first sample set), and select samples with high skip rates and short dwell times as negative feedback (second sample set). By analyzing the characteristics of these samples (such as target object behavior, number of clicks, browsing time, etc.), the system constructs a representative training data set. These data will serve as the basis for model optimization and help to accurately align the preferences of the target object.

[0119] In some embodiments, optimizing the first large language model using training samples includes: after selecting training samples and defining optimization targets, the system uses a target alignment algorithm (such as PPO or DPO) to optimize the first large language model.

[0120] Taking PPO as an example, the system gradually updates the model's generation mechanism by comparing the old model strategy with the new strategy, ensuring that the generated answers better meet the preferences of the target subject. For example, when the model receives negative feedback samples, it will reduce the generation of answers that do not meet the target subject's needs; when processing positive feedback samples, the system will increase the probability of generating answers that are similarly highly satisfactory. By continuously adjusting the model parameters, the system can generate more accurate answers that meet the target subject's expectations. When using the RM model, the system will score the generated answers based on the target subject's feedback, and use these scores as reward signals to guide model training and optimization. The PPO algorithm ensures the stability of model training by controlling the update amplitude, avoiding excessive parameter changes during the training process, and ensuring the consistency and rationality of the generated results.

[0121] By selecting a target alignment algorithm that matches the task and optimizing the model based on selected training samples, we can significantly improve the model's performance on specific tasks and align search answers with preferred features. By selecting training samples that match the target alignment algorithm, we can reduce unnecessary computational overhead and improve model training efficiency.

[0122] In some embodiments, based on the target alignment algorithm, training samples that meet the target alignment algorithm are screened out from the first sample data set and the second sample data set, including: when the target alignment algorithm is DPO, a first answer to the target question is screened out from the first sample data set, and a second answer to the target question is screened out from the second sample data set; and a training sample is generated based on the first answer and the second answer corresponding to the target question.

[0123] In some embodiments, when the target alignment algorithm is DPO, the system optimizes the model parameters directly based on the target object feedback data without building a complex reward model. Traverse the first sample data set, and for each target question, filter out the answer that the target object is most satisfied with or has the highest preference as the first answer. Similarly, traverse the second sample data set, and for the same target question, filter out the answer that the target object is less satisfied with or has obvious shortcomings as the second answer. These answers may cause dissatisfaction to the target object due to insufficient relevance, low accuracy or unclear expression. Pair the first answer and the second answer corresponding to each target question to form a training sample pair. Each sample pair contains a positive example (first answer) and a negative example (second answer) preferred by the target object. Label each training sample pair with a corresponding label. Usually, a binary label (such as 1 for positive preference and 0 for negative preference) can be used to distinguish the first answer from the second answer. Arrange the training sample pairs and their labels in the format required for model training to ensure that they can be directly used in the model training process.

[0124] Figure 6 A schematic diagram of large language model optimization is shown, as Figure 6 As shown in the figure, the SFT sample is used to fine-tune the general large language model to generate the first large language model; after the first large language model is put into operation, it receives questions input by the target object, obtains raw data from multiple information sources, and generates search answers; displays the search answers on the search page; collects feedback data for the search answers and stores the feedback data; generates DPO samples based on the mined preference features; and performs reinforcement learning on the first large language model based on the DPO samples to generate the second large language model.

[0125] By directly selecting the first and second answers based on the target subject's preference feedback, the generated training samples can more accurately reflect the target subject's true needs and preferences, thereby improving the relevance and effectiveness of model training. Compared to traditional random or uniform sampling methods, the DPO algorithm can more quickly locate the most valuable training samples for model optimization, thereby improving model training efficiency. Because training samples are generated directly based on the target subject's preference feedback, the optimized model can better meet the target subject's expectations and needs when handling similar problems, improving generation quality.

[0126] The present disclosure provides a search processing method. Figure 7 1 is a flow chart of a search processing method according to an embodiment of the present disclosure. The search processing method can be applied to electronic devices, including but not limited to fixed devices and / or mobile devices. In some possible implementations, the search processing method can also be implemented by a processor calling computer-readable instructions stored in a memory. Figure 7 As shown, the search processing method includes:

[0127] S701: receiving a question input by a target user on a search page;

[0128] S702: Generate a search answer that matches the question using a pre-trained large language model;

[0129] S703: Display the search answer at the top of the search page;

[0130] S704: Monitor and record the target object's feedback data regarding the search answer; wherein the feedback data is used to optimize the large language model.

[0131] In the disclosed embodiment, the search answer refers to an answer or information generated for a question that is closely related to the question. For example, the search answer may be a short text answer or a detailed answer containing multiple paragraphs, pictures or links.

[0132] In some embodiments, receiving a question input by a target object on a search page includes: the target object inputs a query question through an input box on the search page, the system uses front-end technology to capture the input of the target object, and sends a request to a back-end server.

[0133] In some implementations, a pre-trained large language model is used to generate search answers that match the question. This involves the large language model understanding the question and retrieving relevant information from its knowledge base to generate one or more matching answers. This leverages the powerful natural language processing capabilities of the large language model to quickly generate accurate and relevant search answers.

[0134] In some embodiments, the search answer is displayed at the top of the search page, including: the backend server returns the generated search answer to the front end, and the front end uses the answer to render to the top position of the search page. At the same time, a prominent style can be designed to attract the attention of the target object. In this way, directly displaying the search answer at the top of the search page can reduce the search cost and time of the target object and improve search efficiency.

[0135] In some embodiments, feedback data of the target object for the search answer is monitored and recorded, including: the system collects feedback data of the target object for the search answer through front-end tracking technologies (such as event listening, page dwell time statistics, etc.) and possible explicit feedback (such as satisfaction evaluation, like / dislike buttons, etc.). These data are sent to the backend server for storage and analysis. Subsequently, the parameters or training data of the large language model are adjusted according to the feedback data to optimize its performance. In this way, through the closed-loop feedback mechanism, the large language model is continuously iterated and optimized to make it more consistent with the expectations and needs of the target object, improving the accuracy and relevance of the search answer.

[0136] For example, assuming that the target object inputs “today's weather” on the search page, the search system inputs the question “today's weather” into the large language model, generates an answer using the large language model, such as “today's weather is sunny, temperature 20-25℃, suitable for outdoor activities.”, and displays the answer at the top of the search page. After the target object sees the answer, the target object clicks the “satisfied” button, and the search system records this feedback, which can be used to optimize the model in the future.

[0137] The technical solutions of the embodiments of the present disclosure use a pre-trained large language model, and the search system can quickly understand the question input by the target object and generate an accurate and relevant search answer. This instant feedback mechanism reduces the waiting time of the target object and improves search efficiency. At the same time, the search answer is directly displayed at the top of the search page, so that the target object can quickly obtain the required information, further improving the experience of the target object. The system monitors and records the feedback data of the target object for the search answer, forming a closed-loop optimization mechanism. These feedback data provide valuable basis for model optimization, so that the model can continuously learn and adapt to the preferences and needs of the target object. Through continuous training and adjustment, the performance of the large language model will be continuously improved, thereby providing more high-quality and personalized search services for the target object.

[0138] In some embodiments, a pre-trained large language model is used to generate a search answer that matches the question, including: collecting raw data related to the question from at least one information source; determining preference features corresponding to the question; inputting the raw data and preference features into the large language model, and obtaining a search answer generated by the large language model that matches the question and conforms to the preference features.

[0139] In the disclosed embodiment, preference features refer to the personalized needs of the target object for search answers during the search process, such as language style (such as formal, humorous), answer length (such as introduction, detailed analysis), information type (such as text, picture, video), source reliability, etc.

[0140] In some embodiments, raw data related to the question is collected from at least one information source. This includes: Based on the question posed by the target subject, the system collects raw data related to the question from multiple information sources (such as databases, online documents, web pages, etc.). This data may include, but is not limited to, text, images, videos, and other formats. During the collection process, the search system considers the timeliness, accuracy, and relevance of the data to ensure that the collected data maximizes the accuracy of the data source.

[0141] In some embodiments, determining the preference characteristics corresponding to the question includes: the search system determining the target subject's preference characteristics for search answers based on information such as the target subject's historical behavior, search habits, and personal settings. The preference characteristics are represented in an appropriate form (e.g., labels, weights) for subsequent input into the large language model.

[0142] In some embodiments, inputting raw data and preference characteristics into a large language model to obtain a search answer generated by the large language model that matches the question and conforms to the preference characteristics includes: inputting the collected raw data and the determined preference characteristics into a pre-trained large language model. The large language model utilizes its powerful language understanding and generation capabilities to process and analyze the raw data, while taking into account the preference characteristics of the target subject, to generate a search answer that matches the question and conforms to the target subject's preferences. In the process of generating answers, the large language model considers various factors, such as language fluency, logical rationality, and information accuracy, to ensure the quality of the generated answers.

[0143] For example, suppose the target user asks "How do I make pizza at home?" The search system first collects raw data related to pizza making from multiple sources, such as recipes, step-by-step instructions, and images. The search system then determines the target user's preferred characteristics based on their historical behavior (e.g., frequent visits to food-related websites) and preferences (e.g., preference for detailed step-by-step instructions). Finally, this raw data and preference characteristics are fed into a large language model, which generates a detailed pizza-making guide as the search answer, including the required ingredients, steps, precautions, and more, in a language style that matches the target user's preferences.

[0144] This automated data collection and model generation process significantly reduces manual intervention and waiting time, improves search efficiency, and enhances the quality of search results. By factoring in the preferences of the target audience, the generated answers better meet their personalized needs, increasing their satisfaction.

[0145] In some embodiments, determining the preference feature corresponding to the question includes: obtaining scenario information corresponding to the question; and selecting a preference feature that matches the scenario information from a plurality of candidate preference features corresponding to the question as the preference feature of the question.

[0146] In the disclosed embodiment, the scenario information refers to the specific scenario in which the question occurs, which helps to more accurately understand the intention of the target object.

[0147] For example, scenario information may include but is not limited to the following aspects: Time: The time when the question is asked, such as weekdays, weekends, holidays, etc., which may affect the preferences of the target object. Location: The place where the question occurs, such as home, office, public places, etc. Different places may have different preference requirements. Social environment: The social environment in which the target object asks the question, such as alone, with friends, in a meeting, etc., which will affect the target object's expected answer style and level of detail. Context: The conversation before the question or the target object's previous operational behavior, which helps to understand the target object's current needs and expectations.

[0148] In some embodiments, obtaining contextual information corresponding to a question includes: receiving a question input by a target subject, parsing the question using natural language processing techniques, and extracting key information such as keywords and semantic roles. Further analysis of the contextual environment of the question, including the target subject's historical query history, current time, and geographic location, is performed to construct comprehensive contextual information.

[0149] In the disclosed embodiment, preference features serve as an important basis for generating answers, guiding the large language model to generate answers that are more in line with the expectations of the target object and the needs of the scenario.

[0150] For example, candidate preference features can include: language style: such as formal, informal, humorous, strict, etc., choose appropriate language style according to scene and target object habit. Information amount: whether the target object expects a concise or detailed answer depends on the specific needs of the target object and the scene requirements. Professionalism: for some professional problems, the target object may prefer more professional and accurate answers. Emotional tendency: according to the mood of the target object and the atmosphere of the scene, choose positive, neutral or slightly emotional answer mode.

[0151] In some embodiments, the search system pre-constructs a candidate preference feature library, including: according to business needs and problem types, a set of preference features are pre-defined, including but not limited to language style, information type, etc. Map the keywords and semantic information in the question to the pre-defined preference features to generate multiple possible candidate preference features.

[0152] In some embodiments, from the multiple candidate preference features corresponding to the question, select the preference feature that matches the scene information as the preference feature of the question, including: using pre-set rules or algorithms to match the scene information with the candidate preference features. These rules can be based on statistical models, machine learning models or expert knowledge bases. For each matched preference feature, calculate the corresponding weight value according to its importance and relevance in the current scene. According to the high and low of the weight value, select the preference feature that best matches the scene information as the final result. The selected preference feature is output in a structured or easily understood manner for subsequent large language model processing.

[0153] In some embodiments, according to different scenes (such as work, study, entertainment, etc.) that the target object is in, the large language model can automatically adjust the content and style of the answer to adapt to the needs in different scenes. For example, in the work scene, the model can generate more professional and accurate answers; while in the entertainment scene, it can generate more relaxed and interesting answers.

[0154] In some embodiments, when generating answers, the large language model can take the summarized human preferences as part of the input, and by adjusting the parameters and calculation process inside the model, the generated answers can better meet the preferences of the target object. For example, for target objects who prefer concise and clear answers, the model can generate more refined answers; while for target objects who prefer detailed explanations, the model can generate answers containing more background information and explanations. According to the language style and preferences of the target object, the large language model can adjust the language style of the answer. For example, for target objects who use formal language style, the model can generate more rigorous and professional answers; while for target objects who use relaxed and humorous language style, the model can generate more friendly and interesting answers.

[0155] In some implementations, the large language model can generate richer responses by combining multimodal information, such as images, audio, and video, in addition to textual responses. Based on the preferences of the target audience and the needs of the scenario, the large language model can select appropriate multimodal information to present the answer, providing a more intuitive and engaging response experience.

[0156] For example, suppose the target asks: "Any restaurant recommendations for a family dinner tonight?" The question is parsed, and the keywords "tonight", "family dinner", and "restaurant recommendation" are extracted, while considering the target's historical query records and current geographic location and other scene information. Candidate preference features such as restaurant type (such as Chinese food, Western food), price range, environment (such as quiet, lively), and dish taste are determined. Based on the target's location and current time (tonight), as well as the need for a family dinner, the system may match preference features such as "Chinese food", "moderately priced", and "quiet environment". Recommend several Chinese restaurants that meet these preference features to the target.

[0157] Figure 8 The architecture diagram of the search system for optimizing search answers is shown in FIG. Figure 8 As shown, the system consists of an offline learning system and an online system. After generating an answer generation model based on the base model, the answer generation model is brought online. The online system's answer generation model generates multiple or a single answer version based on the target subject's input, collects feedback data from the target subject, and stores the feedback data on disk to a log repository. The offline learning system extracts feedback data from the logs and uses a satisfaction model to determine preference characteristics. The answer generation model is optimized based on these preference characteristics, aligning the search answers output by the answer generation model with these preference characteristics. The model can continuously adjust and optimize its internal parameters and algorithms to better match the target subject's preferences.

[0158] By combining contextual information and preference characteristics, we can more accurately understand the target audience's intentions, thereby improving the accuracy and efficiency of problem resolution. We can automatically adjust service content and methods based on the preferences of different target audiences, providing personalized services or suggestions and enhancing the user experience.

[0159] The present disclosure provides a device for processing generative search data, such as Figure 9As shown, the device may include: a collection module 901, which is used to collect feedback data of the target object on the search answer displayed first on the search page, wherein the search answer is an answer generated by the search system using the first language model; an analysis module 902, which is used to analyze the feedback data to identify key factors affecting the satisfaction of the search answer; a sample mining module 903, which is used to mine a first sample set and a second sample set from the feedback data in combination with the key factors, wherein the first sample set includes samples with a satisfaction level greater than a first threshold, and the second sample set includes samples with a satisfaction level less than a second threshold; an optimization and adjustment module 904, which is used to optimize the first language model based on the first sample set and / or the second sample set to obtain a second language model, wherein the search answer generated by the second language model has a better match with the target object's preference than the first language model; a deployment module 905, which is used to deploy the second language model into the search system to replace the first language model.

[0160] In some embodiments, the feedback data includes: first-category feedback data and second-category feedback data; wherein, the first-category feedback data includes historical behavior data during the target object search process; the second-category feedback data includes survey data that the system actively collects from the target object; wherein, the first-category feedback data and the second-category feedback data are data generated by a single round of generative question and answer.

[0161] In some embodiments, the feedback data also includes: third-category feedback data and fourth-category feedback data; wherein, the third-category feedback data includes data generated during multiple rounds of interactive dialogues; the fourth-category feedback data includes data on the depth of multiple rounds of interactive dialogues; wherein, the third-category feedback data and the fourth-category feedback data are data generated by multiple rounds of generative question-answering.

[0162] In some embodiments, the collection module 901 includes: a first selection submodule, used to select a target answer generation algorithm that matches the current task requirements from a plurality of answer generation algorithms; a first generation submodule, used to generate multiple versions of search answers for each query word for a given set of query words using the target answer generation algorithm; a first collection submodule, used to display multiple versions of search answers to actual terminals or simulated terminals in turn to collect feedback data of the target object on each version of the search answer.

[0163] In some embodiments, the analysis module 902 includes: a preprocessing submodule for preprocessing feedback data to obtain preprocessed data; an extraction submodule for extracting multiple features that affect satisfaction from the preprocessed data; and an analysis submodule for analyzing multiple features using statistical algorithms to identify key factors related to satisfaction.

[0164] In some embodiments, the sample mining module 903 includes: an input submodule for inputting feedback data into a pre-trained satisfaction model; wherein the satisfaction model is used to predict the target object's satisfaction with the search answer in combination with key factors; and an acquisition submodule for acquiring a first sample set and a second sample set output by the satisfaction model.

[0165] In some embodiments, the optimization and adjustment module 904 includes: a second selection submodule, used to select a target alignment algorithm that matches the current task from a plurality of alignment algorithms; a second generation submodule, used to screen out training samples that meet the target alignment algorithm from the first sample data set and the second sample data set based on the target alignment algorithm; and a training submodule, used to use the target alignment algorithm as a training guide and optimize the first language model using the training samples.

[0166] In some embodiments, the second generation submodule is used to: when the target alignment algorithm is the direct preference optimization (DPO) algorithm, filter out the first answer to the target question from the first sample data set, and filter out the second answer to the target question from the second sample data set; generate training samples based on the first answer and the second answer corresponding to the target question.

[0167] For the description of specific functions and examples of each module and submodule of the device in the embodiment of the present disclosure, please refer to the relevant description of the corresponding steps in the above method embodiment, which will not be repeated here.

[0168] The generative search data processing device of the disclosed embodiment can realize the self-optimization cycle of the generative large language model by collecting multivariate feedback data and automatically mining high-confidence samples, thereby improving the quality of the search answer displayed first and improving the efficiency of the target object in obtaining the answer.

[0169] The embodiment of the present disclosure provides a search processing device, such as Figure 10 As shown, the device may include: a receiving module 1001, used to receive the question entered by the target object on the search page; a generating module 1002, used to use a pre-trained large language model to generate a search answer that matches the question; a display module 1003, used to display the search answer at the top of the search page; a monitoring and recording module 1004, used to monitor and record the feedback data of the target object regarding the search answer; wherein the feedback data is used to optimize the large language model.

[0170] In some embodiments, the generating module 1002 includes: a second collecting submodule configured to collect original data related to the question from at least one information source; a determining submodule configured to determine a preference feature corresponding to the question; and a third generating submodule configured to input the original data and the preference feature into a large language model to obtain a search answer generated by the large language model and matching the question and conforming to the preference feature.

[0171] In some embodiments, the determining submodule is configured to: obtain scene information corresponding to the question; and select, from a plurality of candidate preference features corresponding to the question, a preference feature matching the scene information as the preference feature of the question.

[0172] The specific functions and examples of the modules and submodules of the apparatuses in the embodiments of the present disclosure are described above in the corresponding steps of the method embodiments, which will not be described here.

[0173] The search processing apparatus in the embodiments of the present disclosure can improve the quality of the search answer displayed first and improve the efficiency of the target object obtaining the answer.

[0174] The embodiments of the present disclosure provide a scene schematic diagram of a search processing method, as shown in Figure 11 .

[0175] As described above, the search processing method provided by the embodiments of the present disclosure is applied to an electronic device. The electronic device is intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital assistants, cellular telephones, smart phones, wearable devices, and other similar computing devices.

[0176] Specifically, the electronic device can specifically perform the following operations: receiving a question input by a target object on a search page; generating a search answer matching the question by using a pre-trained large language model; displaying the search answer at the first position of the search page; monitoring and recording feedback data of the target object for the search answer; and wherein the feedback data is used to optimize the large language model.

[0177] It should be understood that Figure 11 The scene diagram shown is merely illustrative and not limiting, and those skilled in the art can make various obvious changes and / or replacements based on the examples Figure 11 , and the resulting technical solutions still belong to the disclosure range of the embodiments of the present disclosure.

[0178] In the technical solutions of the present disclosure, the acquisition, storage and application of user personal information involved comply with relevant laws and regulations and do not violate public order and good customs.

[0179] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0180] Figure 12 A schematic block diagram of an example electronic device 1200 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are provided as examples only and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0181] like Figure 12 As shown, device 1200 includes a computing unit 1201, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 1202 or a computer program loaded from a storage unit 1208 into a random access memory (RAM) 1203. Various programs and data required for the operation of device 1200 can also be stored in RAM 1203. Computing unit 1201, ROM 1202, and RAM 1203 are connected to each other via a bus 1204. An input / output (I / O) interface 1205 is also connected to bus 1204.

[0182] Various components in device 1200 are connected to I / O interface 1205, including an input unit 1206, such as a keyboard and mouse; an output unit 1207, such as various types of displays and speakers; a storage unit 1208, such as a magnetic disk and optical disk; and a communication unit 1209, such as a network card, a modem, a wireless communication transceiver, etc. Communication unit 1209 allows device 1200 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.

[0183] The computing unit 1201 can be various general and / or special purpose processing components with processing and computing capabilities. Some examples of the computing unit 1201 include, but are not limited to, a CPU, a Graphics Processing Unit (GPU), various special-purpose Artificial Intelligence (AI) computing chips, various computing units running machine learning model algorithms, a Digital Signal Processor (DSP), and any appropriate processor, controller, microcontroller, etc. The computing unit 1201 performs various methods and processes described above, such as the processing method of generative search data and / or the search processing method. For example, in some embodiments, the processing method of generative search data and / or the search processing method can be implemented as a computer software program tangibly embodied in a machine-readable medium, such as the storage unit 1208. In some embodiments, part or all of the computer program can be loaded and / or installed onto the device 1200 via the ROM 1202 and / or the communication unit 1209. When the computer program is loaded onto the RAM 1203 and executed by the computing unit 1201, one or more steps of the processing method of generative search data and / or the search processing method described above can be performed. Alternatively, in other embodiments, the computing unit 1201 can be configured to perform the processing method of generative search data and / or the search processing method by other any appropriate means, such as by means of firmware.

[0184] Various embodiments of the systems and techniques described herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), system on chips (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include being implemented in one or more computer programs that are executable and / or interpreted on a programmable system that includes at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0185] The program code for implementing the methods of the present disclosure may be written in any combination of one or more programming languages. The program code may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable generative search data processing device, such that when the program code is executed by the processor or controller, the functions / operations specified in the flowcharts and / or block diagrams are implemented. The program code may be executed entirely on the machine, partially on the machine, as a stand-alone software package, partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0186] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. More specific examples of machine-readable storage media may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory, a read-only memory, an erasable programmable read-only memory (EPROM), a flash memory, an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0187] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a cathode ray tube (CRT) or a liquid crystal display (LCD) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0188] The systems and techniques described herein can be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network (LAN), a wide area network (WAN), and the Internet.

[0189] A computer system may include a client and a server. The client and server are generally remote from each other and typically interact through a communication network. The client-server relationship arises through computer programs running on the respective computers and having a client-server relationship with each other. The server may be a cloud server, a server in a distributed system, or a server integrated with a blockchain.

[0190] It should be understood that the various forms of the processes shown above can be used to reorder, add, or delete steps. For example, the steps described in this disclosure can be performed in parallel, sequentially, or in a different order, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved. This is not limited herein.

[0191] The above specific embodiments do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art will appreciate that various modifications, combinations, sub-combinations, and substitutions may be made based on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the principles of this disclosure shall be included within the scope of protection of this disclosure.

Claims

1. A method for processing generative search data, comprising: Collecting feedback data from the target subject regarding the search answer displayed first on the search page, wherein the search answer is the answer generated by the search system using the first language model; wherein the feedback data includes: fourth category feedback data; the fourth category feedback data is data generated by multiple rounds of generative question-answering, and the fourth category feedback data includes data on the depth of the multiple rounds of interactive dialogue; the fourth category feedback data specifically includes: the number of dialogue rounds, the complexity of the query input by the target subject, the amount of information in the answer generated by the system, and whether the target subject proactively raises more in-depth questions; Analyzing the feedback data to identify key factors that affect satisfaction with the search answers; In combination with the key factors, mining a first sample set and a second sample set from the feedback data, wherein the first sample set includes samples with a satisfaction level greater than a first threshold, and the second sample set includes samples with a satisfaction level less than a second threshold; Based on the first sample set and / or the second sample set, optimizing the first large language model to obtain a second large language model, wherein the search answer generated by the second large language model has a better match with the target object preference than the first large language model; The second largest language model is deployed into the search system to replace the first largest language model.

2. The method according to claim 1, wherein The feedback data also includes: first-category feedback data and second-category feedback data; wherein, the first-category feedback data includes historical behavior data during the target object search process; the second-category feedback data includes survey data actively collected by the system from the target object; wherein, the first-category feedback data and the second-category feedback data are data generated by a single round of generative question and answering.

3. The method according to claim 1, wherein The feedback data also includes: third-category feedback data; wherein, the third-category feedback data includes data generated during multiple rounds of interactive dialogue; wherein, the third-category feedback data is data generated during multiple rounds of generative question and answer.

4. The method according to any one of claims 1 to 3, wherein: Collect feedback from the target audience on the search answer displayed first on the search page, including: Select a target answer generation algorithm that matches the current task requirements from a variety of answer generation algorithms; For a given set of query terms, the target answer generation algorithm is used to generate multiple versions of search answers for each query term; The multiple versions of the search answers are displayed to the actual terminal or the simulated terminal in turn to collect the target object's feedback data on each version of the search answer.

5. The method according to claim 1, wherein Analyzing the feedback data to identify key factors affecting satisfaction with the search answers includes: Preprocessing the feedback data to obtain preprocessed data; extracting a plurality of features affecting satisfaction from the preprocessed data; Statistical algorithms are used to analyze the multiple characteristics and identify key factors related to satisfaction.

6. The method according to claim 1, wherein The step of mining the first sample set and the second sample set from the feedback data in combination with the key factors includes: Inputting the feedback data into a pre-trained satisfaction model; wherein the satisfaction model is used to predict the target object's satisfaction with the search answer in combination with the key factors; A first sample set and a second sample set output by the satisfaction model are obtained.

7. The method according to claim 1, wherein The optimizing the first large language model based on the first sample set and / or the second sample set to obtain a second large language model includes: Select a target alignment algorithm that matches the current task from a variety of alignment algorithms; Based on the target alignment algorithm, screening out training samples that meet the target alignment algorithm from the first sample data set and the second sample data set; The target alignment algorithm is used as a training guide, and the first language model is optimized using the training samples.

8. The method according to claim 7, wherein: The step of selecting training samples that meet the target alignment algorithm from the first sample data set and the second sample data set based on the target alignment algorithm includes: In a case where the target alignment algorithm is a Direct Preference Optimization (DPO) algorithm, a first answer to a target question is selected from the first sample data set, and a second answer to the target question is selected from the second sample data set; The training sample is generated based on the first answer and the second answer corresponding to the target question.

9. A search processing method, comprising: Receive questions entered by the target audience on the search page; Use a pre-trained large language model to generate search answers that match the question; Display the search answer at the first position on the search page; Monitor and record feedback data of the target object regarding the search answer; wherein, the feedback data is used to optimize the large language model, wherein the optimization of the large language model is performed using the method for optimizing the first large language model according to any one of claims 1-8.

10. The method according to claim 9, wherein: The method of using a pre-trained large language model to generate a search answer that matches the question includes: Collecting raw data related to the problem from at least one information source; Determining the preference characteristics corresponding to the question; The original data and the preference features are input into the large language model, and a search answer generated by the large language model that matches the question and conforms to the preference features is obtained.

11. The method according to claim 10, wherein: Determining the preference feature corresponding to the question includes: Obtaining scenario information corresponding to the problem; A preference feature that matches the scenario information is selected from a plurality of candidate preference features corresponding to the question as the preference feature of the question.

12. A device for processing generative search data, comprising: A collection module is configured to collect feedback data from a target subject regarding a search answer displayed first on a search page, wherein the search answer is an answer generated by the search system using the first language model; wherein the feedback data includes: fourth-category feedback data; the fourth-category feedback data is data generated by multiple rounds of generative question-answering, and the fourth-category feedback data includes data on the depth of multiple rounds of interactive dialogue; the fourth-category feedback data specifically includes: the number of dialogue rounds, the complexity of the query input by the target subject, the amount of information in the answer generated by the system, and whether the target subject proactively raises more in-depth questions; An analysis module, configured to analyze the feedback data to identify key factors affecting satisfaction with the search answers; a sample mining module, configured to mine a first sample set and a second sample set from the feedback data in combination with the key factors, wherein the first sample set includes samples whose satisfaction level is greater than a first threshold, and the second sample set includes samples whose satisfaction level is less than a second threshold; an optimization and adjustment module, configured to optimize the first large language model based on the first sample set and / or the second sample set to obtain a second large language model, wherein the search answer generated by the second large language model has a better match with the target object preference than the first large language model; A deployment module is used to deploy the second largest language model into the search system to replace the first largest language model.

13. The device according to claim 12, wherein The feedback data includes: first-category feedback data and second-category feedback data; wherein, the first-category feedback data includes historical behavior data during the target object search process; the second-category feedback data includes survey data actively collected by the system from the target object; wherein, the first-category feedback data and the second-category feedback data are data generated by a single round of generative question and answering.

14. The device according to claim 12, wherein The feedback data also includes: third-category feedback data; wherein, the third-category feedback data includes data generated during multiple rounds of interactive dialogue; wherein, the third-category feedback data is data generated during multiple rounds of generative question and answer.

15. The device according to any one of claims 12 to 14, wherein The collection module includes: The first selection submodule is used to select a target answer generation algorithm that matches the current task requirements from multiple answer generation algorithms; A first generation submodule is configured to generate multiple versions of search answers for each query word using the target answer generation algorithm for a given query word set; The first collection submodule is configured to display the multiple versions of the search answers to the actual terminal or the simulated terminal in turn, so as to collect the target object's feedback data on each version of the search answer.

16. The device according to claim 12, wherein The analysis module includes: A preprocessing submodule, configured to preprocess the feedback data to obtain preprocessed data; An extraction submodule, configured to extract a plurality of features affecting satisfaction from the preprocessed data; The analysis submodule is used to analyze the multiple features using statistical algorithms to identify key factors related to satisfaction.

17. The device according to claim 12, wherein The sample mining module includes: An input submodule, configured to input the feedback data into a pre-trained satisfaction model; wherein the satisfaction model is configured to predict the target object's satisfaction with the search answer in combination with the key factors; The acquisition submodule is used to obtain the first sample set and the second sample set output by the satisfaction model.

18. The device according to claim 12, wherein The optimization and adjustment module includes: The second selection submodule is used to select a target alignment algorithm that matches the current task from multiple alignment algorithms; A second generating submodule is configured to screen out training samples that conform to the target alignment algorithm from the first sample data set and the second sample data set based on the target alignment algorithm; A training submodule is configured to use the target alignment algorithm as a training guide and optimize the first language model using the training samples.

19. The device according to claim 18, wherein The second generation submodule is used to: In a case where the target alignment algorithm is a Direct Preference Optimization (DPO) algorithm, a first answer to a target question is selected from the first sample data set, and a second answer to the target question is selected from the second sample data set; The training sample is generated based on the first answer and the second answer corresponding to the target question.

20. A search processing device, comprising: A receiving module, used to receive questions entered by the target object on the search page; A generation module, configured to generate a search answer matching the question using a pre-trained large language model; A display module, configured to display the search answer at the top of the search page; A monitoring and recording module is used to monitor and record the feedback data of the target object regarding the search answer; wherein, the feedback data is used to optimize the large language model, wherein the optimization of the large language model is performed using the method for optimizing the first large language model according to any one of claims 1-8.

21. The device according to claim 20, wherein The generation module includes: A second collecting submodule is configured to collect raw data related to the problem from at least one information source; A determination submodule, configured to determine the preference characteristics corresponding to the question; The third generation submodule is used to input the original data and the preference characteristics into the large language model, and obtain a search answer generated by the large language model that matches the question and conforms to the preference characteristics.

22. The device according to claim 21, wherein The determining submodule is used to: Obtaining scenario information corresponding to the problem; A preference feature that matches the scenario information is selected from a plurality of candidate preference features corresponding to the question as the preference feature of the question.

23. An electronic device comprising: at least one processor; as well as a memory communicatively connected to at least one processor; wherein, The memory stores instructions that can be executed by at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1 to 11.

24. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are for causing a computer to perform a method according to any one of claims 1-11.

25. A computer program product comprising a computer program stored on a storage medium, the computer program implementing the method according to any one of claims 1 to 11 when executed by a processor.

Images