Data processing method, and computing device, server, storage medium and program product

Abstract

Provided in the present disclosure are a data processing method, and a computing device, a server, a storage medium and a program product. The method relates to the field of artificial intelligence. When a search for information required for generating a reply is performed on the basis of query information, a human-computer interaction model and a process reward model are combined to dynamically plan a next search process of a search path, so as to generate a plurality of candidate search paths, and a high-quality candidate search path is selected from among the plurality of candidate search paths as a new search path, such that the quality of the search path obtained by means of planning can be improved. After a plurality of iterations, a high-quality target search path can be obtained, and on the basis of the high-quality target search path, target information required for replying to the query information can be accurately found, such that a target search result has a higher degree of matching with a user intent, thereby better solving user problems, improving the quality of search results, and further improving the quality of replies generated by the human-computer interaction model.

Description

Data processing methods, computing devices, servers, storage media and software products

[0001] This disclosure claims priority to Chinese Patent Application No. 2025104902281, filed with the China Patent Office on April 16, 2025, entitled “Data Processing Method, Computing Device, Server, Storage Medium and Program Product”, the entire contents of which are incorporated herein by reference. Technical Field

[0002] This disclosure relates to artificial intelligence technology, and more particularly to a data processing method, computing device, server, storage medium, and program product. Background Technology

[0003] With the rapid development of artificial intelligence technology, large language models (LLMs) and other artificial intelligence (AI) models are increasingly being used in the search field. In intelligent human-computer interaction scenarios, large models often need to search for knowledge information related to user questions and generate responses based on the search results.

[0004] Traditional search solutions mainly rely on keyword matching and static ranking algorithms, which cannot fully understand the user's deep needs. This results in search results that do not match the user's intent well, are difficult to solve the user's problems, and have poor search result quality, which in turn leads to low quality of generated responses. Summary of the Invention

[0005] This disclosure provides a data processing method, computing device, server, storage medium, and program product to solve the problems of low matching degree between search results and user intent, difficulty in solving user problems, poor search result quality, and consequently low quality of generated responses in traditional search solutions.

[0006] In a first aspect, this disclosure provides a data processing method, including:

[0007] In response to receiving query information, the following dynamic programming process is performed iteratively based on the query information:

[0008] Through the human-computer interaction model, based on the query information and the search results of the current search path, the next search process of the current search path is dynamically planned to obtain multiple alternative search paths;

[0009] The quality information of each of the proposed alternative search paths is evaluated using a process reward model.

[0010] If the quality information of each of the candidate search paths does not meet the preset conditions, then at least one of the candidate search paths is selected as a new search path based on the quality information of each candidate search path.

[0011] Until a target search path that meets the preset conditions is obtained, the target search result of the target search path is obtained, and the search result of the target search path is used to generate the response information of the query information.

[0012] Secondly, this disclosure provides a data processing method, including:

[0013] Construct a training set, which includes query samples and search path samples corresponding to the query samples, wherein each search path sample includes at least one search process, the search results of each search process, and the quality information of the search path corresponding to each search process;

[0014] For any search process in any of the search path samples, a prediction result of the search process is generated by the human-computer interaction model based on the query sample and the search results of the sub-search paths in the search path sample before the search process, and a quality prediction result of the search path corresponding to the search process is generated by the process reward model.

[0015] Based on the prediction results of the search process, the quality information of the search path corresponding to the search process, and the quality prediction results, the parameters of the human-computer interaction model and the parameters of the process reward model are adjusted to obtain the trained human-computer interaction model and process reward model.

[0016] Thirdly, this disclosure provides a data processing method, including:

[0017] The receiving end-side device sends a search enhancement request, which includes query information;

[0018] Based on the query information, the following dynamic programming process is performed iteratively:

[0019] Through the human-computer interaction model, based on the query information and the search results of the current search path, the next search process of the current search path is dynamically planned to obtain multiple alternative search paths;

[0020] The quality information of each of the proposed alternative search paths is evaluated using a process reward model.

[0021] If the quality information of each of the candidate search paths does not meet the preset conditions, then at least one of the candidate search paths is selected as a new search path based on the quality information of each candidate search path.

[0022] Until a target search path that meets the preset conditions is obtained, the target search result of the target search path is obtained;

[0023] The target search result is returned to the terminal device.

[0024] Fourthly, this disclosure provides a data processing method, including:

[0025] Receive query information sent by the client device;

[0026] Based on the query information, the following dynamic programming process is performed iteratively:

[0027] Through the human-computer interaction model, based on the query information and the search results of the current search path, the next search process of the current search path is dynamically planned to obtain multiple alternative search paths;

[0028] The quality information of each of the proposed alternative search paths is evaluated using a process reward model.

[0029] If the quality information of each of the candidate search paths does not meet the preset conditions, then at least one of the candidate search paths is selected as a new search path based on the quality information of each candidate search path.

[0030] Until a target search path that meets the preset conditions is obtained, the target search result of the target search path is obtained;

[0031] The human-computer interaction model generates a response to the query based on the target search results.

[0032] The response information is returned to the client device.

[0033] Fifthly, this disclosure provides a computing device, including: a memory and a processor;

[0034] The memory is used to store computer programs / instructions, and the processor is used to execute the computer programs / instructions, wherein the computer programs / instructions are executed by the processor as provided in any of the foregoing aspects.

[0035] In a sixth aspect, this disclosure provides a server, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to cause the server to perform the methods provided in any of the preceding aspects.

[0036] In a seventh aspect, this disclosure provides a computer-readable storage medium storing computer-executable instructions that, when executed by a processor, implement the method provided in any of the preceding aspects.

[0037] Eighthly, this disclosure provides a computer program product including a computer program that, when executed by a processor, implements the methods provided in any of the preceding aspects.

[0038] The data processing method, computing device, server, storage medium, and program product disclosed herein describe a method that, when searching for and generating information required for a response based on query information, dynamically plans the next search step by combining a human-computer interaction model and a process reward model. This generates multiple alternative search paths and selects high-quality alternative paths as new search paths, thereby improving the quality of the planned search paths. After multiple iterations, a high-quality target search path is obtained. Based on this high-quality target search path, the target information required for responding to the query information can be accurately searched, resulting in a higher degree of matching between the target search results and the user's intent. This better solves the user's problem, improves the quality of search results, and consequently enhances the quality of the response generated by the human-computer interaction model. Attached Figure Description

[0039] The accompanying drawings, which are incorporated in and form a part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure.

[0040] Figure 1 is a schematic diagram of the architecture of a human-computer interaction system provided in this disclosure;

[0041] Figure 2 is a schematic diagram of the architecture of another human-computer interaction system provided in this disclosure;

[0042] Figure 3 is a flowchart of a data processing method provided in an exemplary embodiment of this disclosure;

[0043] Figure 4 is an example diagram of a dynamic programming search path provided by an exemplary embodiment of this disclosure;

[0044] Figure 5 is a schematic diagram of the model architecture provided in an exemplary embodiment of this disclosure;

[0045] Figure 6 is a flowchart of a model training method provided in an exemplary embodiment of this disclosure;

[0046] Figure 7 is an example diagram of the tree structure of a search path sample provided in an exemplary embodiment of this disclosure;

[0047] Figure 8 is a flowchart of a data processing method provided in another exemplary embodiment of this disclosure;

[0048] Figure 9 is a flowchart of a data processing method provided in another exemplary embodiment of this disclosure;

[0049] Figure 10 is a structural block diagram of a computing device according to an embodiment of the present disclosure;

[0050] Figure 11 is a schematic diagram of the structure of a server provided in an embodiment of this disclosure.

[0051] The accompanying drawings have illustrated specific embodiments of this disclosure, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concepts of this disclosure to those skilled in the art through reference to particular embodiments. Detailed Implementation

[0052] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numerals in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this disclosure. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this disclosure as detailed in the appended claims.

[0053] It should be noted that the user information (including but not limited to user device information, user attribute information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this disclosure are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use and processing of the relevant data must comply with relevant laws, regulations and standards, and corresponding operation entry points are provided for users to choose to authorize or refuse.

[0054] First, let's explain the terms used in this disclosure:

[0055] Large language models: Deep learning models with a large number of parameters and trained on large-scale data, capable of understanding and generating human language.

[0056] Reward Model: An evaluation model used to assess the quality of AI system output.

[0057] Process Reward Model: A special reward model that not only focuses on the output results but also evaluates the entire AI decision-making process.

[0058] Intelligent agent: An AI system that can perceive the environment, make autonomous decisions, and take actions to accomplish a specific goal.

[0059] Step-level Beam Search (SLBS) is a variant of the beam search algorithm, primarily used for sequence generation tasks.

[0060] Large-scale models refer to deep learning models with a massive number of parameters, typically containing hundreds of millions, tens of billions, or even trillions of parameters. Large-scale models are also known as foundation models (FM). They are pre-trained on large-scale unlabeled corpora, producing pre-trained models with hundreds of millions of parameters. These models can adapt to a wide range of downstream tasks and have good generalization ability. Examples include large-scale language models and multi-modal pre-training models.

[0061] In practical applications, large models only require a small number of samples to fine-tune the pre-trained model before they can be applied to different tasks. Large models can be widely used in fields such as Natural Language Processing (NLP) and Computer Vision. Specifically, they can be applied to computer vision tasks such as Visual Question Answering (VQA), Image Captioning (IC), and Image Generation, as well as NLP tasks such as text-based sentiment classification, text summarization, and machine translation. The main application scenarios for large models include digital assistants, intelligent robots, search, online education, office software, e-commerce, and intelligent design.

[0062] To address the problem that traditional search methods often result in poor matching between search results and user intent, difficulty in resolving user problems, and low-quality search results during human-computer interaction, leading to low-quality generated responses, this disclosure provides a data processing method. When searching for relevant knowledge information based on user-input query information, the method iteratively performs dynamic programming processing S1-S3 based on the query information:

[0063] S1. Through a human-computer interaction model, based on the query information and the search results of the current search path, the next search process of the current search path is dynamically planned, resulting in multiple alternative search paths. The search path records the search process based on the query information; specifically, it can sequentially record the search statements used in each search process, with each search statement representing a search process. The search statement can be a question in natural language. The search results for any search path include the search results of each search process (corresponding to the search statement) within that search path.

[0064] S2. Evaluate the quality information of each alternative search path using a process reward model. The quality information of an alternative search path represents the probability / expectation that the human-computer interaction model can correctly respond to the query information (i.e., generate a correct response to the query information) based on that alternative search path. The higher the quality information of an alternative search path, the higher the probability of generating a correct response based on that alternative search path, and the higher the quality of the alternative search path.

[0065] S3. If the quality information of each candidate search path does not meet the preset conditions, then at least one candidate search path is selected as the new search path based on the quality information of each candidate search path. The preset conditions can be that the quality information of the candidate search path is greater than or equal to a quality threshold (e.g., 1), and the specific condition can be set according to the actual application scenario; no specific limitation is made here. When the quality information of the candidate search path is greater than or equal to the quality threshold, it means that the search results based on the candidate search path can correctly answer the query information.

[0066] After multiple iterations, when a target search path (which can be any alternative search path that meets the preset conditions) is obtained, the search result of the target search path is taken as the target search result. The target search result contains the information needed to correctly respond to the query. The human-computer interaction model can then generate a response to the query based on the target search result.

[0067] Through the dynamic programming process described above, combined with the human-computer interaction model and the process reward model, the next search step of the search path is dynamically planned, generating multiple alternative search paths. High-quality alternative search paths are then selected as the new search path, improving the quality of the planned search paths. After multiple iterations, a high-quality target search path is obtained. Based on this high-quality target search path, the target information needed to respond to the query can be accurately retrieved. The target search results have a higher degree of matching with the user's intent, better solving the user's problem and improving the quality of the search results. This, in turn, improves the quality of the responses generated by the human-computer interaction model.

[0068] Figure 1 is a schematic diagram of the architecture of a human-computer interaction system provided in this disclosure. As shown in Figure 1, one architecture of the human-computer interaction system includes a terminal device and a server. The terminal device and the server have a communication link that allows them to communicate with each other, enabling data communication.

[0069] The server can be a server-side device in the human-computer interaction system. A server is a computing device deployed in the cloud or locally, such as a cloud cluster. The server stores trained human-computer interaction and process reward models. It is responsible for dynamically planning a search path based on the user's input query information, obtaining a high-quality target search path, and generating response information based on the target search results. The target search results include the target information required to respond to the query.

[0070] End-side devices can be electronic devices running a human-computer interaction system client. Specifically, they can be hardware devices with network communication, computing, and information display functions, including but not limited to smartphones, tablets, desktop computers, local servers, and cloud servers.

[0071] During human-computer interaction, users input query information using their devices, which then send the query to the server. The server, using a human-computer interaction model and a process reward model, dynamically plans a target search path based on the input query and retrieves the search results for that path, resulting in target search results that include the necessary information to respond to the query. The human-computer interaction model then generates a response based on the target search results. The server returns the response to the device, which then outputs the response to the user.

[0072] It should be noted that in the system architecture shown in Figure 1, the server does not necessarily need to store the human-computer interaction model and the process reward model. The human-computer interaction model and the process reward model can be deployed on other cloud servers, providing the server with application programming interfaces (APIs) for the human-computer interaction model and the process reward model. The server calls the human-computer interaction model through the API of the human-computer interaction model, and calls the process reward model through the API of the process reward model.

[0073] Figure 2 is a schematic diagram of another human-computer interaction system architecture provided in this disclosure. As shown in Figure 2, the alternative architecture of the human-computer interaction system includes a server, a model service, and client and server devices for the human-computer interaction system. The server device and the server have a communicable communication link, enabling data communication between them. The model service provides the server with APIs for the human-computer interaction model and the process reward model, and also provides the server device with an API for the human-computer interaction model. The server can call the human-computer interaction model and the process reward model through the APIs. The server device can call the human-computer interaction model through the APIs.

[0074] A server can be a device that provides data search capabilities (such as retrieval enhancement) to the server-side devices of a human-computer interaction system. Specifically, it can be a computing device deployed in the cloud or locally, such as a cloud cluster. The server is responsible for dynamically planning a high-quality target search path based on the user's query information, using human-computer interaction models and process reward models, and obtaining the target search results. The target search results include the target information needed to respond to the query.

[0075] During human-computer interaction (HCI) communication, users input query information (i.e., queries) into the HCI system using a client device. The client device sends the query information to the server device. The server then sends the query information to the server. The server, using the HCI model and process reward model, dynamically plans the target search path based on the input query information and obtains the target search result. The server returns the target search result to the HCI system's server device. The server device, using the HCI model, generates a response based on the target search result. The server device returns the response to the client device. The client device then outputs the response to the user.

[0076] It should be noted that the solution disclosed herein can be applied to various human-computer interaction systems, specifically intelligent dialogue systems, intelligent assistants, digital assistants, intelligent robots, etc., without any specific limitations here.

[0077] The technical solutions of this disclosure and how they solve the aforementioned technical problems will be described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of this disclosure will now be described with reference to the accompanying drawings.

[0078] Figure 3 is a flowchart of a data processing method provided in an exemplary embodiment of this disclosure. The execution entity in this embodiment is a server in any of the aforementioned system architectures. As shown in Figure 3, the specific steps of the method are as follows:

[0079] Step S301: Receive query information.

[0080] The query information refers to the user query input into the human-computer interaction system. It can be a natural language question input by the user, or text converted from the user's speech. The query information input by the user may differ in different human-computer interaction scenarios, and this embodiment does not impose specific limitations on it.

[0081] After receiving the query information, the server performs one or more searches based on the query information to obtain the target search results, which include the information needed to respond to the query.

[0082] In this embodiment, by iteratively executing steps S302-S305, and combining the human-computer interaction model and the process reward model, the next search process of the current search path is dynamically planned, a new search path is generated, and high-quality new search paths are selected; this continues until a target search path that meets preset conditions is obtained. Based on the target search path, the target search result can be obtained.

[0083] The search path records the search process based on the query information. Specifically, it records the search statements used in each search process sequentially, with each search statement representing a search process. Search statements can be natural language questions. The search results for any search path include the search results for each search process (corresponding to the search statement) within that path.

[0084] Step S302: Through the human-computer interaction model, based on the query information and the search results of the current search path, dynamically plan the next search process of the current search path to obtain multiple alternative search paths.

[0085] The human-computer interaction model refers to the model used in the human-computer interaction system to generate response information for query information. It can be any language model used for human-computer interaction, such as the Large Language Model (LLM). This embodiment does not make any specific limitation here.

[0086] In the first iteration for the query information, the current search path is empty, meaning it does not contain any search process (or search statement). The server inputs the query information into the human-computer interaction model, which then plans the first search process (i.e., the search statement used in the first search) based on the query information, generating multiple alternative search paths. Each alternative search path includes the search statement used in the first search process. An alternative search path can be represented as: SP1 = {SQ1}, where SQ1 represents the search statement used in the first search process in SP1. Optionally, SP1 can also be represented as SP1 = {SQ0, SQ1}, where SQ0 represents the query information, using the query information as the starting point (or root node) of the search path, and SP1 still includes a search process SQ1.

[0087] In subsequent iterations, the current search path is the new search path determined in the previous iteration. There can be one or more current search paths. In this step, for each current search path, the next search process is dynamically planned based on the query information and the search results of the current search path using a human-computer interaction model, resulting in multiple alternative search paths for this current search path. Similarly, alternative search paths for each current search path can be obtained.

[0088] In this step, the server inputs the query information and the search results of the current search path into the human-computer interaction model for reasoning. Based on the query information and the search results of the current search path, it generates multiple alternative searches for the next search process of the current search path (i.e., alternative search processes, or search statements used by alternative search processes). Adding each alternative search for the next search process to the end of the current search path yields an alternative search path.

[0089] For example, assuming the current search path is SP1 = {SQ1}, and after the second iteration, the search statements used in the next search process of SP1 are determined to be SQ2 and SQ3, then the following two alternative search paths for SP1 can be determined: SP1_1 = {SQ1, SQ2}, SP1_2 = {SQ1, SQ3}.

[0090] For example, a first planning prompt can be pre-configured. This prompt guides the human-computer interaction model to analyze the data items needed to generate the response to the query (i.e., the missing data items in the response) based on the given input data, and outputs the search process for obtaining the corresponding data items (i.e., the search statement used). In the first iteration, the input data includes the query information. In subsequent iterations, the input data includes the query information and the search results for the current search path.

[0091] The search results for the current search path are optional input. In the first iteration for the query information, i.e., the first search process of dynamic programming, since the current search path is empty, it is not necessary to input the search results for the current search path into the human-computer interaction model; the input data includes the query information. In subsequent iterations, the input data includes both the query information and the search results for the current search path.

[0092] Optionally, the server can generate (i.e., sample) multiple alternative searches for the next search process by calling the human-computer interaction model multiple times. Each time the human-computer interaction model is called, the first planning prompt and input data are input into the model for reasoning, generating an alternative search for the next search process (i.e., an alternative search statement to be used in the next search). By calling the human-computer interaction model multiple times for the above sampling, multiple alternative searches for the next search can be obtained.

[0093] Optionally, the number of candidate searches generated for the next search process can be configured in the first planning prompt (this number can be multiple). The server inputs the first planning prompt containing the number of generated searches (greater than 1) and the input data into the human-computer interaction model for inference. After one inference, multiple (i.e., the number of generated searches) candidate searches are generated for the next search, which can improve search efficiency. The number of generated searches configured in the first planning prompt can be set to a fixed value, or it can be flexibly set and embedded into the first planning prompt by relevant technical personnel or users. There is no specific limitation here.

[0094] In one optional embodiment, in this step, while dynamically planning the next search process using a human-computer interaction model, information about the thought process for the next search process is generated. By generating this thought process information, the understanding ability of the human-computer interaction model can be improved, thereby enhancing the quality of the dynamically planned next search process and ultimately improving the quality of the search path.

[0095] Specifically, through a human-computer interaction model, based on the query information and the search results of the current search path, multiple alternative searches for the next search step of the current search path are generated, along with the thought process information corresponding to each alternative search. Adding each alternative search for the next search step to the end of the current search path yields multiple alternative search paths.

[0096] Optionally, the thought process information of each search process in the current search path can also be used as data basis for the next search process in dynamic planning, which can improve the quality of the next search process in dynamic planning, thereby improving the quality of the search path.

[0097] Specifically, the query information, the search results of the current search path, and the thought process information corresponding to each search step within the current search path are input into the human-computer interaction model. Based on these elements, the model generates multiple alternative searches for the next search step within the current search path, along with the thought process information for each alternative search. These alternative searches for the next search step are then added to the end of the current search path, resulting in multiple alternative search paths.

[0098] For example, a second planning prompt can be pre-configured. The second planning prompt is used to prompt the human-computer interaction model to analyze the data items that need to be obtained next to generate the response to the query information based on the given input data (i.e., the missing data items in the response), and output the search process for obtaining the corresponding data items (i.e., alternative searches) and the thinking process information corresponding to the search process.

[0099] Among them, the thinking process information is the logical reasoning process of the human-computer interaction model when generating alternative searches for the next search process. It is the logical reasoning process generated by the human-computer interaction model simulating the thinking process information of humans when solving problems or making decisions.

[0100] The search results for the current search path and the thought process information corresponding to each search step within the current search path are optional inputs. In the first iteration targeting the query information (i.e., the first search step of dynamic programming), since the current search path is empty, it is not necessary to input the search results for the current search path and the thought process information corresponding to each search step within the current search path into the human-computer interaction model; the input data includes the query information. In subsequent iterations, the input data includes the query information, the search results for the current search path, and the thought process information corresponding to each search step within the current search path.

[0101] The server inputs the second planning prompt and input data into the human-computer interaction model for reasoning, generating multiple alternative searches for the next search process, as well as the thought process information corresponding to each alternative search.

[0102] It should be noted that when generating multiple alternative searches and corresponding thought process information for the next search process based on the second planning prompt, this can be achieved by calling the human-computer interaction model multiple times, with each call generating one alternative search and corresponding thought process information for the next search process; or, the number of searches to be generated (greater than 1) can be configured in the second planning prompt, and multiple alternative searches and corresponding thought process information for the next search process can be generated by calling the human-computer interaction model once.

[0103] Furthermore, by adding each alternative search in the next search process to the end of the current search path, an alternative search path can be obtained.

[0104] In one optional embodiment, the server can perform data cleaning processes such as deduplication and filtering on multiple candidate searches generated through the human-computer interaction model for the next search process to improve the quality of the candidate search paths. Deduplication removes duplicate candidate searches (keeping only one of the duplicate candidate searches), avoiding the generation of duplicate candidate search paths, preventing waste of computing resources, and improving search efficiency. Filtering out candidate searches whose length is outside the preset range removes excessively long or short candidate searches, further improving the quality of the candidate searches.

[0105] Step S303: Evaluate the quality information of each alternative search path using a process reward model.

[0106] After obtaining multiple alternative search paths, the quality of each alternative search path is evaluated using a process reward model, generating quality information for each alternative search path.

[0107] The quality information of the alternative search path represents the probability / expectation that the human-computer interaction model can correctly respond to the query information (i.e., generate a correct response to the query information) based on the alternative search path. The higher the quality information of the alternative search path, the higher the probability of generating a correct response based on the alternative search path, and the higher the quality of the alternative search path.

[0108] In this step, the server can input the query information and the search results of each alternative search path into the process reward model. The process reward model then predicts the probability of generating a correct response to the query information based on the search results of each alternative search path, thereby obtaining the quality information of each alternative search path.

[0109] In one optional embodiment, the server can input the query information, the search results of each alternative search path, and the thought process information corresponding to each search process in the alternative search path into the process reward model; through the process reward model, based on the query information, the search results of each alternative search path, and the thought process information corresponding to each search process in the alternative search path, the probability of generating a correct response to the query information based on the search results of each alternative search path is predicted, thereby obtaining the quality information of each alternative search path.

[0110] Since the information on the thought process has a significant impact on the prediction of the correct response, providing the information on the thought process corresponding to each search process in the alternative search paths to the process reward model as the basis for evaluation can improve the accuracy of the quality information of each alternative search path.

[0111] Step S304: Determine whether there are alternative search paths that meet the preset conditions.

[0112] In this step, the server determines whether there are any alternative search paths that meet the preset conditions based on the quality information of each alternative search path.

[0113] In this embodiment, the quality information of the search path can take values ​​within the range [0,1]. The preset condition can be that the quality information of the candidate search path is greater than or equal to a quality threshold, or that the quality information of the candidate search path is equal to 1. The quality threshold can be set according to the actual application scenario; for example, it can be 0.99 or 1, etc., and is not specifically limited here. When the quality information of the candidate search path is greater than or equal to the quality threshold, it means that the search results based on the candidate search path have a high probability of correctly answering the query information. When the quality information of the candidate search path is equal to 1, it means that the search results based on the candidate search path can correctly answer the query information.

[0114] If no alternative search path meets the preset conditions, proceed to step S305 to select a high-quality alternative search path as a new search path, and then proceed to step S302 to perform the next round of dynamic programming processing on the new search path.

[0115] If there are alternative search paths that meet the preset conditions, then proceed to step S306 and stop the iteration.

[0116] The solution in this embodiment automatically stops iterating when a search path that meets the preset conditions is obtained. Complex queries require multiple iterations, while simple queries require fewer iterations. The search level (i.e., the length of the search path) can be automatically adjusted according to the complexity of the query information. It can handle both simple and complex queries and balances the relationship between search level and search efficiency, thus optimizing response time while ensuring search quality.

[0117] Step S305: If the quality information of each candidate search path does not meet the preset conditions, then at least one candidate search path is selected as a new search path based on the quality information of each candidate search path.

[0118] If the quality information of each alternative search path does not meet the preset conditions, one or more high-quality alternative search paths are selected as new search paths based on the quality information of each alternative search path.

[0119] For example, the candidate search path with the highest quality information can be selected as the new search path based on the quality information of each candidate search path.

[0120] In one optional embodiment, a step-level beam search (SLBS) method is employed. After dynamically planning the next search process and obtaining multiple candidate search paths, the candidate search paths are sorted based on their quality information to obtain a ranking result. Based on the ranking result, a predetermined number of candidate search paths with higher quality information are selected as new search paths.

[0121] The preset quantity is the preset beam size, which can be set and adjusted according to the needs of the actual application scenario. For example, the preset quantity can be 2 or 3, etc., and there is no specific limitation here.

[0122] By employing bundle search, multiple high-quality alternative search paths are selected based on a preset bundle width during the dynamic planning process of each search step within the search path. This achieves quality evaluation and selection of alternative searches for a single search step, hence the term "process-level bundle search." Using process-level bundle search improves the quality of each search step within the search path, thereby enhancing the quality of the target search path, which in turn improves the quality of the target search results and the quality of the response information generated based on those results.

[0123] In this embodiment, during each iteration of steps S302-S305, when dynamically planning the next search process of the current search path, multiple candidate search paths are obtained by multiple sampling using a human-computer interaction model; then, a process reward model is used to evaluate the quality of each candidate search path, and high-quality candidate search paths are selected as new search paths based on the quality evaluation results. This can improve the quality of the target search path obtained by dynamic planning, thereby improving the search quality of the target search results and the quality of the response information generated based on the target search results.

[0124] Step S306: Until a target search path that meets the preset conditions is obtained, the target search result of the target search path is obtained, and the target search result is used to generate the response information of the query information.

[0125] In this step, if there is one alternative search path that meets the preset conditions, the server will use this alternative search path as the target search path.

[0126] If there are multiple alternative search paths that meet the preset conditions, the server can use any of the alternative search paths that meet the preset conditions as the target search path; or select the shorter search path from the alternative search paths that meet the preset conditions as the target search path.

[0127] For example, Figure 4 is an example diagram of a dynamic programming search path provided by an exemplary embodiment of this disclosure. As shown in Figure 4, for the query information SQ_0: "What is the market capitalization of the world's highest-valued technology company in 2024?", in the first iteration, three alternative searches are obtained through dynamic programming using a human-computer interaction model: SQ_1: "Which company has the highest market capitalization in 2024?"; SQ_2: "Which company has the highest market capitalization this year?"; SQ_3: "Which technology companies have a market capitalization exceeding $1 trillion in 2024?". The following three alternative search paths can be obtained: SP_1 = {SQ_0, SQ_1}, SP_2 = {SQ_0, SQ_2}, and SP_3 = {SQ_0, SQ_3}. SQ_0 serves as the root node of the search path.

[0128] The quality of the alternative search paths corresponding to the above three alternative searches was evaluated using a process reward model. The quality information of the above three alternative search paths SP_1, SP_2 and SP_3 were 0.3, 0.2 and 0.8 respectively.

[0129] Taking a preset beam size of 2 as an example, two candidate search paths SP_1 and SP_3 with higher quality (quality information of 0.3 and 0.8 respectively) are selected from the three candidate search paths in the first search process and used as new search paths.

[0130] In the second iteration, the current search paths include SP_1 and SP_3. For SP_1, two alternative searches for the next search process are obtained through dynamic programming using a human-computer interaction model: SQ_4: "What is the market capitalization of Company A (an oil company)?"; SQ_5: "What is the market capitalization of Company B (an electronics company) in RMB?". Adding SQ_4 and SQ_5 to the end of SP_1 respectively, we get the following two alternative search paths: SP_1_1 = {SQ_0, SQ_1, SQ4}, SP_1_2 = {SQ_0, SQ_1, SQ5}. The quality of SP_1_1 and SP_1_2 is evaluated using a process reward model, and the quality information for alternative search paths SP_1_1 and SP_1_2 is 0.3 and 0.2 respectively.

[0131] For SP_2, dynamic programming using a human-computer interaction model yields two alternative searches for the next search process: SQ_6: "What is the market capitalization of Company C in US dollars?"; SQ_7: "What is the annual revenue of Company C in US dollars?". Adding SQ_6 and SQ_7 to the end of SP_2 respectively, we obtain the following two alternative search paths: SP_2_1 = {SQ_0, SQ_2, SQ6}, SP_2_2 = {SQ_0, SQ_2, SQ7}. Using a process reward model to evaluate the quality of SP_2_1 and SP_2_2, the quality information for alternative search paths SP_2_1 and SP_2_2 is 1.0 and 0.4 respectively.

[0132] Assume the preset condition is that the quality information is equal to 1. Currently, an alternative search path SP_2_1 that meets the preset condition has been obtained. Using alternative search path SP_2_1 as the target search path, the target search result SR is obtained: "3.68 trillion US dollars". Based on the target search result, a response to the query can be generated.

[0133] It should be noted that the target search results include the search results of each search process (including SQ_2 and SQ6) in the target search path SP_2_1. Not all search results are shown in the figure.

[0134] In one example scenario, after obtaining the target search result, the server can output the target search result. For example, it can return the target search result to the endpoint that sent the query. Furthermore, the endpoint can use a human-computer interaction model to generate a response based on the target search result and output the response to the user.

[0135] In another example scenario, after obtaining the target search result, the server can use a human-computer interaction model to generate and output a response based on the search result. For example, the server returns the response to the client device of the human-computer interaction system, enabling the client device to display the response to the user.

[0136] The solution in this embodiment, through the aforementioned dynamic programming process, combined with a human-computer interaction model and a process reward model, dynamically plans the next search process of the search path, generates multiple alternative search paths, and selects high-quality alternative search paths as new search paths, thereby improving the quality of the planned search paths. After multiple iterations, a high-quality target search path can be obtained. Based on the high-quality target search path, the target information required to respond to the query can be accurately searched, the target search results have a higher degree of matching with the user's intent, can better solve the user's problem, improve the quality of search results, and thus improve the quality of the responses generated by the human-computer interaction model.

[0137] In this embodiment, during the dynamic planning process of each search step, a human-computer interaction model dynamically plans each search process and selects high-quality search paths based on quality evaluation results, thus avoiding search instability. During the dynamic planning of each search step, a process reward model evaluates the quality information of each candidate search path and selects high-quality paths. This allows for real-time quality assessment and correction of the search process, ensuring that each search step moves towards a better direction. This enables dynamic adjustment of the search strategy, achieving efficient and high-quality searches and significantly improving the accuracy and reliability of search results.

[0138] In one optional embodiment, the human-computer interaction model includes a first backbone network and a prediction generation module. The process reward model includes a second backbone network and a quality prediction module. The first and second backbone networks can employ the same model structure and share parameters. The first and second backbone networks can be implemented as the backbone network (excluding the output layer or task head) of any pre-trained large language model; this embodiment does not impose specific limitations.

[0139] For example, Figure 5 is a schematic diagram of the model architecture provided in an exemplary embodiment of this disclosure. As shown in Figure 5, the human-computer interaction model and the process reward model can share the same backbone network. Based on the model architecture shown in Figure 5, in the aforementioned step S302, the query information and the search results of the current search path are input into the backbone network, and the backbone network outputs a feature vector; the feature vector is input into the prediction generation module, and the prediction generation module generates alternative searches for the next search process based on the feature vector. Further, in the aforementioned step S303, the feature vector is input into the quality prediction module, and the quality prediction module predicts the quality information of the alternative search paths corresponding to the alternative searches based on the feature vector.

[0140] For example, the query information and the search results of the current search path are input into the backbone network, which outputs a feature vector. The feature vector is then input into the prediction generation module to generate a candidate search for the next search process. This process involves the human-computer interaction model sampling a candidate search. Simultaneously, the feature vector is input into the quality prediction module, which predicts and outputs the quality information of the candidate search path corresponding to the candidate search.

[0141] By sampling multiple times through a human-computer interaction model, multiple alternative searches can be generated for the next search process, and the quality information of the alternative search path corresponding to each alternative search can be obtained.

[0142] It should be noted that this embodiment only uses one dynamic programming process as an example, where the input data includes query information and the search results of the current search path. The implementation principle is similar for other cases where the input data is different, and will not be described in detail here.

[0143] In this embodiment, the machine-interaction model and the process reward model can share the same backbone network, which can make more efficient use of computing resources and improve the efficiency of dynamic programming search paths, thereby improving search efficiency.

[0144] Figure 6 is a flowchart of a model training method provided in an exemplary embodiment of this disclosure. As shown in Figure 6, the process of jointly training the human-computer interaction model and the process reward model in the foregoing embodiment includes:

[0145] Step S601: Construct a training set, which includes query samples, search path samples corresponding to the query samples, search results of each search process in the search path samples, and quality information of the search path corresponding to each search process.

[0146] The search path sample is a single sample of search paths. A search path records the search process based on the query information; specifically, it records the search statements used in each search process sequentially, with each search statement representing a search process. Each search path sample includes at least one search process.

[0147] In this embodiment, a training set can be automatically constructed using a large language model. The large language model can be any pre-trained large language model with powerful natural language processing and understanding capabilities, such as various teacher models. Specifically, the server collects a large number of query samples, which can include natural language questions input by users in various human-computer interaction scenarios. The collected query samples are cleaned, filtering out samples that do not meet the query sample requirements and retaining those that do. The requirements for query samples may include, but are not limited to: length within a specified range and the absence of meaningless symbols (such as links). The requirements for query samples can be set according to actual application needs and are not specifically limited here.

[0148] The search path samples include positive samples and negative samples. Positive samples are search path samples whose quality information meets the preset conditions, while negative samples are search path samples whose quality information does not meet the preset conditions.

[0149] For each query sample, the teacher model iterates through multiple iterations to generate multiple search path samples corresponding to the query sample, and then obtains the search results for each search process within the search path samples. The specific process is as follows:

[0150] The query sample is input into the teacher model, which analyzes whether its existing knowledge is sufficient to correctly respond to the query sample. If it is determined that it is sufficient to correctly respond to the query sample, a response is generated. If it is determined that it is insufficient to correctly respond to the query sample, the teacher model analyzes the data items needed to generate the response (i.e., the missing data items for generating the response) and generates a search process (i.e., the search statements used) to obtain the corresponding data items, resulting in multiple first search processes of the search path sample, as well as information on the thought process corresponding to each search process (optional). An initial search path sample (including the first search process) can be constructed based on each first search process. The search is performed according to the first search process to obtain the search results of the first search process.

[0151] For each current search path sample, the query sample, the search results of the current search path sample, and the corresponding thought process information (optional) are input into the teacher model. The teacher model analyzes whether its existing knowledge and the search results of the current search path sample are sufficient to correctly respond to the query sample. If it is determined that it is sufficient to correctly respond to the query sample, the response information for the query sample is generated, and the current search path sample is considered a complete search path sample for the query sample.

[0152] When the model determines that the response to the query sample is insufficient, it analyzes the data items needed to generate the response (i.e., the missing data items for generating the response) and generates a search process (i.e., the search statement used) to obtain the corresponding data items. This results in multiple next search processes for the current search path sample, along with corresponding thought process information (optional). Each next search process is added to the end of the current search path sample to obtain multiple new search path samples. The search is then performed based on the second search process to obtain the search results for that second search process.

[0153] This process continues until the iteration stopping condition is met, resulting in multiple search path samples for the query sample. At least one of these search path samples must have search results sufficient to correctly respond to the query information; that is, the teacher model can correctly generate a response based on the search results of that search path sample. The stopping condition can be either reaching a preset number of iterations or the maximum length of the current search path sample reaching a preset length. The stopping condition can be set and adjusted according to the actual application scenario; no specific limitations are specified here.

[0154] In this embodiment, for any query sample and multiple search path samples, the search path samples are divided into positive samples and negative samples based on whether the search results based on the search path samples are sufficient to correctly answer the query sample. Search results based on positive samples are sufficient to correctly answer the query sample, while search results based on negative samples are insufficient to correctly answer the query sample.

[0155] Optionally, for any query sample across multiple search path samples, some positive samples and / or some negative samples can be filtered out, ensuring that the proportion of positive samples in the retained search path samples falls within a preset range. This preset range can be set and adjusted according to the needs of the actual application scenario, such as 30%-70%, and is not specifically limited here. When filtering multiple search path samples for the same query sample, positive samples with shorter path lengths can be prioritized for retention, enabling the trained human-computer interaction model to plan shorter search paths and improve response timeliness. Negative samples can be randomly selected for retention. Furthermore, the filtering rules for positive and negative samples can be set according to actual application requirements, and are not specifically limited here.

[0156] Optionally, a Monte Carlo Tree Search (MCTS) algorithm can be used to iteratively perform the above path search to construct multiple search path samples for the query information. These multiple search path samples for any query information can be stored as a tree structure. The root node of the tree structure is the query sample, internal nodes (excluding the root node) correspond to a search process, and leaf nodes correspond to the final search result or generated response. For positive samples in the search path samples, the quality information of the corresponding leaf node is initialized to 1.0, indicating that the quality information of the positive sample meets a preset condition. The quality information of the corresponding leaf node is initialized to 0, indicating that the quality information of the negative sample does not meet the preset condition. An Upper Confidence Bound Applied to Trees (UTC) algorithm is used to update the quality information of the internal nodes on the search path samples based on the quality information of the leaf nodes.

[0157] For example, Figure 7 is an example diagram of a tree structure of search path samples provided in an exemplary embodiment of this disclosure. As shown in Figure 7, taking the query sample "What is the market capitalization of the world's highest-valued technology company in 2024?" as an example, this query sample has 4 search path samples. Starting from the root node of the tree (query sample), a path from the root node to a leaf node corresponds to a search path sample, following the direction indicated by the arrows in the figure. Among them, those marked with "×" correspond to negative samples, and those marked with "√" correspond to positive samples. The Q value on each node represents the quality information of the search path corresponding to the node. The dashed arrows in the figure represent the process of backtracking and updating the quality information of the internal nodes on the search path samples.

[0158] Based on the training set, the human-computer interaction model and the process reward model are jointly trained through steps S602-S603 to obtain the trained human-computer interaction model and process reward model.

[0159] Step S602: For any search process in any search path sample, the human-computer interaction model generates a prediction result for the search process based on the search results of the sub-search paths preceding the search process in the query sample and search path sample, and generates a quality prediction result for the search path corresponding to the search process through the process reward model.

[0160] In this step, for any search process within any search path sample (e.g., the (i+1)th search process), the search results of the sub-search paths preceding the (i+1)th search process are obtained, i.e., the search results of the search path corresponding to the (i)th search process. The query sample and the search results of the search path corresponding to the (i)th search process are input into the human-computer interaction model, and a prediction result for the (i+1)th search process is generated based on these results. The prediction result for the (i+1)th search process includes the probability distribution for predicting the result of the next (i+1)th search process.

[0161] Input the query sample and the search results of the search path corresponding to the i-th search process into the process reward model to generate the quality prediction result of the search path corresponding to the next (i+1) search process.

[0162] In an optional embodiment, based on the model architecture shown in Figure 5, during the joint training of the human-computer interaction model and the process reward model, for any search process in any search path sample, the search results of the query sample and the sub-search paths preceding the search process in the search path sample are input into the backbone network to generate output features. The output features are then input into the prediction generation module to generate the prediction result of the search process. Finally, the output features are input into the quality prediction module to generate the quality prediction result of the search path corresponding to the search process. The human-computer interaction model and the process reward model share the backbone network, which can utilize computing resources more efficiently and improve the efficiency of model training.

[0163] It should be noted that during the training process, the prediction result of the next search process is generated through the human-computer interaction model, and the quality prediction result of the search path corresponding to the next search process is generated through the process reward model. This is consistent with the implementation principle of dynamically planning the next search process and evaluating the quality information of the search path corresponding to the next search process through the process reward model in the previous embodiment. For details, please refer to the relevant content of the previous embodiment, which will not be repeated here.

[0164] Step S603: Based on the prediction results of the search process, the quality information and quality prediction results of the search path corresponding to the search process, adjust the parameters of the human-computer interaction model and the process reward model to obtain the trained human-computer interaction model and process reward model.

[0165] In this embodiment, a loss function value is calculated based on the prediction results of the search process, the quality information of the search path corresponding to the search process, and the quality prediction results. Based on the loss function value, the parameters of the human-computer interaction model and the process reward model are updated through backpropagation.

[0166] Specifically, the path planning loss is calculated based on the prediction results of each search process in the positive samples, and the parameters of the human-computer interaction model are updated through backpropagation based on the path planning loss. Optionally, the cross-entropy loss can be calculated based on the prediction results of each search process in the positive samples and used as the path planning loss.

[0167] Based on the quality information and quality prediction results of the search paths corresponding to each search process, calculate the quality assessment loss. Then, update the parameters of the process reward model using backpropagation based on the quality assessment loss.

[0168] Optionally, the mean squared error (MSE) loss, i.e., the MSE loss function value, can be calculated based on the quality information and quality prediction results of the corresponding search paths for each search process. The MSE loss is then used as the quality assessment loss. Alternatively, the L1 loss can also be calculated based on the quality information and quality prediction results of the corresponding search paths for each search process, and the L1 loss is used as the quality assessment loss. Furthermore, other loss functions can be used to calculate the quality assessment loss; no specific limitations are imposed here.

[0169] In one optional embodiment, the path planning loss is calculated based on the prediction results of each search process in the positive samples; the quality evaluation loss is calculated based on the quality information and quality prediction results of the search paths corresponding to each search process; the path planning loss and the quality evaluation loss are weighted and summed to obtain the comprehensive loss. Based on the comprehensive loss, the parameters of the human-computer interaction model and the process reward model are updated through backpropagation to achieve joint training of the human-computer interaction model and the process reward model, thereby improving the performance of the human-computer interaction model and the process reward model, and thus improving the search quality.

[0170] It should be noted that when jointly training the human-computer interaction model and the process reward model, the training strategy used, including the learning rate, optimization algorithm, and iteration stopping condition, can be set according to the actual application requirements. This embodiment does not impose specific limitations here.

[0171] The solution in this embodiment uses an automatically constructed training set to jointly train the human-computer interaction model and the process reward model. By combining the human-computer interaction model and the process reward model, the dynamic planning scheme of the search path in the above embodiment can be realized, thereby improving the search quality and thus improving the quality of the response generated based on the search results.

[0172] Figure 8 is a flowchart of a data processing method provided in another exemplary embodiment of this disclosure. In this embodiment, the solution provided in the foregoing embodiments is applied to a human-computer interaction scenario based on retrieval enhancement. The executing entity can be a server that provides retrieval enhancement capabilities, used to achieve retrieval enhancement based on user-input queries and obtain target search results. As shown in Figure 8, the specific steps of this method are as follows:

[0173] Step S801: Receive the retrieval enhancement request sent by the receiving end device. The retrieval enhancement request contains query information.

[0174] After receiving the query information, the server iteratively performs dynamic programming steps S802-S805 based on the query information. Combining the human-computer interaction model and the process reward model, it dynamically plans the next search step for the current search path, generates new search paths, and filters high-quality new search paths until a target search path that meets preset conditions is obtained. Based on the target search path, the target search result can be obtained. The target search result includes the information needed to respond to the query information.

[0175] Step S802: Through the human-computer interaction model, based on the query information and the search results of the current search path, dynamically plan the next search process of the current search path to obtain multiple alternative search paths.

[0176] Step S803: Evaluate the quality information of each alternative search path using a process reward model.

[0177] Step S804: Are there any alternative search paths that meet the preset conditions?

[0178] Step S805: If the quality information of each candidate search path does not meet the preset conditions, then at least one candidate search path is selected as a new search path based on the quality information of each candidate search path.

[0179] Step S806: Until a target search path that meets the preset conditions is obtained, the target search result of the target search path is obtained.

[0180] In this embodiment, the implementation principle of steps S802-S806 is the same as that of steps S302-S306 mentioned above. For details, please refer to the relevant content of the aforementioned embodiment, which will not be repeated here.

[0181] Step S807: Return the target search result to the end-side device.

[0182] The obtained target search result is the enhanced search result. The server returns the target search result to the endpoint device.

[0183] Furthermore, the edge device can generate response information based on the target search results.

[0184] In this embodiment, during the dynamic planning process of each search process, a human-computer interaction model is used to dynamically plan each search process and select high-quality search paths based on quality evaluation results, thus avoiding search instability. During the dynamic planning of each search process, a process reward model is used to evaluate the quality information of each candidate search path and select high-quality search paths. This allows for real-time quality assessment and correction of the search process, ensuring that each search process moves towards a better direction. This enables dynamic adjustment of the search strategy, achieving efficient and high-quality search enhancement, and significantly improving the accuracy and reliability of the search enhancement results.

[0185] Figure 9 is a flowchart of a data processing method provided in another exemplary embodiment of this disclosure. In this embodiment, the solution provided in the foregoing embodiments is applied to a human-computer interaction scenario based on retrieval enhancement. The executing entity can be a server-side device of the human-computer interaction system, responsible for implementing retrieval enhancement based on the user's input query, obtaining the target search result, and generating response information based on the target search result. As shown in Figure 9, the specific steps of this method are as follows:

[0186] Step S901: Receive query information sent by the client device.

[0187] During human-computer interaction, users can input query information (i.e., queries) into the human-computer interaction system using a client device. The client device then sends the user-input query information to the server device. The server device receives the query information sent by the client device.

[0188] Step S902: Through the human-computer interaction model, based on the query information and the search results of the current search path, dynamically plan the next search process of the current search path to obtain multiple alternative search paths.

[0189] Step S903: Evaluate the quality information of each alternative search path using a process reward model.

[0190] Step S904: Are there any alternative search paths that meet the preset conditions?

[0191] Step S905: If the quality information of each candidate search path does not meet the preset conditions, then at least one candidate search path is selected as a new search path based on the quality information of each candidate search path.

[0192] Step S906: Until a target search path that meets the preset conditions is obtained, the target search result of the target search path is obtained.

[0193] In this embodiment, the implementation principle of steps S902-S906 is the same as that of steps S302-S306 mentioned above. For details, please refer to the relevant content of the aforementioned embodiment, which will not be repeated here.

[0194] Step S907: Generate a response message for the query information based on the target search results using the human-computer interaction model.

[0195] In this step, after obtaining the target search result, the server device generates a response message based on the query information through a human-computer interaction model.

[0196] Step S908: Return response information to the client device.

[0197] After generating the response information, the server device returns the response information to the client device. The client device then outputs the response information to the user.

[0198] In this embodiment, during human-computer interaction, the next search step is dynamically planned using a human-computer interaction model and a process reward model. Multiple alternative search paths are generated, and high-quality alternative paths are selected as new search paths, thus improving the quality of the planned target search path. After multiple iterations, a high-quality target search path is obtained. Based on this high-quality target search path, accurate search results, including the information needed to respond to the query, can be obtained. The target search results have a higher degree of matching with the user's intent, better solving the user's problem and improving the quality of the search results. Generating response information based on the target search results improves the quality of the generated response.

[0199] Figure 10 is a structural block diagram of a computing device according to an embodiment of the present disclosure. As shown in Figure 10, the computing device may include one or more (only one is shown in the figure) processors 1001 and memory 1002. The memory 1002 is used to store computer programs / instructions, and the processor 1001 is used to execute the computer programs / instructions. When the computer programs / instructions are executed by the processor 1001, they implement the technical solutions provided in any of the foregoing method embodiments. Their specific functions and the technical effects they can achieve are similar, and will not be repeated here.

[0200] The aforementioned computing device can be understood as an integrated smart terminal, including but not limited to servers, desktop computers, PCs (Personal Computers), all-in-one model machines, mobile phones, tablet computers, or other portable smart terminals, and the computing device may have the model in the above embodiments of this disclosure pre-installed.

[0201] Specifically, this computing device can pre-install various types of models, including but not limited to models in natural language processing, visual processing, speech processing, code processing, and multimodal task processing, thus providing diverse model selection. In different product forms, this computing device can support one or more model usage methods, including but not limited to model training, model invocation, model fine-tuning, model deployment, model inference, and application. In some product forms, this computing device also supports model management, including but not limited to multi-type model management (supporting the management of discriminative, generative, and other types of models), model version control (supporting the control of different model versions), and model evaluation (evaluating model performance and effectiveness based on model evaluation tools). In other product forms, this computing device can also create applications based on models, providing API (Application Programming Interface) calling capabilities. Users can call models into created applications through the API interface, and application management tools are also provided to manage and monitor the applications.

[0202] Furthermore, the computing device may also include data management (supporting the creation and management of model tuning datasets), a training center (providing abundant training resources to help users learn and master AI technology), and basic control capabilities (providing enterprise-level basic control capabilities to ensure the security and efficient operation of the system). Through the above functions, it provides a comprehensive and integrated device for AI development, training, deployment, and application.

[0203] Figure 11 is a schematic diagram of a server structure provided in an embodiment of this disclosure. As shown in Figure 11, the server includes a memory 1101 and a processor 1102. The memory 1101 is used to store computer execution instructions and can be configured to store various other data to support operations on the server. The processor 1102 is communicatively connected to the memory 1101 and is used to execute the computer execution instructions stored in the memory 1101 to implement the technical solutions provided in any of the above method embodiments. Their specific functions and the technical effects they can achieve are similar and will not be repeated here.

[0204] Optionally, as shown in Figure 11, the server may also include other components such as a firewall 1103, a load balancer 1104, a communication component 1105, and a power supply component 1106. Figure 11 only schematically illustrates some of the components and does not imply that the server only includes the components shown in Figure 11. Figure 11 uses a cloud server deployed in the cloud as an example for illustrative purposes; the server can also be deployed locally, and this embodiment does not impose specific limitations.

[0205] This disclosure also provides a computer-readable storage medium storing computer-executable instructions. When a processor executes the computer-executable instructions, it implements the method of any of the foregoing embodiments. The specific functions and technical effects to be achieved are not described here.

[0206] This disclosure also provides a computer program product, including a computer program that, when executed by a processor, implements the methods of any of the foregoing embodiments. The computer program is stored in a readable storage medium, and at least one processor of the server can read the computer program from the readable storage medium. The execution of the computer program by the at least one processor causes the server to perform the technical solutions provided in any of the above method embodiments; the specific functions and achievable technical effects are not elaborated here.

[0207] This disclosure provides a chip, including a processing module and a communication interface. The processing module is capable of executing the technical solutions of the server in the aforementioned method embodiments. Optionally, the chip further includes a storage module (e.g., a memory), which stores instructions. The processing module executes the instructions stored in the storage module, and the execution of the instructions stored in the storage module causes the processing module to execute the technical solutions provided in any of the aforementioned method embodiments.

[0208] The integrated modules described above, implemented as software functional modules, can be stored in a computer-readable storage medium. These software functional modules, stored in a storage medium, include several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute some steps of the methods of the various embodiments of this disclosure.

[0209] It should be understood that the aforementioned processor can be a Central Processing Unit (CPU), a graphics processing unit (GPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), etc. A general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in the application can be directly manifested as being executed by a hardware processor, or executed by a combination of hardware and software modules in at least one processor.

[0210] The memory may include high-speed random access memory (RAM) and may also include non-volatile memory, such as at least one disk storage device, and may also be a USB flash drive, external hard drive, read-only memory, disk or optical disc, etc.

[0211] The aforementioned storage device can be object storage service (OSS).

[0212] The aforementioned memory can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as Static Random Access Memory (SRAM), Electrically Erasable Programmable Read Only Memory (EEPROM), Erasable Programmable Read Only Memory (EPROM), Programmable Read Only Memory (PROM), Read Only Memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0213] The aforementioned communication components are configured to facilitate wired or wireless communication between the device containing the communication components and other devices. The device containing the communication components can access wireless networks based on communication standards, such as mobile hotspots (WiFi), second-generation (2G), third-generation (3G), fourth-generation (4G) / Long Term Evolution (LTE), fifth-generation (5G), or combinations thereof. In one exemplary embodiment, the communication components receive broadcast signals or broadcast-related information from an external broadcast management system via a broadcast channel. In one exemplary embodiment, the communication components also include a Near Field Communication (NFC) module to facilitate short-range communication. For example, the NFC module may be implemented based on Radio Frequency Identification (RFID), infrared, Ultra Wide Band (UWB), Bluetooth, and other technologies.

[0214] The aforementioned power supply components provide power to various components within the device in which they reside. These power supply components may include a power management system, one or more power sources, and other components associated with generating, managing, and distributing power to the device in which they reside.

[0215] The aforementioned storage medium can be implemented from any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. The storage medium can be any available medium accessible to general-purpose or special-purpose computers.

[0216] An exemplary storage medium is coupled to a processor, enabling the processor to read information from and write information to the storage medium. Alternatively, the storage medium can be an integral part of the processor. The processor and storage medium can reside within an application-specific integrated circuit (ASIC). Alternatively, the processor and storage medium can exist as discrete components within an electronic device or host device.

[0217] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0218] The order of the embodiments described above is merely for descriptive purposes and does not represent the superiority or inferiority of the embodiments. Furthermore, some processes described in the above embodiments and accompanying drawings include multiple operations appearing in a specific order; however, it should be clearly understood that these operations may not be executed in the order they appear herein or may be executed in parallel. The sequence numbers are merely used to distinguish different operations, and the sequence numbers themselves do not represent any execution order. Additionally, these processes may include more or fewer operations, and these operations may be executed sequentially or in parallel. It should be noted that the descriptions such as "first," "second," etc., in this document are used to distinguish different messages, devices, modules, etc., and do not represent a sequential order, nor do they limit "first" and "second" to different types. "Multiple" means two or more, unless otherwise explicitly specified.

[0219] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this disclosure, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods of the various embodiments of this disclosure.

[0220] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This disclosure is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein.

[0221] The above are merely preferred embodiments of this disclosure and do not limit the patent scope of this disclosure. Any equivalent structural or procedural transformations made using the content of this disclosure and its drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of this disclosure.

Claims

1. A data processing method, wherein, include: In response to receiving query information, the following dynamic programming process is performed iteratively based on the query information: Through the human-computer interaction model, based on the query information and the search results of the current search path, the next search process of the current search path is dynamically planned to obtain multiple alternative search paths; The quality information of each of the proposed alternative search paths is evaluated using a process reward model. If the quality information of each of the candidate search paths does not meet the preset conditions, then at least one of the candidate search paths is selected as a new search path based on the quality information of each candidate search path. Until a target search path that meets the preset conditions is obtained, the target search result of the target search path is obtained, and the target search result is used to generate the response information for the query information.

2. The method of claim 1, wherein, The human-computer interaction model dynamically plans the next search process for the current search path based on the query information and the search results of the current search path, resulting in multiple alternative search paths, including: Based on the query information and the search results of the current search path, the human-computer interaction model generates multiple alternative searches for the next search process of the current search path. Each of the alternative searches is added to the current search path to obtain multiple alternative search paths.

3. The method of claim 1, wherein, The human-computer interaction model dynamically plans the next search process for the current search path based on the query information and the search results of the current search path, resulting in multiple alternative search paths, including: Based on the query information and the search results of the current search path, the human-computer interaction model generates multiple alternative searches for the next search process of the current search path, as well as the thought process information corresponding to each alternative search. Each of the alternative searches is added to the current search path to obtain multiple alternative search paths.

4. The method of claim 1, wherein, The quality information of each of the candidate search paths evaluated through the process reward model includes: By using the process reward model, based on the query information and the search results of each of the alternative search paths, the probability of generating a correct response to the query information based on the search results of each of the alternative search paths is predicted, thereby obtaining the quality information of each of the alternative search paths.

5. The method of claim 2, wherein, The human-computer interaction model includes a backbone network and a prediction and generation module, and the process reward model includes the backbone network and a quality prediction module. The step of generating multiple alternative searches for the next search process of the current search path through the human-computer interaction model, based on the query information and the search results of the current search path, includes: For any of the candidate searches, the query information and the search results of the current search path are input into the backbone network, and a feature vector is output. The feature vector is input into the prediction generation module to generate the candidate search.

6. The method of claim 5, wherein, The quality information of each of the candidate search paths evaluated through the process reward model includes: The feature vector is input into the quality prediction module to generate quality information of the alternative search path corresponding to the alternative search.

7. The method of any one of claims 1-6, wherein, The step of selecting at least one of the candidate search paths as a new search path based on the quality information of each candidate search path includes: Based on the quality information of each of the candidate search paths, the candidate search paths are sorted to obtain the sorting result; Based on the sorting results, a preset number of candidate search paths are selected as new search paths.

8. The method of any one of claims 1-6, wherein, After obtaining the target search results for the target search path, the process further includes: Output the target search result; or, The human-computer interaction model generates and outputs response information for the query based on the target search results.

9. A data processing method, wherein, include: Construct a training set, which includes query samples and search path samples corresponding to the query samples, wherein each search path sample includes at least one search process, the search results of each search process, and the quality information of the search path corresponding to each search process; For any search process in any of the search path samples, a prediction result of the search process is generated by the human-computer interaction model based on the query sample and the search results of the sub-search paths in the search path sample before the search process, and a quality prediction result of the search path corresponding to the search process is generated by the process reward model. Based on the prediction results of the search process, the quality information of the search path corresponding to the search process, and the quality prediction results, the parameters of the human-computer interaction model and the parameters of the process reward model are adjusted to obtain the trained human-computer interaction model and process reward model.

10. The method of claim 9, wherein, The search path samples include positive samples and negative samples. The positive samples are search path samples whose quality information meets the preset conditions, and the negative samples are search path samples whose quality information does not meet the preset conditions. The step of adjusting the parameters of the human-computer interaction model and the process reward model based on the prediction results of the search process, the quality information of the search path corresponding to the search process, and the quality prediction results, to obtain the trained human-computer interaction model and process reward model, includes: Calculate the path planning loss based on the prediction results of each search process in the positive samples; Calculate the quality assessment loss based on the quality information of the search path corresponding to each search process and the quality prediction result. Based on the path planning loss and the quality assessment loss, adjust the parameters of the human-computer interaction model and the process reward model to obtain the trained human-computer interaction model and process reward model.

11. The method of claim 9 or 10, wherein, The human-computer interaction model includes a backbone network and a prediction generation module, and the process reward model includes the backbone network and a quality prediction module. For any search process in any of the search path samples, a prediction result for the search process is generated by a human-computer interaction model based on the query sample and the search results of the sub-search paths preceding the search process in the search path sample. A quality prediction result for the search path corresponding to the search process is generated by a process reward model, including: For any search process in any of the search path samples, the search results of the query sample and the sub-search paths in the search path sample before the search process are input into the backbone network to generate output features; The output features are input into the prediction generation module to generate the prediction results of the search process; The output features are input into the quality prediction module to generate quality prediction results for the search path corresponding to the search process.

12. A data processing method, wherein, include: The receiving end-side device sends a search enhancement request, which includes query information; Based on the query information, the following dynamic programming process is performed iteratively: Through the human-computer interaction model, based on the query information and the search results of the current search path, the next search process of the current search path is dynamically planned to obtain multiple alternative search paths; The quality information of each of the proposed alternative search paths is evaluated using a process reward model. If the quality information of each of the candidate search paths does not meet the preset conditions, then at least one of the candidate search paths is selected as a new search path based on the quality information of each candidate search path. Until a target search path that meets the preset conditions is obtained, the target search result of the target search path is obtained; The target search result is returned to the terminal device.

13. The method of claim 12, wherein, The human-computer interaction model dynamically plans the next search process for the current search path based on the query information and the search results of the current search path, resulting in multiple alternative search paths, including: Based on the query information and the search results of the current search path, the human-computer interaction model generates multiple alternative searches for the next search process of the current search path. Each of the alternative searches is added to the current search path to obtain multiple alternative search paths.

14. The method of claim 12, wherein, The human-computer interaction model dynamically plans the next search process for the current search path based on the query information and the search results of the current search path, resulting in multiple alternative search paths, including: Based on the query information and the search results of the current search path, the human-computer interaction model generates multiple alternative searches for the next search process of the current search path, as well as the thought process information corresponding to each alternative search. Each of the alternative searches is added to the current search path to obtain multiple alternative search paths.

15. The method of claim 12, wherein, The quality information of each of the candidate search paths evaluated through the process reward model includes: By using the process reward model, based on the query information and the search results of each of the alternative search paths, the probability of generating a correct response to the query information based on the search results of each of the alternative search paths is predicted, thereby obtaining the quality information of each of the alternative search paths.

16. A data processing method, wherein, include: Receive query information sent by the client device; Based on the query information, the following dynamic programming process is performed iteratively: Through the human-computer interaction model, based on the query information and the search results of the current search path, the next search process of the current search path is dynamically planned to obtain multiple alternative search paths; The quality information of each of the proposed alternative search paths is evaluated using a process reward model. If the quality information of each of the candidate search paths does not satisfy the preset condition, at least one of the candidate search paths is selected as a new search path according to the quality information of each of the candidate search paths; Until a target search path satisfying the preset condition is obtained, a target search result of the target search path is acquired; Through the man-machine interaction model, reply information of the query information is generated according to the target search result; The reply information is returned to the client device.

17. A computing device, wherein, Comprise: A memory and a processor; The memory is used to store computer programs / instructions, and the processor is used to execute the computer programs / instructions, and the computer programs / instructions are executed by the processor to realize the method of any one of claims 1-16.

18. A server, wherein, Comprise: At least one processor; And A memory connected in communication with the at least one processor; Among them, the memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the server to execute the method of any one of claims 1-16.

19. A computer readable storage medium, wherein, The computer readable storage medium stores computer execution instructions, and when the processor executes the computer execution instructions, the method as claimed in any one of claims 1-16 is realized.

20. A computer program product comprising a computer program, wherein, The computer program is executed by the processor to realize the method as claimed in any one of claims 1-16.

Images