Data processing method and device and storage medium

Abstract

The invention provides a data processing method and device and a storage medium, and the method relates to the technical field of natural language processing, and is used for improving the answering quality of an intelligent question answering system. The method comprises the steps of obtaining a question input by a user; inputting the question into a plurality of large language models to obtain a plurality of question and answer results output by the plurality of large language models, and generating the question and answer results by different large language models aiming at the same question from different description angles; n question and answer results are selected from the multiple question and answer results through a discriminator according to the user preference information of the user, the discriminator is obtained by training the question and answer results selected according to questions historically input by the user, and N is a positive integer; and displaying the N question and answer results.

Description

Technical Field

[0001] The present application relates to the technical field of natural language processing, and in particular to a data processing method, device and storage medium. Background Art

[0002] With the popularization of the Internet and the rapid development of e-commerce, customer service systems have become an indispensable part of corporate operations. Traditional customer service systems usually adopt a rule-based or template-based response method, but this method often cannot adapt to complex customer needs and problems, and it is difficult to achieve automation and intelligence.

[0003] In recent years, with the continuous development of artificial intelligence technology, intelligent question-answering systems based on large language model technology have gradually become a research hotspot. Large language model technology can process large amounts of text data, automatically learn and extract language patterns and knowledge graphs, thereby achieving goals such as automatically answering customer questions and providing personalized services.

[0004] However, the answer quality of existing intelligent question-answering systems based on large language model technology is sometimes difficult to guarantee, resulting in users being unable to obtain the desired question-answering results. Therefore, how to improve the answer quality of intelligent question-answering systems is a technical problem that needs to be solved urgently. Summary of the invention

[0005] The embodiments of the present application provide a data processing method, device and storage medium for improving the answer quality of an intelligent question and answer system.

[0006] In a first aspect, an embodiment of the present application provides a data processing method, the method comprising: obtaining a question input by a user; inputting the question into multiple large language models to obtain multiple question and answer results output by the multiple large language models, wherein different large language models generate question and answer results for the same question from different description angles; selecting N question and answer results from the multiple question and answer results according to the user preference information of the user through a discriminator, the discriminator being trained based on the question and answer results selected according to the questions historically input by the user, and N being a positive integer; and displaying the N question and answer results.

[0007] In a possible implementation, the question is input into multiple large language models to obtain multiple question and answer results output by the multiple large language models, including: expanding the question to obtain M questions, where M is a positive integer; and inputting the M questions into each of the multiple large language models respectively to obtain multiple question and answer results output by the multiple large language models.

[0008] In a possible implementation, the discriminator selects N question and answer results from the multiple question and answer results based on the user preference information of the user, including: the discriminator determines the negative log-likelihood loss of the multiple question and answer results based on the user preference information of the user; and selects N question and answer results with the smallest negative log-likelihood loss from the multiple question and answer results.

[0009] In a possible implementation, when N is greater than 1, N is 2.

[0010] In a possible implementation, the method further includes: responding to the user's operation of selecting a target question and answer result based on the N question and answer results; displaying the target question and answer result and hiding other question and answer results, where the other question and answer results are the question and answer results among the N question and answer results excluding the target question and answer result.

[0011] In a possible implementation, the method further includes: adding the target question and answer result to a first data set, and adding other question and answer results among the N question and answer results excluding the target question and answer result to a second data set, wherein the first data set and the second data set are used to optimize the discriminator.

[0012] In a possible implementation, the method further includes: when it is determined that the amount of data in the first data set and / or the second data set is greater than a preset data amount, optimizing the discriminator based on a direct preference optimization algorithm, and the data sets used to optimize the discriminator are the first data set and the second data set.

[0013] In a second aspect, a data processing device is provided, including: an acquisition module, used to obtain questions input by a user; a processing module, used to input the questions into multiple large language models, and obtain multiple question and answer results output by the multiple large language models, where different large language models generate question and answer results with different description angles for the same question; a discriminator is used to select N question and answer results from the multiple question and answer results according to the user preference information of the user, wherein the discriminator is trained based on the question and answer results selected according to the questions historically input by the user, and N is a positive integer; a display module is used to display the N question and answer results.

[0014] In a possible implementation, the processing module is specifically used to: expand the question to obtain M questions, where M is a positive integer; input the M questions into each of the multiple large language models respectively to obtain multiple question-and-answer results output by the multiple large language models.

[0015] In one possible implementation, the processing module is specifically used to: determine the negative log-likelihood loss of the multiple question and answer results according to the user preference information of the user through the discriminator; and select N question and answer results with the smallest negative log-likelihood loss from the multiple question and answer results through the discriminator.

[0016] In a possible implementation manner, N is 2.

[0017] In a possible implementation, when N is greater than 1, the processing module is further used to: respond to the user's operation of selecting a target question and answer result based on the N question and answer results; the display module is further used to: display the target question and answer result and hide other question and answer results, where the other question and answer results are the question and answer results other than the target question and answer result among the N question and answer results.

[0018] In a possible implementation, the processing module is further used to: add the target question and answer result to a first data set, and add other question and answer results among the N question and answer results except the target question and answer result to a second data set, and the first data set and the second data set are used to optimize the discriminator.

[0019] In a possible implementation, the processing module is specifically used to: when it is determined that the amount of data in the first data set or the second data set is greater than a preset data amount, optimize the discriminator based on a direct preference optimization algorithm, and the data sets used to optimize the discriminator are the first data set and the second data set.

[0020] In a third aspect, an embodiment of the present application provides an electronic device, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the at least one processor executes the instructions stored in the memory, so that the at least one processor executes the method described in the first aspect or any possible implementation of the first aspect.

[0021] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, which is used to store instructions. When the instructions are executed, the method in the first aspect or any possible implementation of the first aspect is implemented.

[0022] In a fifth aspect, an embodiment of the present application provides a computer program product comprising instructions, wherein the computer program product stores instructions that, when executed on a computer, enable the computer to execute a method as described in the first aspect or any possible implementation method of the first aspect.

[0023] In an embodiment of the present application, a question input by a user is obtained, and the question is input into multiple different large language models, and reasoning is performed through the multiple different large language models to obtain multiple question and answer results. Then, a discriminator selects N question and answer results from the multiple question and answer results according to the user's user preference information, and the N question and answer results are displayed.

[0024] That is to say, this application uses multiple different large language models for reasoning, and can obtain multiple different question and answer results for the same question. Compared with reasoning through a large language model, the question and answer results obtained are more comprehensive, which helps to improve the question and answer quality of the intelligent question and answer system. In addition, by using a discriminator to select N question and answer results that are more in line with user preferences from multiple question and answer results obtained from multiple large language models, it helps to improve the user's question and answer experience. In addition, displaying the N question and answer results selected by the discriminator can enable users to select their preferred question and answer results based on the N question and answer results, which can further improve the user's question and answer experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0026] Figure 1 A schematic diagram of possible application scenarios provided for embodiments of the present application;

[0027] Figure 2 A flowchart of a data processing method provided in an embodiment of the present application;

[0028] Figure 3 A schematic diagram of the structure of an intelligent question-answering system provided in an embodiment of the present application;

[0029] Figure 4 A schematic diagram of the structure of a data processing device provided in an embodiment of the present application;

[0030] Figure 5 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0031] In order to make the purpose, technical scheme and advantages of the present application clearer, the technical scheme in the embodiment of the present application will be clearly and completely described below in conjunction with the drawings in the embodiment of the present application. Obviously, the described embodiment is only a part of the embodiment of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of the present application. In the absence of conflict, the embodiments in the present application and the features in the embodiments can be arbitrarily combined with each other. In addition, although the logical order is shown in the flow chart, in some cases, the steps shown or described can be performed in an order different from that here.

[0032] The terms "first" and "second" in the specification and claims of the present application and the above-mentioned drawings are used to distinguish different objects, rather than to describe a specific order. In addition, the term "comprising" and any of their variations are intended to cover non-exclusive protection. For example, a process, method, system, product or device comprising a series of steps or units is not limited to the listed steps or units, but optionally also includes steps or units that are not listed, or optionally also includes other steps or units inherent to these processes, methods, products or devices. "Multiple" in the present application can mean at least two, for example, two, three or more, and the embodiments of the present application are not limited.

[0033] The following is a description of exemplary embodiments of the present application in conjunction with the accompanying drawings, including various details of the embodiments of the present application to facilitate understanding, which should be considered as merely exemplary. Therefore, those of ordinary skill in the art should recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope of the present application. Similarly, for the sake of clarity and conciseness, the description of well-known functions and structures is omitted in the following description. It should be noted that in the embodiments of the present application, certain software, components, models and other existing solutions in the industry may be mentioned, which should be considered as exemplary, and their purpose is only to illustrate the feasibility of the implementation of the technical solution of the present application, but it does not mean that the applicant has or will necessarily use the solution.

[0034] In the technical solution of this application, the acquisition, transmission, storage, and use of data are in compliance with the requirements of relevant national laws and regulations.

[0035] In recent years, with the continuous development of artificial intelligence technology, intelligent question-answering systems based on large language model technology have gradually become a hot topic of research. Large language model technology can process large amounts of text data, automatically learn and extract language patterns and knowledge graphs, thereby achieving goals such as automatically answering customer questions and providing personalized services. However, for certain specific scenarios, the general capabilities of large language models are difficult to cover all aspects, resulting in the phenomenon that the question-answering results inferred by large language models do not answer the questions in certain scenarios or certain questions, resulting in low answer quality of the intelligent question-answering system, and users cannot get the desired question-answering results.

[0036] In order to solve this technical problem, the related technology mainly rewrites the questions input by the user to obtain multiple questions, and then uses a single large language model to infer the multiple questions to obtain multiple question-answering results. However, the semantics of the multiple questions obtained by rewriting are the same, and the multiple question-answering results obtained by reasoning with a single large language model may also give answers that are not related to the questions asked. That is, the quality of the answers given by the intelligent question-answering system may still be low, resulting in the user not being able to obtain the desired question-answering results, and the user's question-answering experience is poor.

[0037] In view of this, the present application proposes a data processing method to improve the answer quality of the intelligent question-answering system. First, the question input by the user can be obtained, and the question can be input into multiple large language models, wherein the general capabilities of the multiple large language models are different, that is, the multiple large language models output different question-answering results for the same question, so that the multiple question-answering results obtained based on the question can be more comprehensive, which helps to reduce the probability that the question-answering results output by the large language model do not answer the question, and can improve the answer quality of the intelligent question-answering system, thereby helping to improve the user's question-answering experience. Then, N question-answering results that are more in line with the user's preferences are selected from the multiple question-answering results, and the N question-answering results are displayed, so that the user can select the question-answering result that best meets his or her preferences, which can further improve the user's question-answering experience.

[0038] After introducing the main inventive ideas of the embodiments of the present application, the specific implementation of the data processing method proposed in the embodiments of the present application will be introduced below through some specific embodiments and drawings.

[0039] The following briefly introduces the application scenarios to which the technical solutions of the embodiments of the present application can be applied. It should be noted that the application scenarios introduced below are only used to illustrate the embodiments of the present application and are not limited. In the specific implementation process, the technical solutions provided by the embodiments of the present application can be flexibly applied according to actual needs.

[0040] Please refer to Figure 1 , which is a schematic diagram of a possible application scenario provided in an embodiment of the present application. In this scenario, a terminal device 101 and a server 102 may be included.

[0041] The terminal device 101 may be a mobile phone, a tablet computer (PAD), a personal computer (PC), a wearable device, a vehicle-mounted terminal, etc. The server 102 may be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDN), and big data and artificial intelligence platforms.

[0042] The terminal device 101 and the server 102 may be directly or indirectly connected to each other through one or more communication networks 103. The communication network 103 may be a wired network or a wireless network, for example, a mobile cellular network or a wireless fidelity (Wireless-Fidelity, WIFI) network, or other possible networks, which are not limited in the embodiment of the present invention.

[0043] The data processing method in the embodiment of the present application can be executed by a computer device, which can be a terminal device 101 or a server 102. Taking the data processing method executed by the server 102 as an example, the server 102 includes multiple large language models and discriminators. The server can receive questions from the terminal device 101 through the communication network 103, input the questions into multiple different large language models, and perform reasoning through the multiple large language models to obtain multiple question and answer results. Then, the server 102 can select N question and answer results that are more in line with the user's preference from the multiple question and answer results through the discriminator according to the user's user preference information. Finally, the server 102 can send the N question and answer results to the terminal device 101 through the communication network 103, and the terminal device 101 can display the N question and answer results when receiving the N question and answer results, for example, on the display screen of the terminal device 101. It can be understood that the multiple large language models and the discriminator can also communicate, that is, after the multiple large language models perform reasoning to obtain multiple question and answer results, the multiple question and answer results can be directly input into the discriminator, and the embodiment of the present application is not limited to this.

[0044] Figure 1 The scenarios shown are only examples. In other scenarios, more terminal devices and / or servers may be included, and the embodiments of the present application are not limited to this.

[0045] The following describes the method of cryptographic resource allocation provided by the exemplary embodiment of the present application in combination with the application scenario described above and with reference to the accompanying drawings. It should be noted that the above application scenario is only shown to facilitate understanding of the spirit and principles of the present application, and the implementation of the present application is not limited in this regard.

[0046] Please refer to Figure 2 , is a flow chart of a data processing method provided in an embodiment of the present application, the method can be Figure 1 The terminal device 101 and / or the server 102 shown in the figure are executed, and this application does not limit this. For the convenience of subsequent description, the following method will be used by Figure 1 Taking the terminal device 101 shown as an example, the data processing method proposed in this application is introduced.

[0047] S201: Obtaining a question input by a user.

[0048] The user may input questions in the form of text input, voice input, or other input methods. The embodiment of the present application does not limit the method in which the user inputs questions.

[0049] For example, the terminal device 101 includes a display screen. When the user conducts intelligent question and answering, the display screen can display a display interface including a question input box, and the user can enter the question he wants to ask in the question input box; or, the terminal device 101 includes a voice module, and the user can voice output the question he wants to ask, and the voice module can recognize the user's voice information and generate the question based on the voice information.

[0050] S202: Input the question into multiple large language models to obtain multiple question-answering results output by the multiple large language models.

[0051] Among them, the Large Language Model (LLM) can fully understand complex language structures and contextual information through large-scale data learning and training, thereby enabling high-quality answer generation. In addition, with the improvement of computing power and optimization of algorithms, the answers (i.e., question-and-answer results) inferred by the large language model are increasingly in line with user preferences, and the accuracy and completeness of the answers are also increasing.

[0052] In the embodiment of the present application, different large language models generate question-answering results from different description angles for the same question, that is, the multiple large language models are different models. For example, the multiple large language models include Qwen1.5-chat, BioBert, ERNIE-Code and other models, and the general capabilities of the multiple large language models are different. Therefore, the multiple large language models generate question-answering results from different description angles for the same question.

[0053] After obtaining the question input by the user, the terminal device 101 can input the question into the multiple large language models respectively, and each of the multiple large language models can perform reasoning on the question to obtain multiple different question-answering results. For example, the terminal device 101 can call the model application programming interface (Application Programming Interface, API) of the multiple large language models, and input the question input by the user into the corresponding large language model through the model API of each large language model.

[0054] Optionally, the terminal device 101 can call the model APIs of the multiple large language models at the same time, and input the questions input by the user into the multiple large language models at the same time. In this way, the reasoning time of the multiple large language models can be saved, which helps to improve the reasoning efficiency.

[0055] Optionally, before the terminal device 101 inputs the question input by the user into the multiple large language models, it can also expand the question input by the user according to the preset prompt words, for example, the question input by the user can be rewritten into multiple (for example, M) semantically identical questions by using synonym rewriting. Among them, common prompt words can be, for example, "You are an assistant who is good at linguistics. This is the question {question} that user 1 wants to ask. Your task is to convert the question into 3 synonymous sentences. You need to ensure that the meaning remains unchanged, and it is forbidden to add irrelevant information.", where {question} is the question input by the user, for example, "Use 20 words to describe how to make scrambled eggs with tomatoes?".

[0056] The terminal device 101 can input the M semantically identical questions into multiple large language models to obtain multiple question-answering results. For example, the terminal device 101 can rewrite the question input by the user into 4 semantically identical questions, and input the 4 semantically identical questions into multiple (e.g., 3) large language models to obtain 12 question-answering results. This helps to improve the comprehensiveness and accuracy of the question-answering results obtained by inference of multiple large language models.

[0057] S203: Selecting N question-and-answer results from the multiple question-and-answer results according to the user preference information of the user through a discriminator.

[0058] The discriminator is trained based on the question-answering results of the question selected by the user's historical input. The user preference information can be, for example, a text description format, such as a segmented description or a whole paragraph description, etc. The user preference information is obtained based on direct preference optimization (DPO). The direct preference optimization is a reinforcement learning method that aims to optimize the decision-making process of the model based on the user's direct feedback. The direct preference optimization is different from the traditional optimization method. The direct preference optimization can directly adjust the behavior of the system by analyzing the user's preference for different options, so as to better meet the user's needs.

[0059] Therefore, optionally, before selecting N question-answer results from the multiple question-answer results according to the user preference information of the user through the discriminator, the discriminator may also be trained according to the direct preference optimization algorithm. The training process is as follows:

[0060] S1: Get the training dataset.

[0061] The training data set includes data set 1 and data set 2, where the data in data set 1 are the question and answer results preferred by the user, and data set 1 can be called a positive data set, for example; the data in data set 2 are the question and answer results not preferred by the user, and data set 2 can be called a negative data set, for example. Data set 1 and data set 2 are obtained by users marking the question and answer results output by the large language model, and data set 1 can be recorded as y w , data set 2 can be recorded as y l .

[0062] S2: Modeling.

[0063] For example, the Bradley-Terry model can be used to model the data distribution of the question-answering results of user preferences to calculate the probability p of comparing different preferences y * , the modeling formula can refer to Formula 1.

[0064]

[0065] Among them, x is the question entered by the user, y w and l are all question-answering results generated by the large language model based on the questions input by the user, where y w is the question-answering result preferred by the user, that is, the data in dataset 1, y l is the question-answering result that is not user-preferred, that is, the data in dataset 2, r * (x, y) means that y is equivalent to the potential ground-truth reward model of x, p * represents the corresponding potential true probability.w and l It can be the question-answering result obtained by reasoning about the same question with the same large language model, or y w and l It can also be the question-answering results obtained by inferring the same question by different large language models. For example, question 1 can be expanded to obtain question 1 and question 2, and question 1 and question 2 can be input into large language model 1 in sequence, and the y can be obtained by inferring through large language model 1. w and l Alternatively, question 1 can be input into large language model 1 and large language model 2 respectively, and the y can be obtained by reasoning through large language model 1 and large language model 2. w and l .

[0066] r * (x,y) can refer to Formula 2.

[0067]

[0068] Among them, Z(x) is the partition function, and Z(x) can satisfy the following relationship, that is, The Bradley-Terry model is based on an exponential function, where y w and l The score r * (x,y) is transformed into the question-answering result y preferred by the user w probability.

[0069] S3: Train the discriminator.

[0070] According to the above formula 1, we sampled and obtained data set 3, and parameterized the reward model. Among them, data set 3 can be recorded as D, for example, and data set 3 includes the question x input by the user and the question-answer result y preferred by the user. w and non-user preferred question-answering results y l , data set 3 can refer to formula 3 for example.

[0071]

[0072] Among them, k is the number of sampled data, x, y w and l For relevant descriptions of , please refer to the relevant descriptions of Formula 1.

[0073] Then, the negative log-likelihood loss can be estimated by the maximum likelihood method. The calculation method of the negative log-likelihood loss can refer to Formula 4.

[0074]

[0075] Among them, x, y w and l For the relevant description of , please refer to the relevant description of the above formula 1, where, Indicates that the current strategy is The question-answering result y representing the user's preference w and non-user preferred question-answering results y l The score difference between them, σ is the Sigmoid function, and σ can refer to Formula 5.

[0076]

[0077] By using Formula 5, the score difference can be mapped to (0, 1). The larger the score difference, the closer the value of σ is to 1, and the corresponding -lohσ is closer to 0, that is, the smaller the error is. By averaging the sample distribution, the generalization of the model can be improved.

[0078] Furthermore, relative entropy (Kullback-Leibler, KL) can be used as the optimization target of reinforcement learning, and a simple classification error can be directly calculated, and the optimal strategy can be calculated in a closed form. The optimization target of reinforcement learning can be referred to, for example, in Formula 6.

[0079]

[0080] The optimal solution of Formula 6 may refer to Formula 7, for example.

[0081]

[0082] By transposing the terms in Formula 7, we can obtain Formula 8.

[0083]

[0084] Substituting formula 8 into the above formula 4, we can get formula 9.

[0085]

[0086] Among them, after training, the parameter θ is optimized to the optimal value, π θ is the optimal strategy that can maximize the long-term cumulative reward, π ref is the reference policy parameter, and β is used to constrain π during training. θ Don't stray too far from π ref Parameters of ref It is an initialization strategy to guide strategy learning. This strategy serves as a benchmark or starting point for training to better adapt to the preferred data and reduce the difference between training data and real data.w ,y l For the relevant explanations of , D and σ, please refer to the relevant explanations of the aforementioned Formulas 1, 3 and 5.

[0087] The discriminator trained by the direct preference optimization algorithm can select N question-and-answer results that are more in line with user preferences from multiple question-and-answer results output by multiple large language models.

[0088] Optionally, the discriminator may select N question-and-answer results from the multiple question-and-answer results output by the multiple large language models in the following ways.

[0089] Method 1: The discriminator can calculate the negative log-likelihood loss of the multiple question-and-answer results based on the user's user preference information, and select the N question-and-answer results from the multiple question-and-answer results based on the negative log-likelihood loss of the multiple question-and-answer results. Among them, the negative log-likelihood loss can be used to indicate the similarity between the question-and-answer result and the question-and-answer result preferred by the user. The smaller the negative log-likelihood loss, the higher the similarity between the corresponding question-and-answer result and the question-and-answer result preferred by the user. The larger the negative log-likelihood loss, the lower the similarity between the corresponding question-and-answer result and the question-and-answer result preferred by the user. The discriminator can select the N question-and-answer results with the smallest negative log-likelihood loss from the multiple question-and-answer results. Among them, the way in which the discriminator calculates the negative log-likelihood loss can refer to Formula 4, for example.

[0090] Method 2: The discriminator can traverse the multiple question and answer results, compare the multiple question and answer results in pairs, select the question and answer result that best meets the user's preference from the two question and answer results based on the user's user preference information, and then re-traverse the remaining question and answer results, and select the question and answer result that best meets the user's preference from the remaining question and answer results based on the user's user preference information, until N question and answer results are selected. Taking the 12 question and answer results obtained by inferring the multiple large language models as an example, the discriminator can compare any two question and answer results (for example, question and answer result 1 and question and answer result 2), and select the question and answer result that meets the user's preference (for example, question and answer result 1) from question and answer result 1 and question and answer result 2 according to the user's user preference information, and then the discriminator can compare the question and answer result 1 with any of the remaining 10 question and answer results (for example, question and answer result 3), and select the question and answer result that meets the user's preference (for example, question and answer result 3) from question and answer result 1 and question and answer result 3 according to the user's user preference information, and then the discriminator can compare the question and answer result 3 with any of the remaining 9 question and answer results (for example, question and answer result 4), and select the question and answer result that meets the user's preference from question and answer result 3 and question and answer result 4 according to the user's user preference information, and finally select the question and answer result that best meets the user's preference (for example, question and answer result 3) among the 12 question and answer results. Repeat the above operation and select the question and answer result that best meets the user's preference (for example, question and answer result 1) from the remaining 11 question and answer results based on the user's user preference information.

[0091] Optionally, N is 2, which allows the user to select a question and answer result that better suits his or her preferences, while also improving the user's visual experience.

[0092] S204: Display the N question-and-answer results.

[0093] The terminal device 101 can display the N question and answer results on the display screen. When N is greater than 1, the user can select the question and answer result that best meets his or her preference (i.e., the target question and answer result) from the N question and answer results. Therefore, optionally, the terminal device 101 can respond to the user's operation of selecting the target question and answer result based on the N question and answer results, display the target question and answer result, and hide other question and answer results other than the target question and answer result in the N question and answer results. Taking the N question and answer results displayed as the aforementioned question and answer result 1 and question and answer result 3 as an example, if the target question and answer result selected by the user is question and answer result 1, the terminal device 101 can display the question and answer result 1 and hide the question and answer result 3. Optionally, before displaying the question and answer result 1, the terminal device 101 can also polish and supplement the question and answer result 1, and display the polished and supplemented question and answer result 1, which can improve the readability of the question and answer result 1 and help improve the user experience.

[0094] Optionally, after detecting that the user has selected a target question and answer result, the terminal device 101 may also add the target question and answer result to the first data set, and add the question and answer result that the user has not selected from the N question and answer results to the second data set, and the terminal device 101 may use the first data set and the second data set to optimize the discriminator. Among them, the question and answer results in the first data set are the question and answer results preferred by the user (the first data set is, for example, the aforementioned data set 1), and the data in the second data set are the question and answer results that are not preferred by the user (the second data set is, for example, the aforementioned data set 2). It can be understood that the question and answer results added to the second data set may also include, for example, the question and answer results screened out by the discriminator, that is, other question and answer results from the multiple question and answer results excluding the N question and answer results. The embodiment of the present application does not limit the question and answer results added to the second data set.

[0095] Optionally, when the amount of data in the first data set and / or the second data set (i.e., the number of question and answer results) is greater than a preset data amount, the terminal device 101 may optimize the discriminator. For example, when the number of question and answer results in the first data set reaches 5000, the terminal device 101 may optimize the discriminator. The process of optimizing the discriminator may refer to the process described in S1 to S3 above and will not be repeated here.

[0096] The above introduces some possible implementation methods of the data processing method provided in the embodiments of the present application. In order to better understand the above scheme, the method will be introduced as a whole through an example below.

[0097] Please refer to Figure 3 , is a schematic diagram of the structure of an intelligent question-answering system provided in an embodiment of the present application, such as Figure 3 As shown in the figure, the intelligent question answering system includes question acquisition and expansion agent, model reasoning agent, discriminator agent, user decision agent, discriminator optimization agent and answer generation agent. The multiple agents can cooperate with each other through API calls to achieve the ultimate goal.

[0098] The question acquisition and expansion agent is used to obtain the question input by the user and expand the question input by the user according to the preset prompt words. For example, the question input by the user is "How to make scrambled eggs with tomatoes in 20 words?" The question acquisition and expansion agent can expand the question into "I want to make scrambled eggs with tomatoes, teach me, within 20 words", "Tell me the strategy of scrambled eggs with tomatoes in only 20 words" and "How to make a delicious scrambled eggs with tomatoes, no more than 20 words", and get 4 questions. After the question acquisition and expansion agent expands the question input by the user to get 4 questions, it can call the interface of the model reasoning agent and send the 4 questions to the model reasoning agent.

[0099] The model reasoning agent includes multiple large language models (such as the aforementioned Qwen1.5-chat, BioBert, and ERNIE-Code). The model reasoning agent can input the four questions from the question acquisition and expansion agent into Qwen1.5-chat, BioBert, and ERNIE-Code for reasoning, and obtain 12 question-answering results. The model reasoning agent can call the interface of the discriminator agent and send the 12 question-answering results to the discriminator agent.

[0100] The discriminator agent is used to select N (taking N=2 as an example) question-answering results from the 12 question-answering results. Taking the method 1 in S203 as an example, the discriminator agent can calculate the negative log-likelihood loss of each question-answering result in the 12 question-answering results, and select the two question-answering results with the smallest negative log-likelihood loss. The selected two question-answering results are, for example:

[0101] Question and answer result A: Scramble eggs until fluffy and serve; stir-fry tomatoes and seasonings, then add eggs and stir well;

[0102] Question and answer result B: 1. Dice tomatoes and beat eggs. 2. Heat the pan and fry eggs, then serve. 3. Fry tomatoes and add salt and sugar. 4. Add eggs and stir fry evenly.

[0103] The discriminator agent can call the interface of the user decision agent and send the two question-answer results to the user decision agent.

[0104] The user decision agent is used to display the two question results for the user to select, and can also be used to respond to the user's operation of selecting the target question and answer result, and call the interface of the answer generation agent to send the target question and answer result to the answer generation agent. Taking the target question and answer result selected by the user as question and answer result B as an example, the user decision agent can call the interface of the answer generation agent to send the question and answer result B to the answer generation agent. In addition, the user decision agent can also call the interface of the discriminator optimization agent to send question and answer results A and B to the discriminator optimization agent.

[0105] The answer generation agent is used to polish and supplement the question and answer results (i.e., question and answer results B) from the user decision agent, and display the polished and supplemented question and answer results B. The polished and supplemented question and answer results B are, for example, "1. Dice the tomatoes and beat the eggs. 2. Fry the eggs in a hot pan and set aside. 3. Fry the tomatoes and season with salt and sugar. 4. Add the eggs and stir well."

[0106] The discriminator optimization agent is used to add the question and answer result B to the first data set, and add the question and answer result A to the second data set, and optimize the discriminator when the number of question and answer results in the first data set or the second data set is greater than a preset data amount.

[0107] Based on the same inventive concept, an embodiment of the present application also provides a data processing device.

[0108] See also Figure 4 , is a structural schematic diagram of a data processing device provided in an embodiment of the present application. The data processing device may be the above-mentioned terminal device or server, or a chip or integrated circuit in the terminal device or server, etc. The device includes modules / units / technical means for executing the method performed by the terminal device 101 in the above-mentioned method embodiment.

[0109] Exemplarily, the data processing device includes:

[0110] The acquisition module 401 is used to acquire the question input by the user;

[0111] Processing module 402, used to input the question into multiple large language models, obtain multiple question-answering results output by the multiple large language models, where different large language models generate question-answering results with different description angles for the same question; select N question-answering results from the multiple question-answering results according to the user preference information of the user by a discriminator, where the discriminator is trained based on the question-answering results selected according to the questions historically input by the user, and N is a positive integer;

[0112] The display module 403 is used to display the N question and answer results.

[0113] In a possible implementation, the processing module 402 is specifically used to: expand the question to obtain M questions, where M is a positive integer; input the M questions into each of the multiple large language models respectively to obtain multiple question-answering results output by the multiple large language models.

[0114] In a possible implementation, the processing module 402 is specifically used to: determine the negative log-likelihood loss of the multiple question and answer results according to the user preference information of the user through the discriminator; and select N question and answer results with the smallest negative log-likelihood loss from the multiple question and answer results through the discriminator.

[0115] In a possible implementation manner, N is 2.

[0116] In a possible implementation, when N is greater than 1, the processing module 402 is further used to: respond to the user's operation of selecting a target question and answer result based on the N question and answer results; the display module is further used to: display the target question and answer result and hide other question and answer results, where the other question and answer results are the question and answer results other than the target question and answer result among the N question and answer results.

[0117] In a possible implementation, the processing module 402 is further used to: add the target question and answer result to a first data set, and add other question and answer results among the N question and answer results except the target question and answer result to a second data set, and the first data set and the second data set are used to optimize the discriminator.

[0118] In a possible implementation, the processing module 402 is specifically used to: when it is determined that the amount of data in the first data set or the second data set is greater than a preset data amount, optimize the discriminator based on a direct preference optimization algorithm, and the data sets used to optimize the discriminator are the first data set and the second data set.

[0119] As an example, Figure 4 The data processing device can be used to execute Figure 2Therefore, for the functions that can be realized by each functional module of the data processing device, reference can be made to Figure 2 The description of the illustrated embodiment will not be repeated here.

[0120] It should be noted that, although several modules or submodules of the device are mentioned in the above detailed description, this division is merely exemplary and not mandatory. In fact, according to an embodiment of the present invention, the features and functions of two or more units described above can be embodied in one module. Conversely, the features and functions of one module described above can be further divided into multiple modules to be embodied.

[0121] As a possible product form of the above data processing device, see Figure 5 The embodiment of the present application further provides an electronic device 500, including:

[0122] At least one processor 501; and a communication interface 503 that is communicatively connected to the at least one processor 501; the at least one processor 501 executes instructions stored in the memory 502, so that the electronic device 500 executes the method steps performed by any device in the above method embodiment through the communication interface 503.

[0123] Optionally, the memory 502 is located outside the electronic device 500 .

[0124] Optionally, the electronic device 500 includes the memory 502, the memory 502 is connected to the at least one processor 501, and the memory 502 stores instructions that can be executed by the at least one processor 501. Figure 5 The dashed lines indicate that the memory 502 is optional for the electronic device 500 .

[0125] The processor 501 and the memory 502 may be coupled via an interface circuit or may be integrated together, which is not limited here.

[0126] The specific connection medium between the processor 501, the memory 502 and the communication interface 503 is not limited in the embodiment of the present application. Figure 5 In the embodiment, the processor 501, the memory 502 and the communication interface 503 are connected via a bus 504. Figure 5 The connections between the other components are shown in bold lines, which are only for illustration and are not intended to be limiting. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 5In the example, only one thick line is used to represent the bus, but it does not mean that there is only one bus or one type of bus. It should be understood that the processor mentioned in the embodiment of the present application can be implemented by hardware or by software. When implemented by hardware, the processor can be a logic circuit, an integrated circuit, etc. When implemented by software, the processor can be a general-purpose processor, which is implemented by reading the software code stored in the memory.

[0127] Exemplarily, the processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or the processor may be any conventional processor, etc.

[0128] It should be understood that the memory mentioned in the embodiments of the present application may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM) and direct memory bus random access memory (DirectRambus RAM, DR RAM).

[0129] It should be noted that when the processor is a general-purpose processor, DSP, ASIC, FPGA or other programmable logic device, discrete gate or transistor logic device, discrete hardware component, the memory (storage module) can be integrated into the processor.

[0130] It should be noted that the memory described herein is intended to include, but is not limited to, these and any other suitable types of memory.

[0131] As another possible product form, an embodiment of the present application also provides a computer-readable storage medium, which is used to store instructions. When the instructions are executed, the computer executes the method steps executed by any device in the above method example.

[0132] As another possible product form, an embodiment of the present application also provides a computer program product including instructions, wherein the computer program product stores instructions that, when executed on a computer, enable the computer to execute the method steps executed by any device in the above method embodiment.

[0133] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0134] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0135] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0136] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0137] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

Claims

1. A data processing method, characterized in that: The method comprises: Get user input questions; Inputting the question into multiple large language models to obtain multiple question-answering results output by the multiple large language models, wherein different large language models generate question-answering results from different description angles for the same question; Selecting N question-and-answer results from the multiple question-and-answer results according to the user preference information of the user by a discriminator, wherein the discriminator is trained based on the question-and-answer results selected according to the questions historically input by the user, and N is a positive integer; The N question and answer results are displayed.

2. The method according to claim 1, characterized in that Inputting the question into a plurality of large language models, and obtaining a plurality of question-answering results output by the plurality of large language models, including: Expand the problem to obtain M problems, where M is a positive integer; The M questions are respectively input into each of the multiple large language models to obtain multiple question-answering results output by the multiple large language models.

3. The method according to claim 1, characterized in that The discriminator selects N question-and-answer results from the plurality of question-and-answer results according to the user preference information of the user, including: The discriminator determines the negative log-likelihood loss of the plurality of question-answering results according to the user preference information of the user; Select N question-answering results with the smallest negative log-likelihood loss from the multiple question-answering results.

4. The method according to claim 3, characterized in that The N is 2.

5. The method according to claim 1, characterized in that When N is greater than 1, the method further includes: Responding to the user's operation of selecting a target question and answer result based on the N question and answer results; The target question and answer result is displayed, and other question and answer results are hidden, where the other question and answer results are question and answer results other than the target question and answer result among the N question and answer results.

6. The method according to claim 5, characterized in that The method further comprises: The target question and answer result is added to a first data set, and other question and answer results among the N question and answer results except the target question and answer result are added to a second data set, and the first data set and the second data set are used to optimize the discriminator.

7. The method according to claim 6, characterized in that The method further comprises: When it is determined that the amount of data in the first data set and / or the second data set is greater than the preset data amount, the discriminator is optimized based on a direct preference optimization algorithm, and the data sets used to optimize the discriminator are the first data set and the second data set.

8. A data processing device, characterized in that: include: The acquisition module is used to obtain the questions input by the user; A processing module, used for inputting the question into a plurality of large language models to obtain a plurality of question-answering results output by the plurality of large language models, wherein different large language models generate question-answering results from different description angles for the same question; Selecting N question-and-answer results from the multiple question-and-answer results according to the user preference information of the user by a discriminator, wherein the discriminator is trained based on the question-and-answer results selected according to the questions historically input by the user, and N is a positive integer; A display module is used to display the N question and answer results.

9. An electronic device, characterized in that: include: at least one processor; and a memory communicatively coupled to the at least one processor; The memory stores instructions that can be executed by the at least one processor, and the at least one processor executes the method as described in any one of claims 1 to 7 by executing the instructions stored in the memory.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium is used to store instructions, and when the instructions are executed, the method according to any one of claims 1 to 7 is implemented.

Images