Answer feedback method to be applied to large-scale language model, device, electronic apparatus, storage medium and computer program

Abstract

SOLUTION: An answer feedback method to be applied to a large-scale language model includes the steps of: receiving a question inputted by a user; generating a candidate answer set for a question, selecting an answer from the candidate answer set as a target answer and displaying the target answer to the user by a pre-training large-scale language model; and generating a feedback page in accordance with the reception of a feedback request transmitted to the target answered by the user, and displaying the feedback page to the user. A content of the feedback page includes the candidate answer set. Also, the method includes a step of specifying an answer indicated by an update request in the candidate answer set as a new target answer, and displaying the new target answer to the user in accordance with the reception of an update request transmitted to the feedback page by the user.EFFECT: An explicit correct answer given from a user can be simply acquired, and the accuracy of an answer feedback result can be improved.SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] The embodiments of the present disclosure relate to the field of data processing, and more particularly, to the field of artificial intelligence techniques such as generative models and intelligent question and answering, and can be applied to scenarios in which answers generated by large-scale language models are fed back. [Background technology]

[0002] Large language models (LLMs) are essentially generative models that can be applied to various downstream tasks, such as ChatGPT (Chat Generative Pre-trained Transformer, a chatbot program developed by OpenAI), such as intelligent question answering, event analysis, text generation, and intelligent translation. In these generative applications of large language models, feedback of the generation results plays an important role in the growth of the large language model. More and better answer feedback can help the large language model to train better, thereby providing better service to users, forming an efficient and benign data flywheel.

[0003] Currently, there are three main answer feedback methods commonly used in large-scale language model generation applications: (1) No feedback of any kind. In this method, users can only passively receive answers generated by large-scale language models, which is a poor experience. (2) Simple positive and negative feedback such as "like" and "dislike" is provided. This method only allows users to know whether the answer is good or bad through feedback, and has limited effect on retraining the large-scale language model. (3) When a user indicates that the answer is not good by selecting "dislike", the user is asked to enter the specific reason through a pop-up window, etc. This method makes it difficult to control the quality of the content entered by the user, and users cannot directly obtain a satisfactory answer. Summary of the Invention

[0004] The embodiments of the present disclosure provide an answer feedback method and device, an electronic device, a storage medium, and a computer program that are applied to a large-scale language model.

[0005] In a first aspect, an embodiment of the present disclosure provides an answer feedback method applied to a large-scale language model, including the steps of receiving a question entered by a user; generating a set of candidate answers to the question using a pre-trained large-scale language model, selecting an answer from the candidate answer set as a target answer, and displaying the target answer to the user; in response to receiving a feedback request sent by the user for the target answer, generating a feedback page and displaying the feedback page to the user, wherein the content of the feedback page includes the candidate answer set; and in response to receiving an update request sent by the user to the feedback page, identifying an answer in the candidate answer set indicated by the update request as a new target answer, and displaying the new target answer to the user.

[0006] In a second aspect, an embodiment of the present disclosure provides an answer feedback device applicable to a large-scale language model, comprising: a receiving module configured to receive a question input by a user; a display module configured to generate a set of candidate answers to the question using a pre-trained large-scale language model, select an answer as a target answer from the candidate answer set, and display the target answer to the user; and a feedback module configured to generate a feedback page and display the feedback page to the user in response to receiving a feedback request sent by the user for the target answer, wherein the content of the feedback page includes the candidate answer set, and the display module is further configured to identify an answer indicated by the update request in the candidate answer set as a new target answer in response to receiving an update request sent by the user for the feedback page, and display the new target answer to the user.

[0007] In a third aspect, an embodiment of the present disclosure provides an electronic device comprising at least one processor and a memory communicatively connected to the at least one processor, the memory storing instructions executable by the at least one processor, the instructions, when executed by the at least one processor, causing the at least one processor to perform an answer feedback method for application to a large scale language model as described in the first aspect.

[0008] In a fourth aspect, an embodiment of the present disclosure provides a non-transitory computer readable storage medium having stored thereon computer instructions, the computer instructions being used to cause a computer to perform an answer feedback method for applying to a large scale language model as described in the first aspect.

[0009] In a fifth aspect, an embodiment of the present disclosure provides a computer program which, when executed by a processor, performs an answer feedback method for applying to a large-scale language model according to the first aspect.

[0010] The answer feedback method applied to a large-scale language model provided in an embodiment of the present disclosure generates a candidate answer set for a user's question and selects and displays a target answer from the candidate answer set, receives a feedback request from the user for the answer, and displays all answers in the candidate answer set to the user, allowing the user to select and update a new answer from all answers generated by the large-scale language model for the question, thereby easily obtaining a clear correct answer from the user and improving the accuracy of the answer feedback results.

[0011] It should be noted that the contents described in the summary of the invention are not intended to limit the key or important features of the embodiments of the present disclosure, and are not intended to limit the scope of the present disclosure. Other features of the present disclosure will become easier to understand from the following description. [Brief description of the drawings]

[0012] Other features, objects and advantages of the present disclosure will become more apparent from a reading of the following detailed description of non-limiting embodiments, taken in conjunction with the drawings in which: [Figure 1] FIG. 1 illustrates an exemplary system architecture to which the present disclosure can be applied. [Diagram 2] 1 is a flow chart of one embodiment of an answer feedback method for application to a large-scale language model according to the present disclosure. [Figure 3A] FIG. 1 is a schematic diagram illustrating an application scenario of an answer feedback method for a large-scale language model according to an embodiment of the present disclosure. [Figure 3B] FIG. 1 is a schematic diagram illustrating an application scenario of an answer feedback method for a large-scale language model according to an embodiment of the present disclosure. [Figure 3C]FIG. 1 is a schematic diagram illustrating an application scenario of an answer feedback method for a large-scale language model according to an embodiment of the present disclosure. [Figure 4A] FIG. 10 is a schematic diagram illustrating another application scenario of the answer feedback method applied to a large-scale language model according to an embodiment of the present disclosure. [Figure 4B] FIG. 10 is a schematic diagram illustrating another application scenario of the answer feedback method applied to a large-scale language model according to an embodiment of the present disclosure. [Figure 4C] FIG. 10 is a schematic diagram illustrating another application scenario of the answer feedback method applied to a large-scale language model according to an embodiment of the present disclosure. [Diagram 5] 4 is a flow chart of a further embodiment of an answer feedback method for application to a large-scale language model according to the present disclosure; [Figure 6A] FIG. 1 is a schematic diagram of a closed loop for obtaining answer feedback and training based on answer feedback. [Figure 6B] FIG. 1 is a schematic diagram of a closed loop for obtaining answer feedback and training based on answer feedback. [Figure 6C] FIG. 1 is a schematic diagram of a closed loop for obtaining answer feedback and training based on answer feedback. [Figure 7] FIG. 1 is a structural schematic diagram illustrating an embodiment of an answer feedback device applied to a large-scale language model according to the present disclosure; [Figure 8] FIG. 1 is a block diagram of an electronic device for implementing an answer feedback method for applying to a large-scale language model according to an embodiment of the present disclosure. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0013] The following describes exemplary embodiments of the present disclosure with reference to the drawings, and various details of the embodiments of the present disclosure are described to facilitate understanding, but it should be understood that these are merely illustrative. Therefore, it should be understood that those skilled in the art can make various changes and modifications to the embodiments of the present disclosure without departing from the scope and spirit of the present disclosure. Note that in the following description, for clarity and simplicity, the description of known functions and structures is omitted.

[0014] In addition, the embodiments and features of the embodiments of the present disclosure can be combined with each other as long as no contradiction occurs. Hereinafter, the present disclosure will be described in detail with reference to the drawings and embodiments.

[0015] FIG. 1 illustrates an exemplary system architecture 100 to which an answer feedback method for a large-scale language model or an answer feedback device for a large-scale language model according to the present disclosure can be applied.

[0016] 1, system architecture 100 may include terminal devices 101, 102, 103, a network 104, and a server 105. Network 104 is used as a medium to provide a communications link between terminal devices 101, 102, 103 and server 105. Network 104 may include various types of connections, such as wires, wireless communications links, or fiber optic cables.

[0017] To send and receive messages, users can use terminal devices 101, 102, 103 to communicate with server 105 via network 104. Applications for realizing information communication between terminal devices 101, 102, 103 and server 105 can be installed, such as a search application, a model training application, and the like.

[0018] The server 105 can provide various services to the terminal devices 101, 102, and 103. For example, the server 105 may receive a question input by a user of the terminal devices 101, 102, and 103, generate a candidate answer set for the question using a pre-trained large-scale language model, and select and display an answer from the candidate answer set as a target answer, or may receive a feedback request sent by the user for the target answer, generate a feedback page including the candidate answer set, and when an update request sent by the user based on the feedback page is received, identify and display the answer indicated by the update request in the candidate answer set as a new target answer.

[0019] The terminal devices 101, 102, 103 may be hardware or software. If the terminal devices 101, 102, 103 are hardware, they may be various electronic devices equipped with a display screen, including but not limited to smartphones, tablet computers, laptop computers, desktop computers, etc. If the terminal devices 101, 102, 103 are software, they may be installed in the electronic devices listed above, or may be realized as multiple software or software modules, or may be realized as a single software or software module, but are not limited thereto.

[0020] The server 105 may be hardware or software. If the server 105 is hardware, it may be realized as a distributed server cluster consisting of multiple servers, or as a single server. If the server is software, it may be realized as multiple software programs or software modules, or as a single software program or software module. No specific limitations are provided here.

[0021] It should be understood that the numbers of terminal devices, networks and servers in Figure 1 are merely exemplary, and the numbers of terminal devices, networks and servers may be arbitrarily increased or decreased according to the needs of implementation.

[0022] 2, which illustrates a flow 200 of an embodiment of an answer feedback method for a large-scale language model according to the present disclosure. The answer feedback method for a large-scale language model includes the following steps:

[0023] In step 201, a question entered by a user is received.

[0024] In this embodiment, a user can input a question to an execution entity (eg, server 105 shown in FIG. 1) via a terminal to be used (eg, terminal devices 101, 102, 103 shown in FIG. 1).

[0025] The question entered by the user may be various questions that the user wants to get the answer for according to the actual needs. For example, the question may be "how fast is the pulse rate?". As another example, the question may be to request the analysis result of a specific security event.

[0026] Depending on the application scenario, users may adopt various question input methods, such as text input, voice input, etc.

[0027] In step 202, a set of candidate answers to the question is generated by a pre-trained large-scale language model, an answer is selected from the candidate answer set as a target answer, and the target answer is displayed to the user.

[0028] In this embodiment, the large-scale language model can be obtained by pre-training. The large-scale language model can generate a set of candidate answers corresponding to an input question. The set of candidate answers may be composed of multiple answers.

[0029] Furthermore, selecting an answer as a target answer from the set of candidate answers from which the large-scale language model has been generated and displaying the target answer to the user can be accomplished in a variety of ways. For example, an answer can be randomly selected as the target answer.

[0030] Also, for example, the large-scale language model may generate a set of candidate answers and generate a matching degree corresponding to each answer in the set of candidate answers. The matching degree may represent the degree of compatibility between the answer and the question input by the user. In general, the matching degree may be represented by the probability of each answer generated by the large-scale language model. Then, the answer with the highest corresponding matching degree (e.g., the highest probability) may be selected as the target answer.

[0031] In general, the selected target answer may be displayed to the user on a display page, for example on a page that displays the question entered by the user.

[0032] In step 203, in response to receiving the feedback request sent by the user in response to the target answer, a feedback page is generated and the feedback page is displayed to the user.

[0033] In this embodiment, the user may send a feedback request corresponding to the target answer as necessary. Here, the feedback request may indicate that the user has a feedback request for the current target answer. Specifically, the user may send the feedback request in various ways according to the application scene and needs. For example, the user may send the feedback request by speaking a preset keyword or inputting a specified sentence such as "feedback".

[0034] When the actor receives a feedback request for a currently displayed target answer, the actor can generate and display a corresponding feedback page to the user, where the content on the feedback page can include a set of candidate answers generated by the large-scale language model for the question entered by the user.

[0035] In some cases, such as when the displayable content of a page is limited or when the answer content is long, it may not be necessary to display the entire content of the answer, and the feedback page may flexibly display keywords or the like corresponding to each answer in the candidate answer set.

[0036] In step 204, in response to receiving the update request sent by the user to the feedback page, the answer indicated by the update request in the candidate answer set is identified as a new target answer, and the new target answer is displayed to the user.

[0037] In this embodiment, the user may send an update request to the feedback page as necessary. Here, the update request may indicate that the user wants to replace the currently displayed answer with another answer in the candidate answer set. Specifically, the user may send the update request in various ways according to the application scene and needs. For example, the user may send the update request by speaking a preset keyword or inputting a specific sentence such as "update".

[0038] In general, when a user sends an update request, the user may specify an answer that satisfies his / her request from among a set of candidate answers displayed on the feedback page by page interaction or the like. In this case, when the execution entity receives the update request sent by the user to the feedback page, the execution entity may display the answer specified by the user to the user as a new target answer. For example, the new target answer is displayed below the previous target answer.

[0039] In the prior art, the user could only provide feedback on whether the answer was good or bad, or input the reason why the answer was considered bad. The answer feedback method of the present disclosure enables the user to select a satisfactory answer from among all candidate answers generated by a large-scale language model for the question entered by the user, when the user is not satisfied with the current answer, by displaying them to the user in an interface. Compared with the prior art, a feedback answer that is clear, accurate, and satisfactory to the user can be obtained, effectively improving the accuracy of user feedback.

[0040] In some optional embodiments of this embodiment, the feedback page content may further include a matching score corresponding to each answer in the set of candidate answers, the matching score representing the degree of fit between the answer generated by the large-scale language model and the user-entered question.

[0041] Optionally, the feedback page content may include reference information corresponding to each answer in the set of candidate answers, where the reference information typically represents information that the large-scale language model references or uses in generating an answer.

[0042] By displaying each answer on the feedback page, as well as the matching degree and reference information corresponding to each answer, it is possible to display the capabilities of the large-scale language model in an interface, allowing users to more accurately grasp information about the answers.

[0043] In some optional embodiments of this embodiment, a preset feedback identifier may be simultaneously displayed on the display page to which the target answer belongs, and the user may trigger the transmission of a feedback request by an interaction action with the feedback identifier.

[0044] Here, the feedback identifier may be designed in advance by a related engineer. For example, the feedback identifier may be a specified pattern, etc. In general, the page area where the feedback identifier is located may be designed as an interactive area, such as designing the feedback identifier as one button control in a page. The interaction action may be set in advance (for example, clicking, swiping in a predetermined direction, etc.). For example, the user may send a feedback request by clicking the feedback identifier.

[0045] Optionally, an update identifier may be displayed on the feedback page, and the user may then trigger the submission of an update request by interacting with the update identifier.

[0046] Here, the update identifier may be designed in advance by a related engineer. For example, the update identifier may be a designated text (e.g., "answer update"). In general, the page area where the update identifier is located may be designed as an interactive area, such as designing the update identifier as a button control in the page. The interaction action may be set in advance (e.g., clicking, swiping in a predetermined direction, etc.). For example, the user may send an update request by clicking the update identifier.

[0047] Further, a cancellation identifier may be displayed on the feedback page. At this time, in response to receiving a cancellation request sent from the user to the feedback page, the display page where the target answer is located is returned to. The user may trigger the sending of the cancellation request by an interaction operation with the feedback identifier.

[0048] The cancel identifier may be designed in advance by a related engineer. For example, the cancel identifier may be a specified text (e.g., "Back"). In general, the page area where the cancel identifier is located may be designed as an interactive area, such as designing the cancel identifier as a button control in a page. The interaction action may be set in advance (e.g., clicking, swiping in a predetermined direction, etc.). For example, the user may send a cancel request by clicking the cancel identifier.

[0049] According to the needs of the actual application, the display page to which the target answer belongs and the feedback page may further display various necessary information, for example, the display page to which the target answer belongs may display an identifier indicating copy, refresh, forward, etc., so as to facilitate the user to perform various operations such as copy, refresh, and forward the displayed target answer.

[0050] By displaying various interaction identifiers in an interface on the target answer display page and the feedback page for the target answer, it becomes easier for the user to provide feedback on the target answer, submit a new target answer, or cancel updates to the target answer as needed, thereby improving the user's operating experience.

[0051] Next, reference is made to FIGS. 3A-3C, which show an exemplary application scene of the answer feedback method applied to the large-scale language model according to the present embodiment. In FIG. 3A, a user may first input a question 301, "How fast is a pulse rate?". The large-scale language model may generate a set of candidate answers corresponding to the question and a probability corresponding to each answer among them, and select the answer with the highest probability as a target answer (e.g., reference numeral 302 in the figure) and display it to the user. At the same time, the display page has five action identifiers 303, which represent refresh, copy, like, dislike, and feedback, in that order. If the user is not satisfied with the current answer or wants to view more answers, he or she can click a feedback identifier among the action identifiers 303 to view a feedback page 304.

[0052] In FIG. 3B, on the left side of the feedback page 304, keywords 305 corresponding to each answer are displayed in order from the highest to the lowest corresponding probability. On the right side of the feedback page 304, the source and specific content 306 corresponding to the answer in the selected state on the left side are displayed, and at the same time, the keywords corresponding to the answer are highlighted. On the lower right corner of the feedback page 304, a "Cancel" button and a "Replace / Regenerate Results" button 307 are displayed. Here, the user can cancel the correction to the target answer and return to the display page shown in FIG. 3A by clicking the Cancel button.

[0053] In addition, the user can select a desired keyword (e.g., 1 to 1.7 times / second) from the keywords of each answer displayed on the left side of the feedback page 304, click the “Replace / Regenerate Results” button, and send an update request to the target answer 302.

[0054] At this time, as shown in Fig. 3C, on the display page shown in Fig. 3A, "Replace 1 to 1.7 times / second" 308 is automatically entered, and a corresponding answer 309 is displayed. That is, the corresponding answer is obtained by combining the answer keyword with a preset prompt.

[0055] Next, reference is made to Figs. 4A to 4C showing another exemplary application scene of the answer feedback method applied to the large-scale language model according to the present embodiment. A question input by a security expert may request an analysis of a specified security event. At this time, the large-scale language model may analyze this security event. As shown in page 401 shown in Fig. 4A, attacker analysis, message analysis (attack message, attack characteristics, attack means, attack purpose, etc.), victim analysis, association analysis, etc. are performed for this security event.

[0056] A security expert can view specific analysis results by clicking an analysis area in page 401. As shown in FIG. 4B, when a user clicks an area corresponding to an attack analysis, specific attack features 402 can be displayed to the security expert, and five action identifiers 403 indicating refresh, copy, like, dislike, and feedback can be displayed in sequence.

[0057] When the feedback identifier is clicked by the security expert, a feedback page 404 can be displayed to the security expert. On the left side of the feedback page 404, keywords 405 of each analysis result are displayed in order from the largest to the smallest corresponding probability. On the right side of the feedback page 404, code analysis related content 406 (e.g., original message and corresponding analyzed code, etc.) in the process of generating the analysis result are displayed. At the same time, a "Cancel" button and a "Replace / Regenerate Results" button 407 are displayed in the lower right corner of the feedback page 404. Here, the security expert can cancel the modifications to the analysis result by clicking the "Cancel" button and return to the display page shown in FIG. 4A.

[0058] Then, security experts can make analysis judgments based on their expertise and experience, select the correct analysis result keyword on the left side of the feedback page, and click the "Replace / Regenerate Result" button in the bottom right corner to generate new corresponding analysis results and display them for security experts to review.

[0059] In some optional embodiments of this embodiment, the large-scale language model may be trained by the following steps.

[0060] Step 1 is to obtain a pre-trained base model.

[0061] In this step, a single base model may be pre-trained, for example, by training the base model on a large amount of unlabeled data based on a specified model architecture.

[0062] In step 2, supervised fine-tuning training is performed on the base model to obtain a supervised fine-tuned model.

[0063] In this step, some high-quality question-and-answer data, such as prompt-response data pairs, can be used to perform supervised fine-tuning training on the base model to obtain a supervised fine-tuned model.

[0064] In step 3, we obtain a pre-trained reward model.

[0065] In this step, you can create a dataset by setting three different responses to the same prompt and marking the quality, and then use this dataset to train a reward model to learn which responses are best for similar prompts.

[0066] In step 4, a large-scale language model is obtained through reinforcement learning training based on the supervised fine-tuning model and the reward model.

[0067] In this step, a large-scale language model can be obtained by training using a supervised fine-tuning model and a reward model through reinforcement learning. For example, a supervised fine-tuning model is trained in advance to generate answers so that each answer can be distinguished from the good and bad, and the supervised fine-tuning model is adjusted according to the positive or negative of the reward function indicated by the reward model. By repeatedly training in this way, a large-scale language model can be obtained.

[0068] Further, reference is made to Fig. 5, which illustrates a flow 500 of a further embodiment of a method for answer feedback applied to a large-scale language model. The flow 500 of the method for answer feedback applied to a large-scale language model includes the following steps:

[0069] In step 501, a question entered by a user is received.

[0070] In step 502, a set of candidate answers to the question is generated using a pre-trained large-scale language model, an answer is selected from the candidate answer set as a target answer, and the target answer is displayed to the user.

[0071] In step 503, in response to receiving the feedback request sent by the user in response to the target answer, a feedback page is generated and the feedback page is displayed to the user.

[0072] In step 504, in response to receiving the update request sent by the user to the feedback page, the answer indicated by the update request is identified as a new target answer in the candidate answer set, and the new target answer is displayed to the user.

[0073] In step 505, the question and the new target answer are associated and stored as supplemental training data.

[0074] In this embodiment, after a new target answer is obtained, the question input by the user and the new target answer can be further associated with each other and stored as supplemental training data.

[0075] In step 506, the large-scale language model is updated and trained using the supplemental training data.

[0076] In this embodiment, the stored supplementary training data can be used to perform further adjustment, optimization training, etc. on the large-scale language model. Usually, after the amount of supplementary training data reaches a certain level, further adjustment and optimization training can be performed on the large-scale language model.

[0077] Next, refer to FIG. 6A-FIG. 6C, which shows an example of a closed loop that obtains answer feedback and trains based on the answer feedback. The front end can be used for various information display. The data transfer can be used for different modules to transmit data. The industry model can be a vertical class model of a specific field that can accurately interpret some technical terms, etc. in a specific field. The computational model performs calculations in combination with a large-scale language model to generate answers.

[0078] In FIG. 6A, a front-end module can receive a question entered by a user, a data transfer module can transfer the question and prompt to an industry model, the industry model can distribute the question and prompt to a large-scale language model and a computational model, and the large-scale language model and the computational model can combine to calculate a response.

[0079] Correspondingly, in FIG. 6B, the Response is first transferred to the industry model, and then transferred by the data transfer module to the front-end module for display. When the user gives feedback through the feedback inlet, as shown in FIG. 6C, the large-scale language model and the computational model can regenerate the Response' corresponding to the feedback, and sequentially transfer it to the front-end module through the industry model, the data transfer module, etc., for display to the user. At the same time, as shown in FIG. 6B, the large-scale language model and the computational model can self-train based on the user's feedback.

[0080] By recording the clear and correct answers actively fed back by users as training data, the large-scale language model can be continuously adjusted, optimized and modified during the user's usage process, thereby improving the training effect of the model.

[0081] Further referring to FIG. 7, as an embodiment of the method shown in each of the above figures, the present disclosure provides an embodiment of an answer feedback device applied to a large-scale language model, which embodiment of the device corresponds to the embodiment of the method shown in FIG. 2, and which can be specifically applied to various electronic devices.

[0082] As shown in FIG. 7, the answer feedback device 700 applied to the large-scale language model provided in this embodiment includes a receiving module 701, a display module 702, and a feedback module 703. Here, the receiving module 701 is configured to receive a question input by a user. The display module 702 is configured to generate a candidate answer set for the question by a pre-trained large-scale language model, select an answer as a target answer from the candidate answer set, and display the target answer to the user. The feedback module 703 is configured to generate a feedback page and display the feedback page to the user in response to receiving a feedback request sent by the user for the target answer. Here, the content of the feedback page includes the candidate answer set. The display module 702 is further configured to identify an answer indicated by the update request in the candidate answer set as a new target answer in response to receiving an update request sent by the user for the feedback page, and display the new target answer to the user.

[0083] In this embodiment, in the answer feedback device 700 applied to a large-scale language model, the specific processing of the receiving module 701, the display module 702, and the feedback module 703 and the technical effects achieved thereby can refer to the relevant descriptions of steps 201 to 204 in the corresponding embodiment of Figure 2, and the descriptions thereof will be omitted here.

[0084] In some optional embodiments of this embodiment, the content of the feedback page further includes a degree of matching of each answer in the set of candidate answers to the question, the degree of matching being generated by a large-scale language model.

[0085] In some optional embodiments of this embodiment, the content of the feedback page further includes reference information corresponding to the answers in the candidate answer set, and the large-scale language model uses the reference information to generate the candidate answer set.

[0086] In some optional embodiments of this embodiment, a preset feedback identifier is displayed on the display page to which the target answer belongs, and a feedback request is sent by an interaction action of the user according to the feedback identifier.

[0087] In some optional embodiments of this embodiment, the feedback page displays an update identifier and a cancel identifier. The update request is sent by an interaction action in response to the user's update identifier. The display module 702 is further configured to return to the display page where the target answer is located in response to receiving a cancel request for the feedback page sent by the user. The cancel request is sent by an interaction action in response to the user's cancel identifier.

[0088] In some optional embodiments of this embodiment, the large-scale language model is trained by the steps of obtaining a pre-trained base model, performing supervised fine-tuning training on the base model to obtain a supervised fine-tuned model, obtaining a pre-trained reward model, and obtaining the large-scale language model by reinforcement learning training based on the supervised fine-tuning model and the reward model.

[0089] In some optional embodiments of this embodiment, the answer feedback device applied to the above-mentioned large-scale language model further includes a storage module (not shown) configured to associate a question with a new target answer and store them as supplemental training data, and a training module (not shown) configured to update and train the large-scale language model using the supplemental training data.

[0090] According to an embodiment of the present disclosure, the present disclosure further provides an electronic device, a non-transitory computer-readable storage medium, and a computer program.

[0091] 8 shows a schematic block diagram of an exemplary electronic device 800 that can be used to implement embodiments of the present disclosure. The electronic device may represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processing, mobile phones, smart phones, wearable devices, and other similar computing devices. It should be noted that the components shown, their connections, and their functions are merely exemplary and are not intended to limit the embodiments of the present disclosure described and / or claimed herein.

[0092] 8, the electronic device 800 includes a computing unit 801 that can execute various suitable operations and processes according to a computer program stored in a read-only memory (ROM) 802 or loaded from a storage unit 808 into a random access memory (RAM) 803. The RAM 803 can further store various programs and data necessary for the operation of the device 800. The computing unit 801, the ROM 802 and the RAM 803 are connected to each other via a bus 804. An input / output (I / O) interface 805 is also connected to the bus 804.

[0093] In the electronic device 800, multiple components including an input unit 806 such as a keyboard, a mouse, etc., an output unit 807 such as various types of displays, speakers, etc., a storage unit 808 such as a magnetic disk, an optical disk, etc., and a communication unit 809 such as a network card, a modem, a wireless communication transceiver, etc. are connected to the I / O interface 805. The communication unit 809 enables the electronic device 800 to exchange information or data with other devices via a computer network such as the Internet and / or various telecommunication networks. The computing unit 801 may be various general-purpose and / or special-purpose processing components having processing and computing capabilities. Some examples of the computing unit 801 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units that execute machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 601 executes various methods and processes such as the answer feedback method applied to the large-scale language model described above. For example, in some embodiments, the answer feedback method for applying to a large-scale language model may be realized as a computer software program tangibly included in a machine-readable medium, such as the storage unit 808. In some embodiments, part or all of the computer program may be loaded and / or installed in the electronic device 800 via the ROM 802 and / or the communication unit 809. The computer program, when loaded into the RAM 803 and executed by the computing unit 801, may perform one or more steps of the answer feedback method for applying to a large-scale language model described above. Alternatively, in other embodiments, the computing unit 801 may be configured to perform the answer feedback method for applying to a large-scale language model in any other suitable manner (e.g., via firmware).

[0094] Various embodiments of the systems and techniques described herein may be realized in digital electronic circuitry systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. Each of these embodiments may include being implemented in one or more computer programs, which may be executed and / or interpreted in a programmable system including at least one programmable processor, which may be a special purpose or general purpose programmable processor, and which may receive data and instructions from, and transfer data and instructions to, a storage system, at least one input device, and at least one output device.

[0095] The program codes for implementing the methods of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general purpose computer, a special purpose computer, or other programmable data processing apparatus, and when executed by the processor or controller, the functions or operations defined in the flowcharts and / or block diagrams are performed. The program codes may be executed entirely on the device, partially on the device, partially on the device as a stand-alone software package and partially on a remote device, or entirely on a remote device or server.

[0096] In the context of this disclosure, a machine-readable medium may be a tangible medium, and may include or store a program for use by or in combination with an instruction execution system, device, or apparatus. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or apparatus, or any suitable combination thereof. More specific examples of machine-readable storage media may include electrical connections based on one or more cables, portable computer disks, hard disks, random access memories (RAMs), read-only memories (ROMs), erasable programmable read-only memories (EPROMs or flash memories), optical fibers, compact disk read-only memories (CD-ROMs), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0097] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer that includes a display device (e.g., a Cathode Ray Tube (CRT) or LCD (Liquid Crystal Display) monitor) for displaying information to a user, and a keyboard and pointing device (e.g., a mouse or trackball) through which a user can provide input to the computer. Other types of devices can also be used to provide interaction with a user. For example, feedback provided to the user can be any form of sensing feedback, e.g., visual feedback, auditory feedback, or haptic feedback, and input from the user can be received in any form, including audio input, speech input, or tactile input.

[0098] The systems and techniques described herein may be implemented in a computing system including a back-end component (e.g., a data server), or in a computing system including a middleware component (e.g., an application server), or in a computing system including a front-end component (e.g., a user computer having a graphical user interface or a web browser) through which a user may interact with the embodiments of the systems and techniques described herein, or in a computing system including any combination of such back-end, middleware, or front-end components. The components of the system may also be connected by any form or medium of digital data communication, such as a communications network. Examples of communications networks include a local area network (LAN), a wide area network (WAN), and the Internet.

[0099] The computer system may include a client and a server. The client and the server are usually remote from each other and communicate with each other via a communication network. The relationship between the client and the server is generated by running computer programs on each computer that have a client-server relationship with each other. The server may be a cloud server, a server of a distributed system, or a server combined with a blockchain.

[0100] It should be understood that the above-mentioned various forms of flow can be used to rearrange, add, or remove steps. For example, each step described in this disclosure can be performed in parallel, sequentially, or in a different order, as long as the desired results of the technical solution described in this disclosure can be achieved. This specification is not limited thereto.

[0101] The above specific embodiments do not limit the scope of protection of the present disclosure. It should be understood that those skilled in the art can make various modifications, combinations, subcombinations, and replacements according to design requirements and other factors. Any modifications, equivalent replacements, improvements, etc. made without departing from the spirit and principle of the present disclosure should be included within the scope of protection of the present disclosure.

Claims

1. receiving a user entered question; generating a set of candidate answers to the question using a pre-trained large-scale language model, selecting an answer from the set of candidate answers as a target answer, and displaying the target answer to the user; In response to receiving a feedback request submitted by the user for the target answer, generating a feedback page and displaying the feedback page to the user, the content of the feedback page including the set of candidate answers; and In response to receiving an update request sent by the user to the feedback page, identifying an answer in the set of candidate answers indicated by the update request as a new target answer and displaying the new target answer to the user; An answer feedback method for large-scale language models including

2. the feedback page content further includes a degree of matching of each answer of the set of candidate answers to the question, the degree of matching being generated by the large-scale language model; The answer feedback method for applying to a large-scale language model according to claim 1.

3. the feedback page content further includes reference information corresponding to answers in the set of candidate answers, and the large-scale language model uses the reference information to generate the set of candidate answers. The answer feedback method for applying to a large-scale language model according to claim 2.

4. A preset feedback identifier is displayed on the display page to which the target answer belongs, the feedback request is transmitted by an interaction action of the user in response to the feedback identifier. The answer feedback method for applying to a large-scale language model according to claim 3.

5. The feedback page displays a renewal identifier and a cancellation identifier, The update request is transmitted by an interaction action of the user in response to the update identifier; The answer feedback method applied to the large-scale language model includes: The method further includes the step of returning to a display page on which the target answer is located in response to receiving a cancellation request for the feedback page sent by the user; The cancellation request is transmitted by an interaction action of the user in response to the cancellation identifier. The answer feedback method for applying to a large-scale language model according to claim 3.

6. The large-scale language model is obtaining a pre-trained base model; performing supervised fine-tuning training on the base model to obtain a supervised fine-tuning model; obtaining a pre-trained reward model; obtaining a large-scale language model by reinforcement learning training based on the supervised fine-tuning model and the reward model; 2. The method of claim 1, wherein the large-scale language model is trained by:

7. storing the questions and new target answers as supplemental training data; updating and training the large-scale language model using the supplemental training data; The method for answer feedback applied to a large-scale language model according to claim 6, further comprising:

8. a receiving module configured to receive a question entered by a user; a display module configured to generate a set of candidate answers to the question using a pre-trained large-scale language model, select an answer from the set of candidate answers as a target answer, and display the target answer to the user; a feedback module configured to generate a feedback page and display the feedback page to the user in response to receiving a feedback request sent by the user for the target answer, the feedback module comprising: a feedback module configured to generate a feedback page and display the feedback page to the user ... in response to receiving a feedback request sent by the user for the target answer, the feedback module configured to generate a feedback page and display the feedback page to the user, the feedback module configured to generate a feedback page and display the feedback page The display module is further configured to, in response to receiving an update request sent by the user to the feedback page, identify an answer in the set of candidate answers indicated by the update request as a new target answer, and display the new target answer to the user. An answer feedback device for large-scale language models.

9. the feedback page content further includes a degree of matching of each answer of the set of candidate answers to the question, the degree of matching being generated by the large-scale language model; An answer feedback device for use with a large-scale language model according to claim 8.

10. the feedback page content further includes reference information corresponding to answers in the set of candidate answers, and the large-scale language model uses the reference information to generate the set of candidate answers. An answer feedback device for use with a large-scale language model according to claim 9.

11. A preset feedback identifier is displayed on the display page to which the target answer belongs, the feedback request is transmitted by an interaction action of the user in response to the feedback identifier. An answer feedback device for use with a large-scale language model according to claim 10.

12. The feedback page displays a renewal identifier and a cancellation identifier, The update request is transmitted by an interaction action of the user in response to the update identifier; The display module includes: The method is further configured to, in response to receiving a cancellation request for the feedback page sent by the user, return to a display page on which the target answer is located; The cancellation request is transmitted by an interaction action of the user in response to the cancellation identifier. An answer feedback device for use with a large-scale language model according to claim 10.

13. The large-scale language model is obtaining a pre-trained base model; performing supervised fine-tuning training on the base model to obtain a supervised fine-tuning model; obtaining a pre-trained reward model; obtaining a large-scale language model by reinforcement learning training based on the supervised fine-tuning model and the reward model; An answer feedback device for applying to a large-scale language model according to any one of claims 8 to 12, which is trained by:

14. a storage module configured to store the questions in association with new target answers as supplemental training data; a training module configured to update and train the large-scale language model using the supplemental training data; The answer feedback apparatus for applying to a large scale language model according to claim 13, further comprising:

15. At least one processor; and a memory communicatively coupled to the at least one processor, The memory stores instructions executable by the at least one processor, the instructions, when executed by the at least one processor, causing the at least one processor to perform a response feedback method for applying to a large-scale language model according to any one of claims 1 to 7. electronic equipment.

16. A non-transitory computer-readable storage medium having computer instructions stored thereon, comprising: The computer instructions are used to cause a computer to carry out the method according to any one of claims 1 to 7, A non-transitory computer-readable storage medium.

17. When executed by a processor, a method for answer feedback for application to a large-scale language model according to any one of claims 1 to 7 is implemented. Computer program.

Images