Language model system, operation method thereof and real-time optimization method thereof

Abstract

A language model system and its operation and real-time optimization method are disclosed. A language model and a retrieval-augmented generation (RAG) unit receive a prompt. The RAG unit retrieves an external dataset based on the prompt to obtain an embedded message. The language model generates a response message based on the prompt and the embedded message, and displays the response message in a display unit. The display unit displays a questionnaire. A feedback analysis unit obtains feedback content and an evaluation score from the questionnaire. The feedback analysis unit determines whether the feedback content or evaluation score is reliable. If the feedback content or evaluation score is reliable, the feedback analysis unit updates the external dataset based on the feedback content or evaluation score.

Claims

1. A method for operating a language model system, comprising: A language model and a retrieval-augmented generation (RAG) unit receive a prompt; the RAG unit searches multiple external datasets based on the prompt to obtain an embedded message; the language model generates a response message based on the prompt and the embedded message, and displays the response message in a display unit; the display unit displays a questionnaire; a feedback analysis unit obtains feedback content and an evaluation score from the questionnaire; the feedback analysis unit determines whether the feedback content or the evaluation score is referential; and if the feedback content or the evaluation score is referential, the feedback analysis unit selects a candidate updated external dataset from the external datasets based on the feedback content or the evaluation score, and updates the candidate updated external dataset.

2. The method of operating a language model system as described in claim 1, wherein the retrieval enhancement generation unit automatically selects one of the external datasets based on a category.

3. The method of operating the language model system as described in Request 1, wherein the feedback analysis unit analyzes whether the feedback content is referable based on the correlation between the feedback content and the prompt word.

4. The method of operating the language model system as described in Request 1, wherein the feedback analysis unit analyzes whether the feedback content is referable based on whether there is a conflict between the feedback content and the evaluation score.

5. The method of operating a language model system as described in claim 1, wherein the feedback analysis unit selects the candidate to update the external dataset based on a category.

6. As described in Request 1, the language model system operates in a manner that the feedback analysis unit performs textual and grammatical corrections on the feedback content.

7. A language model system, comprising: A language model; a retrieval-augmented generation (RAG) unit, the language model and the RAG unit being used to receive a prompt; a display unit, the RAG unit being used to retrieve multiple external datasets based on the prompt to obtain an embedded message, the language model being used to generate a response message based on the prompt and the embedded message, and the display unit being used to display the response message and a questionnaire; And a feedback analysis unit, used to obtain feedback content and an evaluation score from the questionnaire, and to determine whether the feedback content or the evaluation score is referential. If the feedback content or the evaluation score is referential, the feedback analysis unit selects a candidate external dataset from the external datasets based on the feedback content or the evaluation score, and updates the candidate external dataset.

8. The language model system as described in claim 7, wherein the retrieval enhancement generation unit automatically selects the external datasets based on a category.

9. The language model system as described in claim 7, wherein the feedback analysis unit analyzes whether the feedback content is referential based on the correlation between the feedback content and the prompt word.

10. The language model system as described in claim 7, wherein the feedback analysis unit analyzes whether the feedback content is reliable based on whether there is a conflict between the feedback content and the evaluation score.

11. The language model system as described in claim 7, wherein the feedback analysis unit selects the candidate updated external dataset based on a category.

12. The language model system as described in Request 7, wherein the feedback analysis unit performs textual and grammatical corrections on the feedback content.

13. An on-the-spot optimization method for a language model system, comprising: A display unit displays a questionnaire, which is a response message for a language model; a feedback analysis unit obtains feedback content and an evaluation score from the questionnaire; the feedback analysis unit determines whether the feedback content or the evaluation score is referential; and if the feedback content or the evaluation score is referential, the feedback analysis unit selects a candidate updated external dataset from a plurality of external datasets used by a Retrieval-Augmented Generation (RAG) unit based on the feedback content or the evaluation score, and updates the candidate updated external dataset.

Images