Method and device for evaluating model generation quality, equipment and storage medium

Abstract

The embodiment of the invention provides a method and device for evaluating model generation quality, equipment and a storage medium. The method comprises the following steps: acquiring a response generated for user query by utilizing a machine learning model and context information associated with the generation of the response; based on the context information, information defects of the response are detected on at least one dimension to obtain at least one detection result on the at least one dimension, and each detection result at least indicates the defect degree of the response on the corresponding dimension; and based on the at least one detection result, determining an evaluation result for the response, the evaluation result indicating information generation quality of the machine learning model in at least one dimension.

Description

Technical Field

[0001] Example embodiments of the present disclosure generally relate to the field of computers, and particularly to methods, apparatuses, devices, computer-readable storage media, and computer-executable instruction products for evaluating the quality of generated models. Background Art

[0002] With the development of machine learning technology, using machine learning models to generate content has been widely applied in various fields. For example, machine learning models have been widely applied in natural language processing, human-computer dialogue, machine translation, speech synthesis, image generation, and other fields. In this case, it is very necessary to accurately evaluate the quality of model generation. Summary of the Invention

[0003] In a first aspect of the present disclosure, a method for evaluating the quality of generated models is provided. The method includes: obtaining a response generated by a machine learning model for a user query and context information associated with the generation of the response; based on the context information, detecting information defects in the response in at least one dimension to obtain at least one detection result in at least one dimension, each detection result indicating at least the degree of defect of the response in the corresponding dimension; and based on at least one detection result, determining an evaluation result for the response, the evaluation result indicating the information generation quality of the machine learning model in at least one dimension.

[0004] In a second aspect of the present disclosure, an apparatus for evaluating the quality of generated models is provided. The apparatus includes: an obtaining module configured to obtain a response generated by a machine learning model for a user query and context information associated with the generation of the response; a detecting module configured to detect information defects in the response in at least one dimension based on the context information to obtain at least one detection result in at least one dimension, each detection result indicating at least the degree of defect of the response in the corresponding dimension; and an evaluating module configured to determine an evaluation result for the response based on at least one detection result, the evaluation result indicating the information generation quality of the machine learning model in at least one dimension.

[0005] In a third aspect of the present disclosure, an electronic device is provided. The device includes at least one processor; and at least one memory, the at least one memory being coupled to the at least one processor and storing instructions for execution by the at least one processor. The instructions, when executed by the at least one processor, cause the device to perform the method of the first aspect.

[0006] In a fourth aspect of the present disclosure, a computer-readable storage medium is provided. Computer-executable instructions are stored on the computer-readable storage medium, and the computer-executable instructions can be executed by a processor to implement the method of the first aspect.

[0007] In a fifth aspect of the present disclosure, there is provided a computer program product including computer-executable instructions, where when the computer-executable instructions are executed by a processor, a method according to the first aspect of the present disclosure is implemented.

[0008] It should be understood that the content described in this content part is not intended to define the key features or important features of the embodiments of the present disclosure, nor is it used to limit the scope of the present disclosure. Other features of the present disclosure will become easily understandable through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] In combination with the accompanying drawings and with reference to the following detailed description, the above and other features, advantages, and aspects of the embodiments of the present disclosure will become more apparent. In the drawings, the same or similar reference numerals denote the same or similar elements, where:

[0010] Figure 1 A schematic diagram showing an example environment in which embodiments of the present disclosure can be implemented is shown;

[0011] Figure 2 A flowchart showing a process for generating the quality of an evaluation model according to some embodiments of the present disclosure is shown;

[0012] Figure 3 A flowchart showing an example process for generating the quality of an evaluation model according to some embodiments of the present disclosure is shown;

[0013] Figure 4 A flowchart showing an example process for generating the quality of an evaluation model according to some other embodiments of the present disclosure is shown;

[0014] Figure 5 A schematic structural block diagram showing an example device for evaluating the quality of an evaluation model according to some embodiments of the present disclosure is shown; and

[0015] Figure 6 A block diagram of an electronic device capable of implementing multiple embodiments of the present disclosure is shown. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0016] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although some embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. On the contrary, these embodiments are provided to more thoroughly and completely understand the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not used to limit the protection scope of the present disclosure.

[0017] In the description of the embodiments of the present disclosure, the term "including" and its similar terms should be understood as open inclusion, that is, "including but not limited to". The term "based on" should be understood as "at least partially based on". The term "one embodiment" or "the embodiment" should be understood as "at least one embodiment". The term "some embodiments" should be understood as "at least some embodiments". There may also be other explicit and implicit definitions hereinafter.

[0018] In this document, unless otherwise specified, performing a step "in response to A" does not mean that the step is executed immediately after "A", but may include one or more intermediate steps.

[0019] It can be understood that the data involved in this technical solution (including but not limited to the data itself, the acquisition or use of the data) should comply with the requirements of the corresponding laws, regulations and related provisions.

[0020] It can be understood that before using the technical solutions disclosed in the embodiments of the present disclosure, the types, usage scopes, usage scenarios, etc. of the personal information involved in the present disclosure should be informed to the user and the user's authorization should be obtained through appropriate means in accordance with relevant laws and regulations.

[0021] For example, when a user's active request is received, a prompt message is sent to the user to clearly prompt the user that the operation requested by the user will require the acquisition and use of the user's personal information, so that the user can autonomously choose whether to provide personal information to software or hardware such as an electronic device, an application program, a server or a storage medium that performs the operations of the technical solutions of the present disclosure according to the prompt message.

[0022] As an optional but non-limiting implementation manner, the manner of sending a prompt message to the user in response to receiving the user's active request may be, for example, in the form of a pop-up window, and the prompt message may be presented in text in the pop-up window. In addition, the pop-up window may also carry selection controls for the user to choose "agree" or "disagree" to provide personal information to the electronic device.

[0023] It can be understood that the above process of notifying and obtaining the user's authorization is only illustrative and does not limit the implementation manners of the present disclosure. Other manners that meet the relevant laws and regulations can also be applied to the implementation manners of the present disclosure.

[0024] As used herein, the term "model" can learn the corresponding association relationship between inputs and outputs from training data, so that after training is completed, for a given input, the corresponding output can be generated. The generation of the model can be based on machine learning techniques. Deep learning is a machine learning algorithm that processes inputs and provides corresponding outputs by using multiple layers of processing units. A neural network model is an example of a model based on deep learning. In this article, "model" can also be referred to as "machine learning model", "learning model", "machine learning network", or "learning network", and these terms are used interchangeably herein.

[0025] A "neural network" is a machine learning network based on deep learning. A neural network can process inputs and provide corresponding outputs, and it generally includes an input layer and an output layer, as well as one or more hidden layers between the input layer and the output layer. The neural networks used in deep learning applications generally include many hidden layers, thus increasing the depth of the network. The layers of the neural network are connected in sequence, so that the output of the previous layer is provided as the input of the next layer, where the input layer receives the input of the neural network, and the output of the output layer is the final output of the neural network. Each layer of the neural network includes one or more nodes (also called processing nodes or neurons), and each node processes the input from the previous layer.

[0026] As mentioned above, with the development of machine learning techniques, using machine learning models to generate content has been widely applied in various fields. For example, machine learning models have been widely applied in fields such as natural language processing, human-computer dialogue, machine translation, speech synthesis, and image generation. Machine learning models have strong reasoning capabilities. During the execution of generation tasks (such as text generation tasks), this reasoning ability enables machine learning models to have strong divergent thinking and creative thinking. However, in some task scenarios, this reasoning ability also makes machine learning models prone to hallucinations, such as the output content of the model may not follow the input content, or the output content does not conform to the facts. Such hallucinations will affect the reliability of the output content of the model, and further limit the application of machine learning models in fields with high reliability requirements. Therefore, how to accurately evaluate the generation quality of the model has become a technical problem to be solved urgently.

[0027] In view of this, embodiments of the present disclosure propose an improved solution for evaluating the quality of model generation. In this improved solution, responses to user queries and context information associated with the generation of the responses are obtained. The response may be generated based on the context information using a machine learning model. For example, the response to the user query can be obtained by providing the user query and the context information to the machine learning model. Based on the context information, information defects in the response are detected in at least one dimension to obtain at least one detection result in at least one dimension. Each detection result here indicates at least the degree of defect of the response in the corresponding dimension. Thereafter, based on the at least one detection result, an evaluation result for the response is determined, and the evaluation result indicates the information generation quality of the machine learning model in at least one dimension.

[0028] In embodiments of the present disclosure, information defects of the output content (i.e., the response) of the model relative to the context information are detected from one or more dimensions, and then based on one or more detection results of the information defects, the information generation quality of the model in the one or more dimensions can be accurately evaluated. Thereby, it is beneficial to improve the information generation quality and reliability of the model, and further beneficial to improve the task ability of the model.

[0029] Various example implementations of this solution are further described in detail below in conjunction with the accompanying drawings.

[0030] Example environment

[0031] Figure 1 FIG. shows a schematic diagram of an example environment 100 in which embodiments of the present disclosure can be implemented. In this example environment 100, an application 120 is installed in the terminal device 110. The user 140 can interact with the application 120 via the terminal device 110 and / or an attached device of the terminal device 110.

[0032] In some embodiments of the present disclosure, the application 120 can be any suitable application with a human-computer dialogue function. As an example, the application 120 can provide a digital assistant for human-computer dialogue. The digital assistant supports content dialogue with the user 140 based on text dialogue services, voice dialogue services, image dialogue services, video dialogue services, and other modalities.

[0033] In some embodiments of the present disclosure, if the application 120 is in an active state, the terminal device 110 can present the user interface 150 of the application 120. The user interface 150 can include various types of pages that the application 120 can provide, such as a dialogue page between the user and the digital assistant (where the current dialogue and historical dialogues, including text dialogue content, can be presented), and so on. In some embodiments, the client 110 can present text or images in the user interface 150, and play voice or video.

[0034] In some embodiments, the application 120 or the digital assistant therein may utilize the machine learning model 160 (which may include one or more machine learning models, such as may include machine learning models 160-1, machine learning models 160-2, ……, machine learning models 160-N, etc., where N is a positive integer. For convenience of description, one or more machine learning models are collectively referred to as the machine learning model 160 herein) to support the interaction with the user 140. For example, the application 120 or the digital assistant therein may utilize one or more machine learning models 160 to implement the playback control of multimedia content (such as video).

[0035] The machine learning model 160 may be of different types. In some embodiments, one or more machine learning models 160 may be built based on a language model (LM). The machine learning model used is a content generation model that can generate corresponding outputs based on the model input. In some embodiments, the machine learning model based on the language model can understand model inputs of multiple modalities (such as text, speech, image, video, etc.), and can generate desired outputs according to the model input and the prompt words. Here, the prompt words are used to guide the machine learning model to generate outputs that can solve the user needs indicated by the model input. In the application scenario for supporting user conversations, the input of the user 140 can be provided to the machine learning model 160 as at least a part of the model input (the other part may include prompt words). This user input is regarded as a query. Based on the model output, a corresponding response can be generated and provided to the user 140. As an example, the machine learning model 160 may be built based on a multimodal large language model (which may also be referred to as a multimodal model).

[0036] In some embodiments, the terminal device 110 communicates with the server 130 to implement the supply of the services of the application 120. As Figure 1As shown, the server 130 may invoke the machine learning model 160 to support the human-machine dialogue function between the application 120 and the user 140 based on the output of the machine learning model 160. The terminal device 110 may be any type of mobile terminal, fixed terminal, or portable terminal, including mobile phones, desktop computers, laptop computers, notebook computers, netbook computers, tablet computers, media computers, multimedia tablets, personal communication system (PCS) devices, personal navigation devices, personal digital assistants (PDAs), audio / video players, digital cameras / video cameras, positioning devices, television receivers, radio broadcast receivers, e-book devices, gaming devices, or any combination of the foregoing, including accessories and peripherals of these devices or any combination thereof. In some embodiments, the terminal device 110 is also capable of supporting any type of interface for the user (such as a "wearable" circuit, etc.). The server 130 may be various types of computing systems / servers capable of providing computing power, including but not limited to mainframes, edge computing nodes, computing devices in a cloud environment, and the like. The server 130 may be implemented, for example, based on a cloud environment.

[0037] It should be understood that the structures and functions of the various elements in the environment 100 are described only for exemplary purposes and do not imply any limitation on the scope of the present disclosure.

[0038] Example process

[0039] Some example embodiments of the present disclosure will be described below with continued reference to the accompanying drawings. Figure 2 A flowchart of a process 200 for evaluating the quality of a generated model is shown according to some embodiments of the present disclosure. For ease of discussion hereinafter, the process 200 is described from the perspective of the server 130, but this is merely exemplary. At least a part of the process 200 may also be implemented in the terminal device 110, or may also be implemented by the server 130 in cooperation with other devices such as, for example, the terminal device 110.

[0040] In block 210, the server 130 obtains a response generated by the machine learning model 160-1 for a user query and context information associated with the generation of the response. The machine learning model 160-1 may be trained to be capable of generating a response to a user query based on the user query input by the user 140. In different application scenarios, the specific content of the user query and the response may be different. The user query and the response will be described by way of example in connection with some exemplary application scenarios below, but it should not be understood that the improvement solutions of the embodiments of the present disclosure are limited to being applied in the application scenarios shown below.

[0041] In some embodiments, the machine learning model 160-1 can be applied to the human-machine dialogue scenario. The terminal device 110 can receive a user query input by the user 140, and the user query can include multi-modal data such as text, audio, image, video, etc. The terminal device 110 can send a response request to the server 130 based on the user query. The server 130 can, in response to the response request, obtain context information related to the user query. By providing the user query and the context information to the machine learning model 160-1, a response output by the machine learning model 160-1 is obtained. The response can include the reply content to the question indicated by the user query. In this way, it can be applied to the human-machine dialogue scenario to evaluate the response generated by the machine learning model 160-1, which is beneficial to improving the quality of human-machine dialogue.

[0042] In some embodiments, the machine learning model 160-1 can be applied to the generation task scenario. The response can also include content such as text, audio, image, video, etc. generated based on the generation requirements indicated by the user query. For example, the user 140 can provide a creation requirement (i.e., the user query) to the machine learning model 160-1. Using the machine learning model 160-1, creation content such as an article, an image, or a video can be created based on the creation requirement. The server 130 can use the creation requirement and the creation content to evaluate the generation quality of the creation content.

[0043] In some embodiments, the machine learning model 160-1 can also be applied to the scenario of constructing a training set of a model. For example, a training set of a supervised fine-tuning (SFT) model can be constructed, or a training set for training a reward model can be constructed. As an example, a sample query (i.e., the user query) and the context information associated with the sample query can be provided to the trained machine learning model 160-1 to obtain the model output generated by the machine learning model 160-1. Based on the model output, a sample response (i.e., the response) corresponding to the sample query is determined. The server 130 can obtain the sample query, the sample response, and the context information to evaluate the generation quality of the sample response.

[0044] In some embodiments, the machine learning model 160-1 can also be the model to be evaluated. The server 130 can provide a sample query (i.e., a user query) and context information associated with the sample query to the machine learning model 160-1, and obtain the model output output by the machine learning model 160-1. The server 130 can determine a predicted response (i.e., a response) for the sample query based on the model output. The server 130 can obtain the sample query, the context information, and the predicted response, and evaluate the training effect of the machine learning model 160-1 by evaluating the information generation quality of the predicted response. It can be understood that the above application scenarios are only illustrative, and the improvement solutions of the embodiments of the present disclosure can be applied to any appropriate application scenarios according to actual needs, and the embodiments of the present disclosure are not limited thereto.

[0045] The context information can include various information associated with the generation of the response. In some examples, the context information can include at least one of historical conversation information or reference information. The historical conversation information can be understood as the conversation information of historical conversation turns, and can include at least one of historical user queries or historical responses to historical user queries. The reference information can include the information provided by the user 140. For example, the user 140 can actively upload information such as text, image, video, audio, etc. as reference information via the upload control in the user interface 150. The reference information can also include the information retrieved by the application 120 that is associated with the user query. For example, the application 120 can use a retrieval engine to retrieve network information such as web pages, network documents, etc. that are associated with the user query as reference information.

[0046] Continuing with Figure 2 As shown, at block 220, the server 130 detects information defects in the response in at least one dimension based on the context information to obtain at least one detection result in at least one dimension. Specifically, one or more dimensions for detecting information defects in the response can be determined in advance, and the server 130 can detect the information defects in the response in each dimension to obtain a detection result corresponding to that dimension. Each detection result at least indicates the degree of defect of the response in the corresponding dimension.

[0047] It can be understood that a high-quality response should generally possess basic characteristics (or basic attributes) such as accuracy, integrity, timeliness, effectiveness, consistency with the input, or faithfulness to the input. One or more of the basic characteristics that a response should possess can be selected as the dimensions for evaluating the information defects of the response. In other words, the at least one dimension can include one or more of the basic characteristics that a response should possess. Information defects can be understood as the lack of basic characteristics that a response should possess. For example, the lack of accuracy, integrity, timeliness, effectiveness, consistency with the input, or faithfulness to the input in the response.

[0048] Exemplary descriptions will be given below in connection with some exemplary dimensions for the detection of information defects in responses, but it should not be construed as being limited to detecting information defects in responses starting from the dimensions shown below. In actual application scenarios, information defects in responses can be detected based on any appropriate basic characteristics that the responses should possess. Embodiments of the present disclosure do not limit this.

[0049] In some embodiments, Figure 3 A flowchart of an example process 300 for generating the quality of an evaluation model according to some embodiments of the present disclosure is shown. As Figure 3 shown, the at least one dimension may include an information integrity dimension. The server 130 may obtain a user query 302, context information 304, and a response 310. Here, the context information 304 may include reference information 306 and historical conversation information 308. At block 312, the server 130 may determine the degree of information missing in the response 310 by detecting the missing information in the response 310 on the information integrity dimension based on the user query 302 and the context information 304. Thereafter, a first detection result 314 on the information integrity dimension may be generated based on the degree of information missing. Thus, it can be detected whether there are defects in the information integrity of the response 310.

[0050] As an example, the server 130 may generate prompt word information for the trained machine learning model 160-2 based on the user query 302, the context information 304, and the response 310. The server 130 may obtain a model output output by the machine learning model 160-2 by providing the prompt word information to the machine learning model 160-2. Thereafter, based on the model output, the first detection result 314 on the information integrity dimension is determined. Thus, the inference ability of the machine learning model 160-2 can be utilized to efficiently and accurately detect whether there is missing information in the response 310. Of course, in actual application scenarios, the information integrity of the response 310 can also be detected based on, for example, rules or algorithms. Embodiments of the present disclosure do not limit this.

[0051] In some embodiments, the server 130 may determine a set of key contents for answering the user query 302 based on the user query 302 and the context information 304. Thereafter, the server 130 may determine the degree of information missing by detecting the missing of the response 310 in the set of key contents. Here, the set of key contents can be understood as a set of basic points (which can also be called basic elements) that the response 310 to the user query 302 should include. The server 130 may detect whether there is a missing in the response in these basic points.

[0052] In actual application, the server 130 may determine a set of key contents for answering the user query 302 in various ways. In some embodiments,Figure 4 FIG. 4 is a flowchart of a process 400 for evaluating the quality of a generated evaluation model according to some embodiments of the present disclosure. As shown in FIG. 4, the server 130 may determine at least one of the keywords 402 of the user query 302, the conditions 404 for restricting the response to the user query 302, or the requirements 406 to be met for responding to the user query 302 based on the user query 302 and the context information 304. After that, the server 130 may determine a set of key contents 408 based on at least one of the keywords 402, the conditions 404, or the requirements 406.

[0053] The keyword 402 may include the words in the user query 302 or the words associated with the words in the user query 302. For example, the server 130 may detect one or more first keywords in the user query 302. The server 130 may also determine one or more second keywords associated with the one or more first keywords. After that, these first keywords and second keywords are determined as the keywords 402 of the user query 302.

[0054] The condition 404 may indicate the requirements for responding to the user query 302. In some examples, the condition 404 may indicate the requirements for the data format or data structure of the response 310. The condition 404 may also indicate the requirements for the content of the response 310. For example, the condition 404 may indicate that the content of the response 310 should conform to a certain specification or a certain standard, etc.

[0055] The requirement 406 may indicate the response content expected to be obtained by the user query 302. In some examples, the requirement 406 may include at least one of a core requirement, an explicit requirement, and an implicit requirement. The core requirement may be understood as the most critical or main requirement. The explicit requirement may be understood as the requirement for the response content that can be directly derived from the user query 302 or the context information 304. The implicit requirement may be understood as the potential requirement for the response content that cannot be directly derived from the user query 302. For example, the user query 302 does not directly point out, but the requirement that has a strong correlation with the response content indicated by the core requirement or the explicit requirement.

[0056] As an example, the server 130 may determine the keyword 402, the condition 404, and the requirement 406 based on the user query 302 and the context information 304 using the machine learning model 160-2. After that, the server 130 may also use the machine learning model 160-2 to determine a set of key contents 408 based on the keyword 402, the condition 404, and the requirement 406, such as determining a set of reply points.

[0057] In some embodiments, the server 130 may determine the query scenario corresponding to the user query 302. Subsequently, the server 130 may, based on the user query 302 and the context information 304, use a determination rule that matches the query scenario to determine a set of key content 408. The query scenario may indicate the type of task corresponding to the user query 302, such as a question-and-answer task, a paraphrasing task, a summarization task, a generation task (such as a writing task, an image generation task, an audio generation task, or a video generation task, etc.), a search task, and so on. The determination rule may indicate at least one of the determination criteria for the key content in the corresponding query scenario or the candidate key content. In some examples, the determination rule may also include an example of key content determination in the corresponding query scenario.

[0058] As an example, the server 130 may generate prompt information for the machine learning model 160-2 based on the determination criteria for the key content in the corresponding query scenario, the candidate key content, the example of determining the key content, the keywords 402, the requirements 404, and the conditions 406. By providing the prompt information to the machine learning model 160-2, the model output of the machine learning model 160-2 is obtained. Subsequently, based on the model output, a set of key content 408 is determined. Of course, in practice, it is not limited to using the machine learning model 160-2 to determine a set of key content, and a set of key content may also be determined based on, for example, rules and algorithms. For example, multiple candidate contents may be given for the query scenario, and a set of key content may be selected from the multiple candidate contents based on the user query 302 and the context information 304.

[0059] In some embodiments, as shown in Figure 4 After determining a set of key content 408, the server 130 may also determine a score 420 for the degree of match between the set of key content 408 and the user query 302. At block 422, the server 130 may determine whether the score 420 exceeds a score threshold. If it is determined at block 422 that the score 420 exceeds the score threshold, process 400 may proceed to block 424. At block 424, the server 130 may output a set of key content. Subsequently, the server 130 may detect the absence of the response 310 in the set of key content 408 and determine the degree of information absence of the response 310. If it is determined at block 422 that the score does not exceed the score threshold, the server 130 may update the set of key content 408 based on the score.

[0060] In some examples, server 130 may be pre-configured with multiple matching dimensions for determining the matching degree between the key content and the user query. For example, the multiple matching dimensions may include the degree of satisfaction of requirements, the accuracy of facts, the suitability of weights, the tightness of logic, or the richness of key content, etc. Server 130 determines the corresponding sub-ratings of a set of key content 408 and the user query 302 on the multiple matching dimensions. Thereafter, server 130 may determine a rating based on the corresponding sub-ratings.

[0061] As an example, as Figure 4 shown, the multiple matching dimensions may include matching dimension 410, matching dimension 412, matching dimension 414, matching dimension 416, and matching dimension 418. The total score of the sub-ratings on each matching dimension is M points, and the total score of the matching degree rating 420 is 5M points. Server 130 may generate prompt information for the machine learning model 160-3 based on a set of key content 408, the user query 302, the keywords 402, the conditions 404, the requirements 406, the multiple matching dimensions, and the rating criteria. By providing the prompt information to the machine learning model 160-3, a set of sub-ratings is obtained. Server 130 may determine the matching degree rating 420 based on the summation of a set of sub-ratings. If the rating 420 exceeds the rating threshold, server 130 may output a set of key content 408. If the rating 420 does not exceed the rating threshold, server 130 may update the set of key content 408 based on the value combination of the set of sub-ratings. For example, server 130 may re-provide the set of sub-ratings and the set of key content 408 to the machine learning model 160-2, instructing the machine learning model 160-2 to regenerate a set of key content 408. It should be noted that the above evaluation method for determining the rating is only exemplary, and any appropriate method can be selected according to actual needs to evaluate the matching degree between the key content and the user query. For example, server 130 may also determine the matching degree between the key content and the user query based on, for example, semantic matching. The embodiments of the present disclosure are not limited thereto.

[0062] In some embodiments, the at least one dimension may include an information source dimension. In some cases, the information source dimension may also be referred to as the faithfulness dimension to the input. Combining Figure 3As shown, the server 130 may generate a second detection result 324 in the information source dimension by detecting unfounded content in the response 310 based on the context information 304. Unfounded content can be understood as the content in the response 310 that does not follow the context information 304. In some examples, the unfounded content may be the content included in the response 310 that does not appear in the context information 304 and cannot be directly or indirectly deduced from the context information 304. In some cases, the unfounded content may also be referred to as "fictitious content".

[0063] As an example, if the content deduced from the context information 304 in the response 310 has a lack of logic or obvious defects in the reasoning process. The server 130 may determine this part of the content as unfounded content. Alternatively or additionally, if the response 310 includes content that is clearly not present in the context information 304 and exceeds the semantic scope of the context information 304, the server 130 may determine this part of the content as unfounded content. Alternatively or additionally, if the response 310 includes evaluative content, hypothetical content, subjective judgment content, etc. that cannot be directly or indirectly deduced from the context information 304, the server 130 may also determine these evaluative content, hypothetical content, or subjective judgment content as unfounded content. It should be noted that the content directly cited from the context information 304 in the response 310, the content that can be directly or indirectly deduced from the context information 304, and the content supplemented based on prior knowledge and not exceeding the semantic scope of the context information 304 may be considered not to belong to unfounded content.

[0064] As another example, the server 130 may provide the context information 304, the response 310, and the user query 302 to the machine learning model 160-4. Use the machine learning model 160-4 to determine the unfounded content in the response. Utilizing the semantic understanding ability of the machine learning model 160-4 can improve the detection efficiency and accuracy of unfounded content.

[0065] In some embodiments, in combination with Figure 3 As shown, at block 316, the server 130 may determine multiple response segments of the response 310, and the server 130 may also determine multiple information segments of the context information 304. At block 318, the server 130 may determine the correspondence between the multiple response segments and the multiple information segments. At block 322, for each response segment among the multiple response segments, the server 130 may detect whether there is unfounded content in the response segment relative to the corresponding information segment based on the correspondence.

[0066] For example, the server 130 may determine multiple information segments of the context information 304, such as multiple statements, paragraphs, or sections, etc., based on, for example, a predetermined rule. The server 130 may add indexes to the multiple information segments in the context information 304. The server 130 may also determine multiple response segments in the response 310, such as multiple words, short phrases, sentences, or paragraphs, etc., based on, for example, a predetermined rule. The server 130 may use the indexes to identify the correspondence between the multiple response segments and the multiple information segments. After that, the server 130 may generate prompt word information for the machine learning model 160-4 based on the context information 304 with indexes added, the response 310 with indexes added, and the user query 302. By providing the prompt word information to the machine learning model 160-4, it is determined whether there is unfounded content in the response 310.

[0067] In some embodiments, continuing with Figure 3 As shown, at block 320, for each of the multiple response segments, the server 130 may determine the degree of consistency between the response segment and the corresponding information segment based on the correspondence. If the degree of consistency does not exceed the consistency threshold, the process 300 may proceed to 322. At block 300, the server 130 may detect whether there is unfounded content in the response segment relative to the corresponding information segment. If the degree of consistency exceeds the consistency threshold, it indicates that there is a high degree of consistency between the response segment and the corresponding information segment, and it is not likely to have unfounded content. The server 130 may directly determine that the response segment does not include unfounded content without detecting whether there is unfounded content in the response segment relative to the corresponding information segment. Thus, while ensuring the accuracy of the detection of unfounded content, the resource (e.g., computing resource) consumption of the server 130 can be reduced, which is beneficial to improving the evaluation efficiency and speed.

[0068] In some embodiments, the at least one dimension may include an information consistency dimension. Continuing with Figure 3 As shown, the server 130 may generate a third detection result 328 in the information consistency dimension by detecting the conflicting content between the response 310 and the context information 304 in the information consistency dimension based on the context information 310. In some examples, the conflicting content may be the response content in the response 310 related to an entity, object, or event, and the expression of the entity, object, or event in the response content is inconsistent with the corresponding content in the context information. As an example, the server 130 may generate prompt word information for the machine learning model 160-5 based on the user query 302, the context information 304, and the response 310. By providing the prompt word information to the machine learning model 160-5, the conflicting content between the response 310 and the context information 304 is determined.

[0069] In some embodiments, continuing with reference to Figure 3 As shown, the server 130 may utilize the processes shown in blocks 316 and 318 to determine the correspondence between multiple response segments in the response 310 and multiple information segments in the context information 304. At block 326, for each response segment among the multiple response segments, the server 130 may, based on the correspondence, detect whether there is conflicting content between the response segment and the corresponding information segment. As an example, the server 130 may generate prompt word information for the machine learning model 160-5 based on the indexed context information 304, the indexed response 310, and the user query 302. By providing the prompt word information to the machine learning model 160-5, the conflicting content between the response segment and the corresponding information segment is determined. Of course, the server 130 may also determine the conflicting content between the response segment and the corresponding information segment based on semantic matching, and the embodiments of the present disclosure do not limit this.

[0070] In some embodiments, the server 130 may pre-determine multiple detection items. Each detection item may indicate a corresponding detection rule. For each response segment among the multiple response segments, the server 130 may, based on the pre-determined multiple detection items, detect the conflicting content between the response segment and the corresponding information segment. Specifically, the server 130 may detect the conflicting content between the response segment and the corresponding information segment based on each of the multiple detection items.

[0071] As an example, the multiple detection items may include a first detection item. The first detection item may indicate detecting whether there is a conflict between the description of the key content of the event in the response segment and the description of the key content of the event in the information segment. Alternatively or additionally, the multiple detection items may further include a second detection item. The second detection item may indicate detecting whether there is a conflict between the description of the temporal relationship of the event in the response segment and the description of the temporal relationship of the event in the information segment. Alternatively or additionally, the multiple detection items may further include a third detection item. The third detection item may indicate detecting whether there is a conflict between the description of the participating entities of the event in the response segment and the description of the participating entities of the event in the information segment. Alternatively or additionally, the multiple detection items may further include a fourth detection item. The fourth detection item may indicate detecting whether there is a conflict between the description of the causal relationship of the event in the response segment and the description of the causal relationship of the event in the information segment. Alternatively or additionally, the multiple detection items may further include a fifth detection item. The fifth detection item may indicate detecting whether there is a conflict between the description of the event background in the response segment and the description of the event background in the information segment. Of course, the above detection items are only exemplary, and any appropriate detection items may be selected according to actual needs to detect the conflicting content, and the embodiments of the present disclosure do not limit this.

[0072] It should be noted that the dimensions of the information defects in the above detection responses are exemplary. In actual application scenarios, the above-mentioned multiple dimensions can be freely combined or split according to actual needs, or any other appropriate dimensions can be selected to detect the information defects in the responses to meet diversified or customized requirements. The embodiments of the present disclosure do not limit this.

[0073] Return to process 200. At block 230, the server 130 determines an evaluation result for the response based on at least one detection result. The evaluation result indicates the quality of information generation of the machine learning model 160-1 in the at least one dimension.

[0074] In some embodiments, if each detection result in the at least one detection result indicates that the response has no information defect, the server 130 may generate an evaluation result indicating that the information generation quality of the response (which may also indicate the machine learning model 160-1) meets the predetermined requirements, or the server 130 may also generate an evaluation result indicating that the information generation quality of the response meets the predetermined requirements. If any one of the at least one detection results indicates that the response has an information defect, the server 130 may generate an evaluation result indicating that the information generation quality of the response (which may also indicate the machine learning model 160-1) does not meet the predetermined requirements.

[0075] As an example, in combination with Figure 3 As shown, if the first detection result 314 indicates that there is no information loss in the response 310, the second detection result 324 indicates that there is no unfounded content in the response 310 relative to the context information 304, and the third detection result 328 indicates that there is no conflicting content between the response 310 and the context information 304, the server 130 may generate an evaluation result 330 indicating that the response 310 meets the predetermined requirements. If any one of the first detection result 314, the second detection result 324, or the third detection result 328 indicates that the response 310 has an information defect, the server 130 may generate an evaluation result 330 indicating that the response 310 does not meet the predetermined requirements.

[0076] It can be understood that the above method for determining the evaluation result is relatively strict, which is beneficial to ensuring the quality of the response. In actual applications, the evaluation result can also be determined by other means. For example, the server 130 can also determine the evaluation result based on the at least one detection result and the corresponding at least one weight. The embodiments of the present disclosure do not limit this.

[0077] In some embodiments, if the evaluation result indicates that the information generation quality of the response does not meet the predetermined requirements in at least one dimension, the server 130 may update the response based on the evaluation result. As an example, the evaluation result may further include suggestions for updating the response. The server 130 may provide the evaluation result, the response, the user query, and the context information to the machine learning model 160-1 to regenerate a response to the user query through the machine learning model 160-1. Thus, the generation quality of the response can be improved, ensuring that the final output result of the machine learning model 160-1 highly conforms to the task objective.

[0078] In this way, in the embodiments of the present disclosure, information defects of the output content (i.e., the response) of the model relative to the context information are detected from one or more dimensions, and then based on one or more detection results of the information defects, the information generation quality of the model in the one or more dimensions can be accurately evaluated. Thus, it is beneficial to improve the information generation quality and reliability of the model, and further beneficial to improve the task ability of the model.

[0079] Example device and equipment

[0080] Embodiments of the present disclosure also provide corresponding apparatuses for implementing the above methods or processes. Figure 5 FIG. shows a schematic structural block diagram of an exemplary apparatus 500 for evaluating the generation quality of a model according to certain embodiments of the present disclosure. The apparatus 500 may be implemented as or included in the server 130. Each module / component in the apparatus 500 may be implemented by hardware, software, firmware, or any combination thereof.

[0081] As Figure 5 shown, the apparatus 500 includes: an acquisition module 510 configured to acquire a response generated by a machine learning model for a user query and context information associated with the generation of the response; a detection module 520 configured to, based on the context information, detect information defects of the response in at least one dimension to obtain at least one detection result in at least one dimension, each detection result at least indicating the degree of defect of the response in the corresponding dimension; and an evaluation module 530 configured to, based on at least one detection result, determine an evaluation result for the response, the evaluation result indicating the information generation quality of the machine learning model in at least one dimension.

[0082] In some embodiments, the detection module 520 is further configured to: based on the user query and the context information, determine the degree of missing information of the response by detecting the missing information of the response in the information integrity dimension in at least one dimension; and generate a first detection result in the information integrity dimension based on the degree of missing information.

[0083] In some embodiments, the detection module 520 is further configured to: determine a set of key content for answering a user query based on the user query and context information; and determine the degree of information loss by detecting the absence of an answer in the set of key content.

[0084] In some embodiments, the detection module 520 is further configured to: determine at least one of a keyword of the user query, a condition for restricting an answer to the user query, or a requirement to be met for answering the user query based on the user query and context information; and determine a set of key content based on at least one of the keyword, condition, or requirement.

[0085] In some embodiments, the detection module 520 is further configured to: determine a query scenario corresponding to the user query; and determine a set of key content based on the user query and context information by using a determination rule that matches the query scenario.

[0086] In some embodiments, the apparatus 500 further includes: a first update module configured to determine a score for the degree of match between a set of key content and the user query; and update the set of key content based on the score in response to the score not exceeding a score threshold.

[0087] In some embodiments, the first update module is further configured to: determine corresponding sub-scores of the set of key content and the user query in multiple matching dimensions; and determine the score based on the corresponding sub-scores.

[0088] In some embodiments, the detection module 520 is further configured to: generate a second detection result in the information source dimension by detecting unfounded content in the answer in the information source dimension among at least one dimension based on the context information.

[0089] In some embodiments, the detection module 520 is further configured to: determine the correspondence between multiple answer segments in the answer and multiple information segments in the context information; and based on the correspondence, detect whether there is unfounded content in each answer segment relative to the corresponding information segment among the multiple answer segments.

[0090] In some embodiments, the detection module 520 is further configured to: for each of the multiple answer segments, determine the degree of consistency between the answer segment and the corresponding information segment based on the correspondence; and detect whether there is unfounded content in the answer segment relative to the corresponding information segment in response to the degree of consistency not exceeding a consistency threshold.

[0091] In some embodiments, the detection module 520 is further configured to: based on the context information, generate a third detection result in the information consistency dimension by detecting the conflicting content between the response and the context information in the information consistency dimension in at least one dimension.

[0092] In some embodiments, the detection module 520 is further configured to: determine the correspondence between multiple response segments in the response and multiple information segments in the context information; and based on the correspondence, detect whether there is any conflicting content between each response segment in the multiple response segments and the corresponding information segment.

[0093] In some embodiments, the detection module 520 is further configured to: for each response segment in the multiple response segments, detect the conflicting content between the response segment and the corresponding information segment based on a predetermined multiple detection items.

[0094] In some embodiments, the apparatus 500 further includes: a second update module, configured to update the response based on the evaluation result in response to the evaluation result indicating that the information generation quality of the response does not meet the predetermined requirements in at least one dimension.

[0095] The units and / or modules included in the apparatus 500 can be implemented in various ways, including software, hardware, firmware, or any combination thereof. In some embodiments, one or more units and / or modules can be implemented using software and / or firmware, such as machine-executable instructions stored on a storage medium. In addition to or as an alternative to the machine-executable instructions, some or all of the units and / or modules in the apparatus 500 can be at least partially implemented by one or more hardware logic components. By way of example and not limitation, exemplary types of hardware logic components that can be used include field programmable gate arrays (FPGA), application specific integrated circuits (ASIC), application specific standard products (ASSP), system on a chip (SOC), complex programmable logic devices (CPLD), and so on.

[0096] Figure 6 The block diagram of an electronic device 600 in which one or more embodiments of the present disclosure can be implemented is shown. It should be understood that Figure 6 The electronic device 600 shown is merely exemplary and should not constitute any limitation to the functions and scopes of the embodiments described herein. Figure 6 The electronic device 600 shown can include or be implemented as Figure 1 the server 130, or Figure 5 the apparatus 500.

[0097] Such as Figure 6As shown, the electronic device 600 is in the form of a general-purpose electronic device. The components of the electronic device 600 may include, but are not limited to, one or more processors 610, a memory 620, a storage device 630, one or more communication units 640, one or more input devices 650, and one or more output devices 660. The processor 610 may be a physical or virtual processor and is capable of performing various processes according to computer-executable instructions stored in the memory 620. In a multi-processor system, multiple processors execute computer-executable instructions in parallel to improve the parallel processing ability of the electronic device 600.

[0098] The electronic device 600 generally includes multiple computer storage media. Such media can be any accessible media that can be obtained by the electronic device 600, including but not limited to volatile and non-volatile media, removable and non-removable media. The memory 620 may be volatile memory (such as registers, caches, random access memory (RAM)), non-volatile memory (such as read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory), or some combination thereof. The storage device 630 may be removable or non-removable media and may include machine-readable media, such as a flash drive, a magnetic disk, or any other media that can be used to store information and / or data and can be accessed within the electronic device 600.

[0099] The electronic device 600 may further include additional removable / non-removable, volatile / non-volatile storage media. Although not shown in Figure 6 , a disk drive for reading from or writing to a removable, non-volatile magnetic disk (such as a "floppy disk") and an optical disk drive for reading from or writing to a removable, non-volatile optical disk may be provided. In these cases, each drive may be connected to a bus (not shown) by one or more data media interfaces. The memory 620 may include a computer program product 625 having one or more computer-executable instruction modules configured to perform the various methods or actions of the various embodiments of the present disclosure.

[0100] The communication unit 640 enables communication with other electronic devices through a communication medium. Additionally, the functions of the components of the electronic device 600 may be implemented by a single computing cluster or multiple computer machines that are capable of communicating through a communication connection. Thus, the electronic device 600 may operate in a networked environment using a logical connection with one or more other servers, network personal computers (PCs), or another network node.

[0101] The input device 650 can be one or more input devices, such as a mouse, a keyboard, a trackball, etc. The output device 660 can be one or more output devices, such as a display, a speaker, a printer, etc. The electronic device 600 can also communicate with one or more external devices (not shown) as needed through the communication unit 640. The external devices such as a storage device, a display device, etc., communicate with one or more devices that enable a user to interact with the electronic device 600, or communicate with any device that enables the electronic device 600 to communicate with one or more other electronic devices (e.g., a network card, a modem, etc.). Such communication can be performed via an input / output (I / O) interface (not shown).

[0102] According to an exemplary implementation of the present disclosure, there is provided a computer-readable storage medium having computer-executable instructions stored thereon, wherein the computer-executable instructions are executed by a processor to implement the method described above. According to an exemplary implementation of the present disclosure, there is also provided a computer program product, the computer program product being tangibly stored on a non-transitory computer-readable storage medium and including computer-executable instructions, and the computer-executable instructions being executed by a processor to implement the method described above.

[0103] Aspects of the present disclosure are described herein with reference to the flowcharts and / or block diagrams of methods, apparatuses, devices, and computer program products implemented according to the present disclosure. It should be understood that each block of the flowcharts and / or block diagrams, and the combinations of blocks in the flowcharts and / or block diagrams, can be implemented by computer-executable instructions.

[0104] These computer-executable instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, thereby producing a machine such that when these instructions are executed by the processor of the computer or other programmable data processing device, a device is produced that implements the functions / actions specified in one or more blocks of the flowchart and / or block diagram. These computer-executable instructions can also be stored in a computer-readable storage medium, and these instructions cause the computer, programmable data processing device, and / or other devices to work in a specific manner. Thus, the computer-readable storage medium storing the instructions includes a manufacture that includes instructions for implementing various aspects of the functions / actions specified in one or more blocks of the flowchart and / or block diagram.

[0105] The computer-executable instructions can be loaded onto a computer, other programmable data processing device, or other device such that a series of operation steps are performed on the computer, other programmable data processing device, or other device to produce a computer-implemented process, so that the instructions executed on the computer, other programmable data processing device, or other device implement the functions / actions specified in one or more blocks of the flowchart and / or block diagram.

[0106] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer-executable instruction products according to various implementations of the present disclosure. In this regard, each block in the flowchart or block diagram may represent a module, an executable instruction, or a portion of an instruction that contains one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions noted in the blocks may occur in a different order than noted in the accompanying drawings. For example, two consecutive blocks may actually be executed substantially in parallel, or they may sometimes be executed in the reverse order, depending on the functionality involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented by a dedicated hardware-based system that performs the specified functions or actions, or can be implemented by a combination of dedicated hardware and computer instructions.

[0107] The various implementations of the present disclosure have been described above. The above description is exemplary, not exhaustive, and is not limited to the disclosed implementations. Many modifications and variations will be apparent to those of ordinary skill in the art in the field of this technology without departing from the scope and spirit of the described implementations. The choice of terms used herein is intended to best explain the principles of the implementations, the practical application, or the improvement of the technology in the market, or to enable other ordinary skilled persons in the field of this technology to understand the various implementation manners disclosed herein.

Claims

1. A method for evaluating the quality of model-generated responses, comprising: Obtaining a response generated by a machine learning model for a user query and context information associated with the generation of the response; Based on the context information, detecting information defects in the response in at least one dimension to obtain at least one detection result in the at least one dimension, each detection result at least indicating the degree of defect of the response in the corresponding dimension; And Based on the at least one detection result, determining an evaluation result for the response, the evaluation result indicating the information generation quality of the machine learning model in the at least one dimension.

2. The method according to claim 1, wherein detecting information defects in the response in the at least one dimension comprises: Based on the user query and the context information, determining the degree of information missing in the response by detecting the missing information in the information integrity dimension among the at least one dimension; And Based on the degree of information missing, generating a first detection result in the information integrity dimension.

3. The method according to claim 2, wherein determining the degree of information missing in the response comprises: Based on the user query and the context information, determining a set of key contents for answering the user query; And By detecting the missing of the response in the set of key contents, determining the degree of information missing.

4. The method according to claim 3, wherein determining the set of key contents comprises: Based on the user query and the context information, determining at least one of the keywords of the user query, the conditions for restricting the response to the user query, or the requirements to be met for answering the user query; And Based on at least one of the keywords, the conditions, or the requirements, determining the set of key contents.

5. The method according to claim 3, wherein determining the set of key contents comprises: Determining the query scenario corresponding to the user query; And Based on the user query and the context information, using a determination rule matching the query scenario, determining the set of key contents.

6. The method according to claim 3, further comprising: Determining a score for the degree of match between the set of key contents and the user query; And In response to the score not exceeding a score threshold, updating the set of key contents based on the score.

7. The method according to claim 6, wherein determining a score for the degree of match between the set of key contents and the user query comprises: Determining corresponding sub-scores of the set of key contents and the user query in multiple match dimensions; And Based on the corresponding sub-scores, determining the score.

8. The method according to claim 1, wherein detecting information defects in the response in the at least one dimension comprises: Based on the context information, detecting unfounded content in the response in the information source dimension among the at least one dimension, and generating a second detection result in the information source dimension.

9. The method according to claim 8, wherein detecting unfounded content in the response comprises: Determine the correspondence between multiple response segments in the response and multiple information segments in the context information; And Based on the correspondence, for each response segment among the multiple response segments, detect whether there is unfounded content in the response segment relative to the corresponding information segment.

10. The method according to claim 9, wherein detecting whether there is unfounded content in each response segment among the multiple response segments relative to the corresponding information segment includes: For each segment among the multiple response segments, Based on the correspondence, determine the degree of consistency between the response segment and the corresponding information segment; And In response to the degree of consistency not exceeding a consistency threshold, detect whether there is unfounded content in the response segment relative to the corresponding information segment.

11. The method according to claim 1, wherein detecting information defects in the response in at least one dimension includes: Based on the context information, by detecting conflict content between the response and the context information in the information consistency dimension among the at least one dimension, generate a third detection result in the information consistency dimension.

12. The method according to claim 11, wherein detecting conflict content between the response and the context information includes: Determine the correspondence between multiple response segments in the response and multiple information segments in the context information; And Based on the correspondence, for each response segment among the multiple response segments, detect whether there is conflict content between the response segment and the corresponding information segment.

13. The method according to claim 12, wherein detecting whether there is conflict content between each response segment among the multiple response segments and the corresponding information segment includes: For each response segment among the multiple response segments, based on a plurality of predetermined detection items, detect conflict content between the response segment and the corresponding information segment.

14. The method according to claim 1, further comprising: In response to the evaluation result indicating that the information generation quality of the response in the at least one dimension does not meet a predetermined requirement, update the response based on the evaluation result.

15. An apparatus for evaluating the quality of model generation, comprising: An acquisition module configured to acquire a response generated by a machine learning model for a user query and context information associated with the generation of the response; A detection module configured to, based on the context information, detect information defects in the response in at least one dimension to obtain at least one detection result in the at least one dimension, each detection result at least indicating the degree of defect of the response in the corresponding dimension; And An evaluation module configured to, based on the at least one detection result, determine an evaluation result for the response, the evaluation result indicating the information generation quality of the machine learning model in the at least one dimension.

16. An electronic device, comprising: At least one processor; And at least one memory coupled to the at least one processor and storing instructions for execution by the at least one processor, the instructions when executed by the at least one processor causing the electronic device to perform the method according to any one of claims 1 to 14.

17. A computer-readable storage medium having computer-executable instructions stored thereon, the computer-executable instructions being executable by a processor to implement the method according to any one of claims 1 to 14.

18. A computer program product comprising computer-executable instructions, wherein the computer-executable instructions when executed by a processor implement the method according to any one of claims 1 to 14.

Images