Model evaluation method and device and related product

Abstract

The embodiment of the invention provides a model evaluation method and device and related products, and the method comprises the steps: obtaining first information which is reply information of a target model to a first question; extracting at least one knowledge point from the first information; for each knowledge point, retrieving and acquiring at least one piece of second information for introducing the knowledge point; and evaluating the accuracy of the first information according to each piece of second information corresponding to each knowledge point.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technology, and in particular to a model evaluation method, device and related products. Background Art

[0002] At present, the accuracy of the model's response information needs to be evaluated in many scenarios. For example, when a user is talking to a large language model, the large language model returns a response to the user based on the question raised by the user. In this scenario, it is necessary to evaluate the accuracy of the large language model's response information to promptly discover errors in the large language model's response and improve the user's conversation experience with the large language model.

[0003] Based on this, it is necessary to provide a technical solution to efficiently and accurately evaluate the accuracy of the model's response information. Summary of the invention

[0004] The embodiments of the present disclosure provide a model evaluation method, device and related products, which can efficiently and accurately evaluate the accuracy of the model's response information.

[0005] In a first aspect, an embodiment of the present disclosure provides a model evaluation method, comprising: Acquire first information, where the first information is answer information of the target model to the first question; Extracting at least one knowledge point from the first information; For each of the knowledge points, retrieve and obtain at least one second information for introducing the knowledge point; The accuracy of the first information is evaluated according to each piece of the second information corresponding to each of the knowledge points.

[0006] In a second aspect, an embodiment of the present disclosure provides a model evaluation device, including: A first acquisition unit, configured to acquire first information, where the first information is answer information of the target model to the first question; A content extraction unit, configured to extract at least one knowledge point from the first information; A second acquisition unit, configured to retrieve and acquire, for each of the knowledge points, at least one second information for introducing the knowledge point; The content evaluation unit is used to evaluate the accuracy of the first information according to each piece of the second information corresponding to each of the knowledge points.

[0007] In a third aspect, an embodiment of the present disclosure provides an electronic device, comprising: a processor; and a memory configured to store computer-executable instructions, wherein the computer-executable instructions, when executed, enable the processor to implement the steps of the method described in the first aspect above.

[0008] In a fourth aspect, an embodiment of the present disclosure provides a computer-readable storage medium, wherein the computer-readable storage medium is used to store computer-executable instructions, and the computer-executable instructions implement the steps of the method described in the first aspect when executed by a processor.

[0009] In a fifth aspect, an embodiment of the present disclosure provides a computer program product, wherein the computer program product includes a computer program, and when the computer program is executed by a processor, the steps of the method described in the first aspect are implemented.

[0010] In one or more embodiments of the present disclosure, first, the first information is obtained, and the first information is the answer information of the target model to the first question. Then, at least one knowledge point is extracted from the first information. Then, for each knowledge point, at least one second information for introducing the knowledge point is retrieved and obtained. Finally, the accuracy of the first information is evaluated based on the second information corresponding to each knowledge point. It can be seen that through this embodiment, when evaluating the accuracy of the first information, each knowledge point in the first information can be determined, and the corresponding second information can be retrieved for each knowledge point. The second information is the more accurate information retrieved. The accuracy of the first information is evaluated using the second information corresponding to each knowledge point, thereby achieving the effect of efficiently and accurately evaluating the accuracy of the model's reply information and improving the accuracy of evaluating the accuracy of the model's reply information. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] In order to more clearly illustrate one or more embodiments of the present disclosure or the technical solutions in the prior art, the drawings required for use in the embodiments or the prior art descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present disclosure. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative labor. Figure 1 A schematic diagram of a flow chart of a model evaluation method provided in one embodiment of the present disclosure; Figure 2 A schematic diagram of the structure of a model evaluation device provided in one embodiment of the present disclosure; Figure 3 A schematic diagram of the structure of an electronic device provided in one embodiment of the present disclosure. DETAILED DESCRIPTION

[0012] In order to enable those skilled in the art to better understand the technical solutions in one or more embodiments of the present disclosure, the technical solutions in one or more embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in one or more embodiments of the present disclosure. Obviously, the described embodiments are only part of the embodiments of the present disclosure, not all of the embodiments. Based on one or more embodiments of the present disclosure, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the protection scope of the present disclosure.

[0013] It is understandable that before using the technical solutions disclosed in the embodiments of the present disclosure, the types, scope of use, usage scenarios, etc. of the information involved in the present disclosure should be informed to the relevant parties and their authorization should be obtained in an appropriate manner in accordance with relevant laws and regulations.

[0014] The disclosed embodiments provide a model evaluation method that can efficiently and accurately evaluate the accuracy of information.

[0015] Figure 1 A flow chart of a model evaluation method provided in an embodiment of the present disclosure is shown as follows: Figure 1 As shown, the process includes: Step S102, obtaining first information; the first information is the answer information of the target model to the first question; Step S104, extracting at least one knowledge point from the first information; Step S106, for each knowledge point, retrieve and obtain at least one second information for introducing the knowledge point; Step S108, evaluating the accuracy of the first information according to each piece of second information corresponding to each knowledge point.

[0016] In this embodiment, first, the first information is obtained, and the first information is the answer information of the target model to the first question. Then, at least one knowledge point is extracted from the first information. Then, for each knowledge point, at least one second information for introducing the knowledge point is retrieved and obtained. Finally, the accuracy of the first information is evaluated based on the second information corresponding to each knowledge point. It can be seen that through this embodiment, when evaluating the accuracy of the first information, each knowledge point in the first information can be determined, and the corresponding second information can be retrieved for each knowledge point. The second information is the more accurate information retrieved. The accuracy of the first information is evaluated using the second information corresponding to each knowledge point, which achieves the effect of efficiently and accurately evaluating the accuracy of the model's reply information, and improves the accuracy of evaluating the accuracy of the model's reply information.

[0017] In the above step S102, the first information is obtained. The first information may include any text to be evaluated. In one scenario, the first information may be the reply information returned by the target model, such as a dialogue robot, to the user based on the first question raised by the user. For example, the user raises the first question "When was the Western Han Dynasty" to the target model, and the first information replied by the target model is: "The time range of the Western Han Dynasty is from 202 BC to 8 AD, a total of 210 years. Wang Mang usurped the throne at the end of 8 AD and announced the establishment of a new dynasty, marking the demise of the Western Han Dynasty. In 202 BC, Liu Bang proclaimed himself emperor in Chang'an and established the Han Dynasty, known in history as the Western Han Dynasty. The Western Han Dynasty was the second feudal unified dynasty after the Qin Dynasty, and the country was named Han". Among them, the target model may be an LLM (Large Language Model).

[0018] In the above step S104, at least one knowledge point is extracted from the first information. In this embodiment, the first information is used to introduce at least one knowledge point. For example, in the above first information, at least the time range of the Western Han Dynasty, which dynasty the Western Han Dynasty was established after, what was the name of the Western Han Dynasty, where was the capital of the Western Han Dynasty, and other knowledge points are introduced. Based on this, in this step, one or more knowledge points introduced by the first information are extracted from the first information.

[0019] In one embodiment, extracting at least one knowledge point from the first information includes: At least one knowledge point is extracted from the first information through the first language model based on the knowledge point extraction rule.

[0020] In this embodiment, the first language model can be an LLM large language model. The first language model is positioned as a question extraction model, and the first information and knowledge point extraction rules are input into the first language model. The knowledge point extraction rules are used as prompt words for the first language model, so that the first language model can have the ability to extract various knowledge points from the first information. Then, through the first language model, based on the knowledge point extraction rules, at least one knowledge point is extracted from the first information. In this embodiment, the knowledge points extracted from the first information can be in the form of questions. The knowledge point extraction rules can be exemplified as: Now you are a knowledge point extraction expert, please summarize what the various knowledge points introduced in the following text are, and each knowledge point is expressed in the form of a question.

[0021] Taking the above first information as an example, the knowledge points extracted by the first language model have at least: 1. In which year was the Western Han Dynasty usurped by Wang Mang? 2. How many years did the Western Han Dynasty last? 3. In which year was the Xin Dynasty established by Wang Mang announced? 4. In which year did Liu Bang become emperor in Chang'an? 5. Which dynasty was the Western Han Dynasty established after? 6. What was the name of the Western Han Dynasty? 7. Where was the capital of the Western Han Dynasty? 8. Which feudal unified dynasty is the Western Han Dynasty in Chinese history? It can be seen that through this embodiment, starting from the knowledge points introduced in the first information, the text processing capabilities of the first language model can be used to extract the various knowledge points introduced in the first information from the first information, thereby achieving the effect of accurately performing fine-grained analysis of the first information according to the knowledge point dimension.

[0022] In the above step S106, for each knowledge point in the first information, at least one second information for introducing the knowledge point is retrieved. In one embodiment, for any knowledge point, retrieving and obtaining at least one second information for introducing the knowledge point includes: Retrieving the knowledge point from the knowledge base to obtain multiple search results for introducing the knowledge point; If the content of the knowledge point has a time-limited attribute, then at least one second information corresponding to the knowledge point is obtained based on each search result according to the generation time of the first information, the publication time of each search result and the credibility of each search result; the publication time of the second information corresponding to the knowledge point is equal to or earlier than the generation time of the first information; If the content of the knowledge point does not have a time-limited attribute, then at least one second information corresponding to the knowledge point is obtained based on each search result according to the credibility of each search result.

[0023] This embodiment has one or more knowledge bases, each of which can be a real-time knowledge base. The information stored in each knowledge base meets the requirements of content authority and content timeliness. In each knowledge base, authoritative and real-time latest news can be retrieved for a certain issue.

[0024] Based on the real-time and authority of the content in the knowledge base, in this embodiment, for each knowledge point in the first information, at least one second information for introducing the knowledge point can be obtained based on the knowledge base retrieval. It can be understood that the second information is real-time and authoritative information for introducing the knowledge point.

[0025] Taking any one knowledge point in the first information as an example, the specific process of acquiring at least one second information for introducing the knowledge point based on knowledge base retrieval is described in detail below.

[0026] In this embodiment, first, the knowledge point is retrieved from one or more knowledge bases, and the knowledge point can be in the form of a question, and multiple search results for introducing the knowledge point are obtained. For example, "In which year was the Western Han Dynasty usurped by Wang Mang?" is searched in three knowledge bases, and the search results for answering the question in the three knowledge bases are obtained, and 100 search results are obtained for each knowledge base, and a total of 300 search results are obtained.

[0027] Then, determine whether the content of the knowledge point has a time-limited attribute. The content of the knowledge point has a time-limited attribute, which means that the content of the knowledge point will change over time, and the content of the knowledge point does not have a time-limited attribute, which means that the content of the knowledge point will not change over time. For example, for the knowledge point "Where is the capital of the Western Han Dynasty?", the content of this knowledge point is solidified historical knowledge and will not change over time. For the knowledge point "How many medals have our athletes won in the current sports meeting?", the content of this knowledge point is not fixed and will change over time.

[0028] If the content of the knowledge point has a time-limited attribute, then at least one second information corresponding to the knowledge point is obtained based on each search result according to the generation time of the first information, the publication time of each search result, and the credibility of each search result. Considering that the second information is used to evaluate the accuracy of the first information, and the accuracy of the first information requires reference to the latest content of the knowledge point when the first information is generated, and cannot refer to the latest content of the knowledge point after the first information is generated (because the latest content of the knowledge point after the first information is generated does not exist when the first information is generated), therefore, in this embodiment, the publication time of the second information corresponding to the knowledge point is configured to be equal to or earlier than the generation time of the first information, so as to use the second information to reasonably evaluate the accuracy of the first information.

[0029] If the content of the knowledge point does not have a time-limited attribute, then at least one second information corresponding to the knowledge point is obtained based on each search result according to the credibility of each search result. When the knowledge point does not have a time-limited attribute, the publication time of the second information corresponding to the knowledge point can be later than the generation time of the first information, can be equal to the generation time of the first information, or can be earlier than the generation time of the first information, and there is no limitation here.

[0030] It can be seen that through this embodiment, it is possible to determine whether the content of the knowledge point has a timeliness attribute. When the content of the knowledge point has a timeliness attribute, at least one second information corresponding to the knowledge point is obtained based on each search result according to the generation time of the first information, the publication time of each search result and the credibility of each search result. The publication time of the second information corresponding to the knowledge point is equal to or earlier than the generation time of the first information. When the content of the knowledge point does not have a timeliness attribute, at least one second information corresponding to the knowledge point is obtained based on each search result according to the credibility of each search result, thereby distinguishing different situations and accurately obtaining the second information corresponding to the knowledge point.

[0031] In one embodiment, for any knowledge point with a time-limited attribute, according to the generation time of the first information, the publication time of each search result and the credibility of each search result, obtaining at least one second information corresponding to the knowledge point based on each search result includes: Screening each search result according to the credibility of each search result to obtain at least one first target result; Among the first target results, select at least one second target result whose publishing time is equal to or earlier than the generation time of the first information; The second target result is summarized in content, and based on the summarized information, the second information corresponding to the knowledge point is generated; the second target result corresponds to the second information one-to-one.

[0032] In this embodiment, first, each search result corresponding to the knowledge point in the knowledge base is obtained, and the credibility of each search result is determined. The credibility of each search result can be determined by a method in the relevant technology, for example, the credibility of the search result is determined according to the source website of the search result. The credibility of the search result can be represented by a credibility value, which will not be described here.

[0033] Next, the search results are screened according to the credibility of each search result to obtain at least one first target result. For example, the search results are sorted according to the credibility of each search result from high to low, and the first plurality of search results in the sorting are selected as the first target result, for example, the top 10 search results after sorting according to the credibility from high to low are selected as the first target result, thereby obtaining at least one first target result.

[0034] Next, among the first target results, at least one second target result whose publication time is equal to or earlier than the generation time of the first information is selected according to the publication time of the first target result and the generation time of the first information. For example, the first target results are sorted from back to front according to the publication time of each first target result. In this sorting, five first target results whose publication time is equal to or earlier than the generation time of the first information and whose publication time is closest to the generation time of the first information are selected as the second target results, thereby obtaining five second target results.

[0035] Finally, the content of each second target result is summarized, and based on the summarized information, the second information corresponding to the knowledge point is generated, wherein the second target result corresponds to the second information one by one. For example, the content of a second target result is summarized through a generative model, and the summarized content is used as a second information corresponding to the knowledge point, or the content of a second target result is summarized through a generative model, and the summarized content is merged with the publication time of the second target result as a second information corresponding to the knowledge point. The publication time of the second target result can also be extracted from the knowledge base or from the second target result through a generative model. In this embodiment, the publication time of the retrieval result, the publication time of the first target result, and the publication time of the second target result can be the time when the corresponding result is published to the knowledge base, such as the time when it is published to the network.

[0036] In this embodiment, a suitable prompt word can be input into the generative model so that the generative model has the function of summarizing the content of the second target result and extracting the publication time of the second target result. The publication time of the second target result is the publication time of the second information.

[0037] It can be seen that through this embodiment, it is possible to combine multiple factors such as the credibility of the search results, the publication time of the search results, the generation time of the first information, etc., to accurately obtain one or more second information for knowledge points with timeliness attributes, and the second information is information that accurately introduces the knowledge point.

[0038] In one embodiment, for any knowledge point without a time-limited attribute, obtaining at least one second information corresponding to the knowledge point based on each search result according to the credibility of each search result includes: Screening each search result according to the credibility of each search result to obtain at least one third target result; The third target result is summarized in content, and based on the summarized information, the second information corresponding to the knowledge point is generated; the third target result corresponds to the second information one-to-one.

[0039] In this embodiment, first, each search result corresponding to the knowledge point in the knowledge base is obtained, and the credibility of each search result is determined. The credibility of each search result can be determined by a method in the relevant technology, for example, the credibility of the search result is determined according to the source website of the search result. The credibility of the search result can be represented by a credibility value, which will not be described here.

[0040] Next, the search results are screened according to the credibility of each search result to obtain at least one third target result. For example, the search results are sorted according to the credibility of each search result from high to low, and the first plurality of search results in the sorting are selected as the third target result, for example, the top 10 search results after sorting according to the credibility from high to low are selected as the third target result, thereby obtaining at least one third target result.

[0041] Finally, the content of each third target result is summarized, and based on the summarized information, the second information corresponding to the knowledge point is generated, wherein the third target result corresponds to the second information one by one. For example, the content of a third target result is summarized through a generative model, and the summarized content is used as a second information corresponding to the knowledge point, or the content of a third target result is summarized through a generative model, and the summarized content is merged with the publication time of the third target result as a second information corresponding to the knowledge point. The publication time of the third target result can also be extracted from the knowledge base or from the third target result through a generative model. In this embodiment, the publication time of the retrieval result, the publication time of the first target result, the publication time of the second target result, and the publication time of the third target result can be the time when the corresponding result is published to the knowledge base, such as the time when it is published to the network.

[0042] In this embodiment, a suitable prompt word can be input into the generative model so that the generative model has the function of summarizing the content of the third target result and extracting the publication time of the third target result. The publication time of the third target result is the publication time of the second information.

[0043] It can be seen that through this embodiment, it is possible to combine the credibility of the search results to accurately obtain one or more second information for knowledge points that do not have timeliness attributes, and the second information is information that accurately introduces the knowledge point.

[0044] Through the above process, the second information corresponding to each knowledge point in the first information can be obtained, and one knowledge point corresponds to one or more second information. The publishing time of the second information is also the publishing time of the second target result or the third target result mentioned above.

[0045] Next, in the above step S108, the accuracy of the first information is evaluated according to each piece of second information corresponding to each knowledge point. In one embodiment, the accuracy of the first information is evaluated according to each piece of second information corresponding to each knowledge point, including: dividing the first information into at least one information segment; For each information segment, determining, in each second information corresponding to each knowledge point, at least one target information for describing the target knowledge point introduced by the information segment; For each information segment, the accuracy of the information segment is evaluated using each target information corresponding to the information segment.

[0046] In this embodiment, first, the first information is divided into at least one information segment. The first information can be divided into at least one information segment using the period in the first information as a dividing mark. For example, for the previous first information: "The time range of the Western Han Dynasty is from 202 BC to 8 AD, a total of 210 years. Wang Mang usurped the throne at the end of 8 AD and announced the establishment of a new dynasty, marking the demise of the Western Han Dynasty. In 202 BC, Liu Bang proclaimed himself emperor in Chang'an and established the Han Dynasty, known in history as the Western Han Dynasty. The Western Han Dynasty was the second feudal unified dynasty after the Qin Dynasty, and the country was named Han", the following information segments can be obtained: 1. The Western Han Dynasty lasted from 202 BC to 8 AD, a total of 210 years.

[0047] 2. Wang Mang usurped the throne at the end of 8 AD and announced the establishment of the Xin Dynasty, marking the end of the Western Han Dynasty.

[0048] 3. In 202 BC, Liu Bang proclaimed himself emperor in Chang'an and established the Han Dynasty, known in history as the Western Han Dynasty. The Western Han Dynasty was the second feudal unified dynasty after the Qin Dynasty, and the country was named Han.

[0049] Then, for each information segment, in each second information corresponding to each knowledge point, at least one target information for describing the target knowledge point introduced by the information segment is determined. In this embodiment, the first information is parsed according to the dimension of the knowledge point, and one or more corresponding second information is obtained for each knowledge point. In this embodiment, the first information is also sliced ​​to obtain one or more information segments, so it can be known that each information segment may contain one or more knowledge points. Therefore, in this embodiment, for each information segment, the target knowledge point introduced by the information segment is determined, and in each second information corresponding to each knowledge point, the second information for introducing the target knowledge point is determined as the target information corresponding to the information segment, thereby obtaining at least one target information corresponding to the information segment.

[0050] Finally, for each information segment, the accuracy of the information segment is evaluated using each target information corresponding to the information segment.

[0051] It can be seen that through this embodiment, the first information can be divided into at least one information segment. For each information segment, in each second information corresponding to each knowledge point, at least one target information used to describe the target knowledge point introduced by the information segment is determined, and the accuracy of the information segment is evaluated using each target information corresponding to the information segment, thereby using each second information corresponding to each knowledge point to evaluate the accuracy of the first information, thereby achieving the effect of efficiently and accurately evaluating the accuracy of the first information and improving the precision of evaluating the accuracy of the information.

[0052] In one embodiment, taking any information segment as an example, in each second information corresponding to each knowledge point, at least one target information for describing the target knowledge point introduced by the information segment is determined, including: Determine, among each piece of second information corresponding to each knowledge point, at least one piece of second information whose content similarity with the information segment meets the similarity requirement; The determined second information is used as target information for describing the target knowledge point introduced by the information segment.

[0053] In this embodiment, taking any information segment as an example, among the second information corresponding to each knowledge point, at least one second information whose content similarity with the information segment meets the similarity requirement is determined, and each second information is used as a target information to obtain at least one target information for describing the target knowledge point introduced by the information segment.

[0054] In one example, the text vector of the information segment is determined, as well as the text vectors of each second information, the cosine distance between the text vector of the information segment and the text vector of each second information is calculated, and multiple second information with the closest cosine distances are selected as the target information corresponding to the information segment.

[0055] When determining the text vector of the information fragment, first split the information fragment into individual words and remove invalid characters in the information fragment. Then, use a word segmenter to convert the above words into a coding sequence that can be understood by the language processing model, for example, convert "hello" into "001". Then, input the converted coding sequence into the language processing model, and output the text vectors of the above words through the language processing model. The text vectors of the above words are combined into the text vector of the information fragment. The output of the specified position can be extracted from the hidden layer of the language processing model as the text vector of the above word. The text vector of the information fragment is usually obtained by using the last layer of the language processing model or a pooling operation (such as average pooling or maximum pooling).

[0056] When determining the text vector of the second information, first split the second information into individual words, and remove invalid characters in the second information. Then, use a word segmenter to convert the above words into a coding sequence that can be understood by the language processing model, for example, convert "hello" into "001". Then, input the converted coding sequence into the language processing model, and output the text vector of the above word through the language processing model. The text vectors of the above word are combined into the text vector of the second information. The output of the specified position can be extracted from the hidden layer of the language processing model as the text vector of the above word. The text vector of the second information is usually obtained by using the last layer of the language processing model or a pooling operation (such as average pooling or maximum pooling).

[0057] It can be seen that through this embodiment, the content similarity between the second information and the information segment can be utilized to determine one or more target information corresponding to the information segment in each second information corresponding to each knowledge point, thereby accurately obtaining the target information describing the target knowledge point introduced by the information segment based on the content similarity.

[0058] In one embodiment, taking any information segment as an example, the accuracy of the information segment is evaluated using each target information corresponding to the information segment, including: The target information corresponding to the information fragment is spliced ​​to obtain the detection sample corresponding to the information fragment; If the content of the information fragment has a time-sensitive attribute, the timeliness and authenticity of the information fragment are evaluated using the test sample and the release time of the test sample; If the content of the information fragment does not have a time-limited attribute, the authenticity of the information fragment is evaluated using the detection sample.

[0059] In this embodiment, taking any information segment as an example, first, each target information corresponding to the information segment is spliced ​​to obtain a detection sample corresponding to the information segment. The target information may carry the publication time of the target information or may not carry the publication time of the target information. The publication time of the target information is the publication time of the second information. In one example, the information segment corresponds to 5 target information. These 5 target information are spliced ​​in a text splicing manner to obtain a detection sample corresponding to the information segment. The detection sample carries the publication time of each target information.

[0060] Then, determine whether the content of the information segment has a time-limited attribute. The content of the information segment has a time-limited attribute, which means that the content of the information segment will change over time. The content of the information segment does not have a time-limited attribute, which means that the content of the information segment will not change over time. For example, for the information segment "The capital of the Western Han Dynasty was in Chang'an", the content of this information segment is solidified historical knowledge and will not change over time. For the information segment "Our athletes won 33 medals in the current sports meeting", the content of this information segment is not fixed and will change over time.

[0061] In this embodiment, if the content of the information segment has a timeliness attribute, the accuracy of the information segment is determined to include authenticity and timeliness, and the timeliness and authenticity of the information segment are evaluated using the detection sample and the publication time of the detection sample. The publication time of the detection sample may be carried in the inspection sample or may not be carried in the detection sample, and the publication time of the detection sample includes the publication time of each target information constituting the detection sample. If the content of the information segment does not have a timeliness attribute, the accuracy of the information segment is determined to include authenticity, and the authenticity of the information segment is evaluated using the detection sample. Among them, the authenticity of the information segment is evaluated to evaluate whether the content of the information segment is authentic. The timeliness of the information segment is evaluated to evaluate whether the content of the information segment is the latest content.

[0062] In one example, for an information segment with a timeliness attribute, when both the timeliness and authenticity of the information segment pass the evaluation, it can be determined that the accuracy of the information segment has passed the evaluation; when the timeliness or authenticity of the information segment fails the evaluation, it can be determined that the accuracy of the information segment has failed the evaluation; for an information segment that does not have a timeliness attribute, when the authenticity of the information segment passes the evaluation, it can be determined that the accuracy of the information segment has passed the evaluation; when the authenticity of the information segment fails the evaluation, it can be determined that the accuracy of the information segment has failed the evaluation.

[0063] It can be seen that through this embodiment, the various target information corresponding to the information segment can be spliced ​​together to obtain the detection sample corresponding to the information segment, and according to whether the content of the information segment has a time attribute, the detection sample can be used to perform different types of evaluations on the information segment, and the information segment can be evaluated based on different situations, thereby improving the accuracy of the evaluation of the accuracy of the information segment.

[0064] In one embodiment, for information segments with timeliness attributes, the timeliness and authenticity of the information segments are evaluated using the detection samples and the release time of the detection samples, including: By using the second language model, determining whether there is a contradiction between the content in the test sample and the content in the information fragment, and determining whether the information fragment contains specific content in the test sample; the specific content is the content in the test sample whose publication time is equal to or earlier than the generation time of the first information and is closest to the generation time of the first information; Based on the judgment results, determine whether the timeliness and authenticity of the information fragment have passed the evaluation.

[0065] In this embodiment, a second language model is provided, and the second language model may be the above-mentioned LLM large language model. Through the second language model, it is determined whether there is a contradiction between the content in the detection sample and the content in the information segment. The contradiction means that the description of the same object in the detection sample and the content in the information segment is inconsistent. For example, if the detection sample describes "Xi'an is located in Shaanxi" and the information segment describes "Xi'an is located in Shanxi", it is determined that the content in the detection sample and the content in the information segment are inconsistent.

[0066] And, through the second language model, determine specific content in the test sample, the specific content is the content in the test sample whose publication time is equal to or earlier than the generation time of the first information and is closest to the generation time of the first information. Through the second language model, determine whether the specific content exists in the information segment. For example, there are three contents in the test sample: as of 5:20 when the information is published, we have won 33 medals; as of 5:19 when the information is published, we have won 30 medals; as of 5:18 when the information is published, we have won 29 medals. If the generation time of the first information is 5:20, it can be determined that the specific content in the test sample is: as of 5:20 when the information is published, we have won 33 medals, and then determine whether the specific content exists in the information segment.

[0067] In this embodiment, a suitable prompt word can be input for the second language model, so that the second language model has the ability to determine whether there is a contradiction between the content in the detection sample and the content in the information segment, and has the ability to determine whether the information segment contains specific content in the detection sample. The prompt word can be exemplified as: "Now you are a contradiction detection and timeliness detection model. You need to determine whether the following first paragraph of text contains content that contradicts the following second paragraph of text, and determine whether the following first paragraph of text contains specific content in the following second paragraph of text. The specific content is the content in the second paragraph of text that is published at or before the specified time and is closest to the specified time. The specified time is XXX".

[0068] In this embodiment, based on the judgment result, it is determined whether the timeliness and authenticity of the information fragment have passed the evaluation.

[0069] It can be seen that through this embodiment, for information fragments with timeliness attributes, the timeliness and authenticity of the information fragments can be evaluated, thereby improving the accuracy of evaluating the accuracy of the information fragments.

[0070] In one embodiment, determining whether the timeliness and authenticity of the information fragment have passed the evaluation according to the judgment result includes: If the judgment result indicates that there is no contradiction between the content in the test sample and the content in the information fragment, it is determined that the authenticity of the information fragment has passed the evaluation; if the judgment result indicates that there is a contradiction between the content in the test sample and the content in the information fragment, it is determined that the authenticity of the information fragment has failed the evaluation; If the judgment result indicates that the information segment contains the above-mentioned specific content, then it is determined that the timeliness of the information segment has passed the evaluation; if the judgment result indicates that the information segment does not contain the above-mentioned specific content, then it is determined that the timeliness of the information segment has not passed the evaluation.

[0071] In this embodiment, if the judgment result indicates that there is no contradiction between the content in the detection sample and the content in the information segment, then it is determined that the authenticity of the information segment has passed the evaluation; if the judgment result indicates that there is a contradiction between the content in the detection sample and the content in the information segment, then it is determined that the authenticity of the information segment has not passed the evaluation.

[0072] For example, the information fragment is: Xi'an is located in Henan and is a very prosperous city in my country. The test sample is: Xi'an is located in Shaanxi and is the ancient capital of 13 dynasties, with scenic spots such as the Big Wild Goose Pagoda and Qujiang Lake. Therefore, it can be found that there is a contradiction in the description of Xi'an's location in the information fragment and the test sample. Therefore, the judgment result indicates that there is a contradiction between the content in the test sample and the content in the information fragment, and it is determined that the authenticity of the information fragment has not passed the evaluation.

[0073] In this embodiment, if the judgment result indicates that the information segment contains the above-mentioned specific content, it is determined that the timeliness of the information segment has passed the evaluation; if the judgment result indicates that the information segment does not contain the above-mentioned specific content, it is determined that the timeliness of the information segment has not passed the evaluation.

[0074] For example, the information segment is: As of 5:19, we have won 30 medals. The test samples are: As of 5:20 when the information is published, we have won 33 medals; As of 5:19 when the information is published, we have won 30 medals; As of 5:18 when the information is published, we have won 29 medals. If the generation time of the first information is 5:20, it can be determined that the specific content in the test sample is: As of 5:20 when the information is published, we have won 33 medals. Furthermore, it is determined that the specific content does not exist in the information segment, and it is determined that the timeliness of the information segment has not passed the evaluation.

[0075] It can be seen that through this embodiment, it is possible to judge whether the timeliness and authenticity of the information fragment have passed the evaluation respectively. When the timeliness and authenticity of the information fragment have passed the evaluation, it can be determined that the accuracy of the information fragment has passed the evaluation. When the timeliness or authenticity of the information fragment has not passed the evaluation, it can be determined that the accuracy of the information fragment has not passed the evaluation. From the two aspects of authenticity and timeliness, the accuracy of evaluating the accuracy of the information fragment is improved.

[0076] In one embodiment, for an information segment without a time-limited attribute, the authenticity of the information segment is evaluated using a detection sample, including: Through the third language model, determine whether there is a contradiction between the content in the test sample and the content in the information fragment; If there is no contradiction between the content in the test sample and the content in the information fragment, it is determined that the authenticity of the information fragment has passed the evaluation. If there is a contradiction between the content in the test sample and the content in the information fragment, it is determined that the authenticity of the information fragment has failed the evaluation.

[0077] In this embodiment, a third language model is provided, and the third language model may be the above-mentioned LLM large language model. Through the third language model, it is determined whether there is a contradiction between the content in the detection sample and the content in the information segment. The contradiction means that the description of the same object in the detection sample and the content in the information segment is inconsistent. For example, if the detection sample describes "Xi'an is located in Shaanxi" and the information segment describes "Xi'an is located in Shanxi", it is determined that the content in the detection sample and the content in the information segment are inconsistent.

[0078] In one example, the information segment is: Xi'an is located in Shaanxi and is a very prosperous city in my country. The test sample is: Xi'an is located in Shaanxi and is the ancient capital of 13 dynasties, with scenic spots such as Big Wild Goose Pagoda and Qujiang Lake. Therefore, it can be found that although the content of the information segment is less than that of the test sample, there is no contradictory content in the information segment and the test sample. Therefore, the judgment result indicates that there is no contradiction between the content in the test sample and the content in the information segment, and the authenticity of the information segment is determined to pass the evaluation.

[0079] It can be seen that through this embodiment, it is possible to determine whether the authenticity of the information segment has passed the evaluation. When the authenticity of the information segment has passed the evaluation, it can be determined that the accuracy of the information segment has passed the evaluation, thereby improving the accuracy of the evaluation of the accuracy of the information segment.

[0080] In step S108, after obtaining the evaluation results of each information segment, the evaluation results of each information segment are also summarized to obtain the evaluation results of the first information. For example, the first information has three information segments, which are numbered 1, 2, and 3. After the above process, it is determined that information segment 1 has a timeliness attribute, and the authenticity and timeliness of information segment 1 have passed the evaluation. It is determined that information segment 2 does not have a timeliness attribute, and the authenticity of information segment 2 has passed the evaluation. It is determined that information segment 3 has a timeliness attribute, and the authenticity of information segment 3 has passed the evaluation, but the timeliness has not passed the evaluation. Furthermore, the evaluation results of the first information are summarized as follows: the authenticity of the first information has passed the evaluation, and the timeliness has not passed the evaluation. The reason for not passing is that information segment 3 does not contain XXXX content.

[0081] In this embodiment, the evaluation result of the first information can also be used to feed back the training of the target model to improve the accuracy of the target model.

[0082] It can be seen that through this embodiment, the first information can be evaluated at a fine-grained level (information fragment granularity), thereby achieving the effect of efficiently and accurately evaluating the accuracy of the first information and improving the accuracy of evaluating the accuracy of the information.

[0083] Figure 2 A schematic diagram of the structure of a model evaluation device provided in an embodiment of the present disclosure is shown in FIG. Figure 2 As shown, the device comprises: A first acquisition unit 21 is used to acquire first information, where the first information is the answer information of the target model to the first question; A content extraction unit 22, configured to extract at least one knowledge point from the first information; A second acquisition unit 23 is used to retrieve and acquire, for each of the knowledge points, at least one second information used to introduce the knowledge point; The content evaluation unit 24 is used to evaluate the accuracy of the first information according to each piece of the second information corresponding to each of the knowledge points.

[0084] Optionally, the content extraction unit 22 is specifically configured to: At least one knowledge point is extracted from the first information through the first language model based on a knowledge point extraction rule.

[0085] Optionally, the second acquiring unit 23 is specifically configured to: Retrieving the knowledge point from a knowledge base to obtain a plurality of retrieval results for introducing the knowledge point; If the content of the knowledge point has a time-limited attribute, then according to the generation time of the first information, the publication time of each of the search results and the credibility of each of the search results, at least one of the second information corresponding to the knowledge point is obtained based on each of the search results; the publication time of the second information corresponding to the knowledge point is equal to or earlier than the generation time of the first information; If the content of the knowledge point does not have a time-limited attribute, then at least one second information corresponding to the knowledge point is obtained based on each of the search results according to the credibility of each of the search results.

[0086] Optionally, the second acquiring unit 23 is further specifically configured to: Screening each of the search results according to the credibility of each of the search results to obtain at least one first target result; Among the first target results, select at least one second target result whose publishing time is equal to or earlier than the generation time of the first information; The second target result is summarized in content, and based on the summarized information, the second information corresponding to the knowledge point is generated; the second target result corresponds to the second information one-to-one.

[0087] Optionally, the second acquiring unit 23 is further specifically configured to: Screening each of the search results according to the credibility of each of the search results to obtain at least one third target result; The third target result is summarized in content, and based on the summarized information, the second information corresponding to the knowledge point is generated; the third target result corresponds to the second information one-to-one.

[0088] Optionally, the content evaluation unit 24 is specifically used for: dividing the first information into at least one information segment; For each of the information segments, determining, in each of the second information corresponding to each of the knowledge points, at least one target information for describing the target knowledge point introduced by the information segment; For each of the information segments, the accuracy of the information segment is evaluated using each of the target information corresponding to the information segment.

[0089] Optionally, the content evaluation unit 24 is further specifically configured to: Determine, among each of the second information corresponding to each of the knowledge points, at least one of the second information whose content similarity with the information segment meets the similarity requirement; The determined second information is used as target information for describing the target knowledge point introduced by the information segment.

[0090] Optionally, the content evaluation unit 24 is further specifically configured to: Splicing the target information corresponding to the information fragments to obtain detection samples corresponding to the information fragments; If the content of the information fragment has a timeliness attribute, the timeliness and authenticity of the information fragment are evaluated using the detection sample and the publication time of the detection sample; If the content of the information fragment does not have a time-limited attribute, the authenticity of the information fragment is evaluated using the detection sample.

[0091] Optionally, the content evaluation unit 24 is further specifically configured to: By using the second language model, determining whether there is a contradiction between the content in the detection sample and the content in the information fragment, and determining whether the information fragment contains specific content in the detection sample; the specific content is content in the detection sample whose publishing time is equal to or earlier than the generation time of the first information and is closest to the generation time of the first information; Based on the judgment result, it is determined whether the timeliness and authenticity of the information fragment have passed the evaluation.

[0092] Optionally, the content evaluation unit 24 is further specifically configured to: If the judgment result indicates that there is no contradiction between the content in the detection sample and the content in the information segment, it is determined that the authenticity of the information segment has passed the evaluation; if the judgment result indicates that there is a contradiction between the content in the detection sample and the content in the information segment, it is determined that the authenticity of the information segment has not passed the evaluation; If the judgment result indicates that the information segment contains the specific content, it is determined that the timeliness of the information segment has passed the evaluation; if the judgment result indicates that the information segment does not contain the specific content, it is determined that the timeliness of the information segment has not passed the evaluation.

[0093] Optionally, the content evaluation unit 24 is further specifically configured to: By using a third language model, determining whether there is a contradiction between the content in the detection sample and the content in the information fragment; If there is no contradiction between the content in the detection sample and the content in the information segment, it is determined that the authenticity of the information segment has passed the evaluation; if there is a contradiction between the content in the detection sample and the content in the information segment, it is determined that the authenticity of the information segment has failed the evaluation.

[0094] The model evaluation device in the embodiment of the present disclosure can implement each process of the above-mentioned model evaluation method embodiment and achieve the same effect and function, which will not be repeated here.

[0095] An embodiment of the present disclosure further provides an electronic device, Figure 3 A schematic diagram of the structure of an electronic device provided by an embodiment of the present disclosure, such as Figure 3 As shown, the electronic device may have relatively large differences due to different configurations or performances, and may include one or more processors 301 and memory 302, and one or more applications or data may be stored in the memory 302. Among them, the memory 302 may be a temporary storage or a permanent storage. The application stored in the memory 302 may include one or more modules (not shown in the figure), and each module may include a series of computer executable instructions in the electronic device. Furthermore, the processor 301 may be configured to communicate with the memory 302 to execute a series of computer executable instructions in the memory 302 on the electronic device. The electronic device may also include one or more power supplies 303, one or more wired or wireless network interfaces 304, one or more input or output interfaces 305, one or more keyboards 306, etc.

[0096] In a specific embodiment, the electronic device includes a processor; and a memory configured to store computer executable instructions, wherein when the computer executable instructions are executed, the processor implements the following process: Acquire first information, where the first information is answer information of the target model to the first question; Extracting at least one knowledge point from the first information; For each of the knowledge points, retrieve and obtain at least one second information for introducing the knowledge point; The accuracy of the first information is evaluated according to each piece of the second information corresponding to each of the knowledge points.

[0097] The electronic device in the embodiment of the present disclosure can implement each process of the above-mentioned model evaluation method embodiment and achieve the same effects and functions, which will not be repeated here.

[0098] Another embodiment of the present disclosure further provides a computer-readable storage medium, wherein the computer-readable storage medium is used to store computer-executable instructions, and when the computer-executable instructions are executed by a processor, the following process is implemented: Acquire first information, where the first information is answer information of the target model to the first question; Extracting at least one knowledge point from the first information; For each of the knowledge points, retrieve and obtain at least one second information for introducing the knowledge point; The accuracy of the first information is evaluated according to each piece of the second information corresponding to each of the knowledge points.

[0099] The computer-readable storage medium in the embodiment of the present disclosure can implement each process of the above-mentioned model evaluation method embodiment and achieve the same effects and functions, which will not be repeated here.

[0100] Another embodiment of the present disclosure further provides a computer program product, the computer program product comprising a computer program, and when the computer program is executed by a processor, the following process is implemented: Acquire first information, where the first information is answer information of the target model to the first question; Extracting at least one knowledge point from the first information; For each of the knowledge points, retrieve and obtain at least one second information for introducing the knowledge point; The accuracy of the first information is evaluated according to each piece of the second information corresponding to each of the knowledge points.

[0101] The computer program product in the embodiments of the present disclosure can implement the various processes of the above-mentioned model evaluation method embodiments and achieve the same effects and functions, which will not be repeated here.

[0102] In various embodiments of the present disclosure, the computer-readable storage medium includes a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0103] In the 1990s, it was very clear whether the improvement of a technology was hardware improvement (for example, improvement of the circuit structure of diodes, transistors, switches, etc.) or software improvement (improvement of the method flow). However, with the development of technology, many improvements of the method flow today can be regarded as direct improvements of the hardware circuit structure. Designers almost always obtain the corresponding hardware circuit structure by programming the improved method flow into the hardware circuit. Therefore, it cannot be said that the improvement of a method flow cannot be implemented with a hardware entity module. For example, a programmable logic device (PLD) (such as a field programmable gate array (FPGA)) is such an integrated circuit whose logical function is determined by the user's programming of the device. Designers can "integrate" a digital system on a PLD by programming themselves, without having to ask chip manufacturers to design and make dedicated integrated circuit chips. Moreover, nowadays, instead of manually making integrated circuit chips, this kind of programming is mostly implemented by "logic compiler" software, which is similar to the software compiler used when developing and writing programs, and the original code before compilation must also be written in a specific programming language, which is called hardware description language (HDL). There is not only one kind of HDL, but many kinds, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, RHDL (Ruby Hardware Description Language), etc. The most commonly used ones are VHDL (Very-High-Speed ​​Integrated Circuit Hardware Description Language) and Verilog. Those skilled in the art should also know that it is only necessary to program the method flow slightly in the above-mentioned hardware description languages ​​and program it into the integrated circuit, and then it is easy to obtain the hardware circuit that implements the logic method flow.

[0104] The controller may be implemented in any suitable manner, for example, the controller may take the form of a microprocessor or processor and a computer-readable medium storing a computer-readable program code (such as software or firmware) executable by the (micro)processor, a logic gate, a switch, an application-specific integrated circuit (ASIC), a programmable logic controller, and an embedded microcontroller. Examples of controllers include, but are not limited to, the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicone Labs C8051F320. The memory controller may also be implemented as part of the control logic of the memory. It is also known to those skilled in the art that, in addition to implementing the controller in a purely computer-readable program code manner, the controller may be implemented in the form of a logic gate, a switch, an application-specific integrated circuit, a programmable logic controller, and an embedded microcontroller by logically programming the method steps. Therefore, such a controller may be considered as a hardware component, and the devices for implementing various functions included therein may also be considered as structures within the hardware component. Or even, the devices for implementing various functions may be considered as both software modules for implementing the method and structures within the hardware component.

[0105] The systems, devices, modules or units described in the above embodiments may be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, the computer may be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

[0106] For the convenience of description, the above devices are described in terms of functions and are divided into various units. Of course, when implementing the embodiments of the present disclosure, the functions of each unit can be implemented in the same or multiple software and / or hardware.

[0107] It should be understood by those skilled in the art that one or more embodiments of the present disclosure may be provided as a method, system or computer program product. Therefore, one or more embodiments of the present disclosure may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, one or more embodiments of the present disclosure may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0108] The present disclosure is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present disclosure. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of the processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0109] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0110] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0111] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0112] One or more embodiments of the present disclosure may be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. One or more embodiments of the present disclosure may also be practiced in distributed computing environments in which tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules may be located in local and remote computer storage media, including storage devices.

[0113] Each embodiment in the present disclosure is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0114] The above description is only an embodiment of the present disclosure and is not intended to limit the present disclosure. For those skilled in the art, the present disclosure may have various modifications and variations. Any modification, equivalent substitution, improvement, etc. made within the spirit and principle of the present disclosure shall be included in the scope of the claims of the present disclosure.

Claims

1. A model evaluation method, characterized in that: include: Acquire first information, where the first information is answer information of the target model to the first question; Extracting at least one knowledge point from the first information; For each of the knowledge points, retrieve and obtain at least one second information for introducing the knowledge point; The accuracy of the first information is evaluated according to each piece of the second information corresponding to each of the knowledge points.

2. The method according to claim 1, characterized in that The extracting at least one knowledge point from the first information includes: At least one knowledge point is extracted from the first information through the first language model based on a knowledge point extraction rule.

3. The method according to claim 1, characterized in that: The retrieving and acquiring at least one second information for introducing the knowledge point includes: Retrieving the knowledge point from a knowledge base to obtain a plurality of retrieval results for introducing the knowledge point; If the content of the knowledge point has a time-limited attribute, then according to the generation time of the first information, the publication time of each of the search results and the credibility of each of the search results, at least one of the second information corresponding to the knowledge point is obtained based on each of the search results; the publication time of the second information corresponding to the knowledge point is equal to or earlier than the generation time of the first information; If the content of the knowledge point does not have a time-limited attribute, then at least one second information corresponding to the knowledge point is obtained based on each of the search results according to the credibility of each of the search results.

4. The method according to claim 3, characterized in that The acquiring, based on each of the search results, at least one of the second information corresponding to the knowledge point according to the generation time of the first information, the publication time of each of the search results, and the credibility of each of the search results, includes: Screening each of the search results according to the credibility of each of the search results to obtain at least one first target result; Among the first target results, select at least one second target result whose publishing time is equal to or earlier than the generation time of the first information; The second target result is summarized in content, and based on the summarized information, the second information corresponding to the knowledge point is generated; the second target result corresponds to the second information one-to-one.

5. The method according to claim 3, characterized in that: The acquiring, based on the credibility of each of the search results, at least one of the second information corresponding to the knowledge point, comprises: Screening each of the search results according to the credibility of each of the search results to obtain at least one third target result; The third target result is summarized in content, and based on the summarized information, the second information corresponding to the knowledge point is generated; the third target result corresponds to the second information one-to-one.

6. The method according to claim 1, characterized in that The step of evaluating the accuracy of the first information according to each piece of the second information corresponding to each piece of the knowledge point includes: dividing the first information into at least one information segment; For each of the information segments, determining, in each of the second information corresponding to each of the knowledge points, at least one target information for describing the target knowledge point introduced by the information segment; For each of the information segments, the accuracy of the information segment is evaluated using each of the target information corresponding to the information segment.

7. The method according to claim 6, characterized in that Determining, in each of the second information corresponding to each of the knowledge points, at least one target information for describing the target knowledge point introduced by the information segment includes: Determine, among each of the second information corresponding to each of the knowledge points, at least one of the second information whose content similarity with the information segment meets the similarity requirement; The determined second information is used as target information for describing the target knowledge point introduced by the information segment.

8. The method according to claim 6, characterized in that The step of evaluating the accuracy of the information fragments by using the target information corresponding to the information fragments includes: Splicing the target information corresponding to the information fragments to obtain detection samples corresponding to the information fragments; If the content of the information fragment has a timeliness attribute, the timeliness and authenticity of the information fragment are evaluated using the detection sample and the publication time of the detection sample; If the content of the information fragment does not have a time-limited attribute, the authenticity of the information fragment is evaluated using the detection sample.

9. The method according to claim 8, characterized in that The use of the test sample and the release time of the test sample to evaluate the timeliness and authenticity of the information fragment includes: By using the second language model, determining whether there is a contradiction between the content in the detection sample and the content in the information fragment, and determining whether the information fragment contains specific content in the detection sample; the specific content is content in the detection sample whose publishing time is equal to or earlier than the generation time of the first information and is closest to the generation time of the first information; Based on the judgment result, it is determined whether the timeliness and authenticity of the information fragment have passed the evaluation.

10. The method according to claim 9, characterized in that Determining whether the timeliness and authenticity of the information fragment have passed the evaluation according to the judgment result includes: If the judgment result indicates that there is no contradiction between the content in the detection sample and the content in the information segment, it is determined that the authenticity of the information segment has passed the evaluation; if the judgment result indicates that there is a contradiction between the content in the detection sample and the content in the information segment, it is determined that the authenticity of the information segment has not passed the evaluation; If the judgment result indicates that the information segment contains the specific content, it is determined that the timeliness of the information segment has passed the evaluation; if the judgment result indicates that the information segment does not contain the specific content, it is determined that the timeliness of the information segment has not passed the evaluation.

11. The method according to claim 8, characterized in that The using the detection sample to evaluate the authenticity of the information fragment includes: By using a third language model, determining whether there is a contradiction between the content in the detection sample and the content in the information fragment; If there is no contradiction between the content in the detection sample and the content in the information segment, it is determined that the authenticity of the information segment has passed the evaluation; if there is a contradiction between the content in the detection sample and the content in the information segment, it is determined that the authenticity of the information segment has failed the evaluation.

12. A model evaluation device, characterized in that: include: A first acquisition unit, configured to acquire first information, where the first information is answer information of the target model to the first question; A content extraction unit, configured to extract at least one knowledge point from the first information; A second acquisition unit, configured to retrieve and acquire, for each of the knowledge points, at least one second information for introducing the knowledge point; The content evaluation unit is used to evaluate the accuracy of the first information according to each piece of the second information corresponding to each of the knowledge points.

13. An electronic device, characterized in that: include: processor; as well as, A memory configured to store computer executable instructions, which, when executed, cause the processor to implement the steps of the method according to any one of claims 1 to 11.

14. A computer-readable storage medium, characterized in that: The computer-readable storage medium is used to store computer-executable instructions, and the computer-executable instructions, when executed by a processor, implement the steps of the method described in any one of claims 1 to 11.

15. A computer program product, characterized in that The computer program product comprises a computer program, and when the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 11 are implemented.

Images