Model evaluation method and device, equipment, medium and product

Abstract

The invention relates to the technical field of computers, and discloses a model evaluation method and device, equipment, a medium and a product, and the method comprises the steps: obtaining an evaluation data set of a to-be-evaluated model, the evaluation data set comprises prompt information, a reference answer and real-time data, and the real-time data is obtained based on real-time retrieval; based on the prompt information and the reference answer, extracting an evaluation keyword; retrieving the real-time data according to the evaluation keyword, and determining a target answer matched with the evaluation keyword; and determining a detection result of the to-be-evaluated model based on the reference answer and the target answer. By implementing the technical scheme of the invention, identification and screening can be carried out for the security problem with high timeliness, the problem of insufficient timeliness in model security assessment is solved, and the capability of processing the timeliness content by the model is enhanced, so that the output of the model is ensured to be safe at any given moment.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technology, and in particular to a model evaluation method, device, equipment, medium and product. Background Art

[0002] With the increase of computing resources and the improvement of various algorithms, large language models have been able to learn language rules by analyzing large amounts of text data, enabling them to generate fluent text, answer questions, and write code, and have been able to demonstrate human-level performance in a variety of tasks. However, this ability of the model is based on its training data, and the bad features of the training data will affect the training of the model, making it difficult for the trained model to effectively identify potential risks, which can easily lead to problems. Summary of the invention

[0003] In view of this, the present disclosure provides a model evaluation method, device, equipment, medium and product to solve the problem that the model has difficulty in identifying inappropriate answers, resulting in poor model recognition effect.

[0004] In a first aspect, the present disclosure provides a model evaluation method, comprising: obtaining an evaluation data set of the model to be evaluated, the evaluation data set including prompt information, reference answers and real-time data, the real-time data being obtained based on real-time retrieval; extracting evaluation keywords based on the prompt information and the reference answers; retrieving the real-time data according to the evaluation keywords, and determining a target answer that matches the evaluation keywords; and determining a detection result of the model to be evaluated based on the reference answers and the target answers.

[0005] In a second aspect, the present disclosure provides a model evaluation device, comprising: an acquisition module, used to obtain an evaluation data set of the model to be evaluated, the evaluation data set including prompt information, reference answers and real-time data, and the real-time data is obtained based on real-time retrieval; an extraction module, used to extract evaluation keywords based on prompt information and reference answers; a retrieval module, used to retrieve real-time data according to the evaluation keywords, and determine a target answer that matches the evaluation keywords; a security assessment module, used to determine the detection result of the model to be evaluated based on the reference answer and the target answer.

[0006] In a third aspect, the present disclosure provides a computer device, comprising: a memory and a processor, the memory and the processor are communicatively connected to each other, computer instructions are stored in the memory, and the processor executes the model evaluation method of the first aspect or any corresponding embodiment thereof by executing the computer instructions.

[0007] In a fourth aspect, the present disclosure provides a computer-readable storage medium having computer instructions stored thereon, the computer instructions being used to enable a computer to execute the model evaluation method of the first aspect or any corresponding embodiment thereof.

[0008] In a fifth aspect, the present disclosure provides a computer program product, including computer instructions, which are used to enable a computer to execute the model evaluation method of the above-mentioned first aspect or any corresponding embodiment.

[0009] The evaluation method, device, equipment, medium and product of the model provided by the present disclosure introduces real-time data with high timeliness into the evaluation data set, extracts evaluation keywords according to prompt information and reference answers, and retrieves real-time data according to the evaluation keywords to determine the target answers that match the evaluation keywords, so that the detection results of the model can be determined in combination with the reference answers and the target answers to evaluate whether the model has recognition problems. Therefore, by constructing a problem evaluation data set with high timeliness and introducing real-time data in the evaluation process of the model, it is possible to identify and screen problems with high timeliness, making up for the problem of insufficient timeliness in model evaluation, enabling the evaluation to adapt to and respond to new content and information more quickly, and enhancing the model's ability to process timely content, thereby ensuring that the output of the model is accurate at any given moment. By reducing the lag in information reception and processing during the evaluation process, the response speed and accuracy of the model in emergency situations are effectively improved, and the evaluation loopholes caused by data lag are reduced. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] In order to more clearly illustrate the specific embodiments of the present disclosure or the technical solutions in the prior art, the drawings required for use in the specific embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0011] Figure 1 is a schematic diagram of manual evaluation in the related art according to an embodiment of the present disclosure;

[0012] Figure 2 is a schematic diagram of a method for evaluating a model according to an embodiment of the present disclosure;

[0013] Figure 3 is a flowchart of a method for evaluating a model according to an embodiment of the present disclosure;

[0014] Figure 4 is a flow chart of another model evaluation method according to an embodiment of the present disclosure;

[0015] Figure 5 is a flow chart of another model evaluation method according to an embodiment of the present disclosure;

[0016] Figure 6is a structural block diagram of an evaluation device of a model according to an embodiment of the present disclosure;

[0017] Figure 7 It is a schematic diagram of the hardware structure of the computer device of the embodiment of the present disclosure. DETAILED DESCRIPTION

[0018] In order to make the purpose, technical solution and advantages of the embodiments of the present disclosure clearer, the technical solution in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are part of the embodiments of the present disclosure, rather than all the embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present disclosure.

[0019] The ability of large language models to generate fluent text, answer questions, and write code is based on their training data. If the training data contains bias, misleading information, or sensitive content, the model may also learn and replicate these undesirable characteristics, resulting in problems with the model's output.

[0020] Currently, the evaluation of models mainly relies on manual evaluation, such as Figure 1 As shown. When evaluating the model, we first collect the data set to be evaluated, which covers a wide range of issues, such as inaccurate information. Then, two manual groups evaluate the same batch of data and indicate whether there are any problems. For questions with different opinions, we explain the opinions separately, unify the results, align the opinions, and randomly check the questions that have been evaluated according to a certain proportion to verify their correctness. Then, we adjust and train the model according to the test results to improve its capabilities.

[0021] However, when evaluating issues with high timeliness, the above model evaluation scheme cannot effectively identify potential risks, as the model output is based on the data pattern during its training period rather than real-time data, and may therefore produce inappropriate answers. And due to the lag in receiving real-time information, this lag may cause related problems.

[0022] Based on this, the embodiments of the present disclosure provide a systematic solution to the above problems. Specifically, Figure 2As shown in the figure, the workflow is used to decompose the model evaluation task with high complexity into multiple subtasks, including structured processing subtasks, evaluation keyword extraction subtasks, real-time data matching subtasks, etc., and then multiple subtasks are connected in series, which helps to improve the confidence of each link and improve the final accuracy. When an abnormality occurs, each subtask can be located for easy and timely resolution. At the same time, the real-time retrieval plug-in is deployed to retrieve real-time data through the real-time retrieval plug-in, and real-time data with high timeliness is added to the evaluation data set, which significantly improves the response quality and accuracy of the large language model, and also greatly enhances the application effect of the model in a real-time environment.

[0023] According to an embodiment of the present disclosure, an embodiment of a model evaluation method is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0024] In this embodiment, a model evaluation method is provided, which can be used for computer equipment, such as computers, servers, etc. Figure 3 is a flow chart of a method for evaluating a model according to an embodiment of the present disclosure, such as Figure 3 As shown, the process includes the following steps:

[0025] Step S101, obtaining an evaluation data set of the model to be evaluated, wherein the evaluation data set includes prompt information, reference answers and real-time data, and the real-time data is obtained based on real-time retrieval.

[0026] The model to be evaluated is the intelligent question-answering model that needs to be evaluated, such as a large language model. The evaluation dataset is a pre-collected dataset of questions covering various categories, which may include rumors, bias, data privacy, violence, abuse, fraud, and inaccurate information data. Each category can contain multiple subcategories so that the evaluation dataset can comprehensively evaluate the performance of the model in different scenarios.

[0027] The prompt information is guidance or clues provided to the model to be evaluated, so that the model to be evaluated provides an answer according to the prompt information. Specifically, the prompt information can be description information of the question, relevant background information, examples, or other forms of information. The prompt information is not specifically limited here.

[0028] The reference answer is the correct answer or expected answer that the model to be evaluated should give based on the prompt information. In the process of evaluating the model, the prompt information and the reference answer will be used together. The model to be evaluated will answer the question based on the prompt information, and then compare the answer obtained from the question with the reference answer to determine the accuracy of the answer.

[0029] In the process of evaluating the model, by using prompt information and reference answers, the model's performance in terms of comprehension ability, knowledge level, thinking ability and expression ability can be evaluated.

[0030] In a specific implementation, the computer device may obtain the evaluation data set from a public data set, locally collected data, or other relevant sources, and then determine the prompt information and reference answer according to the parameters of the model to be evaluated and the task to be achieved.

[0031] Real-time data is retrieved by computer devices in real time. Specifically, a real-time search plug-in is deployed in the computer device to retrieve the current hot content through the real-time search plug-in to obtain data with high timeliness, such as the latest news hot spots, new fraud methods, new legal updates, etc.

[0032] By collecting real-time data in the evaluation data set, we can more comprehensively evaluate the performance of the model to be evaluated when processing data with high timeliness, such as the correct understanding of news hotspots, the ability to identify and prevent new fraud methods, and the ability to adapt to new legal updates. As a result, the model to be evaluated can better reflect its accuracy and reliability in real-world scenarios.

[0033] Step S102: extracting evaluation keywords based on the prompt information and reference answers.

[0034] Evaluation keywords are used to represent the key information contained in the prompt information and reference answers. Specifically, the prompt information and reference answers are combined into a text, and the text information is analyzed to extract keywords related to the text topic.

[0035] In some specific implementations, the frequency of occurrence of each word in the text can be counted, and the words with higher frequency of occurrence can be determined as keywords; the text can be tagged with parts of speech to extract meaningful part-of-speech keywords such as nouns, verbs, and adjectives; the text can also be semantically analyzed using natural language processing technology to extract text keywords; and keywords can also be extracted from the text using decision trees, support vector machines, and other methods. The method for extracting keywords is not specifically limited here, and those skilled in the art can determine it according to the usage scenario.

[0036] Step S103, searching the real-time data according to the evaluation keyword to determine the target answer that matches the evaluation keyword.

[0037] The target answer is the matching answer obtained by searching the real-time data based on the evaluation keyword. After the evaluation keyword is determined, the information matching the evaluation keyword can be searched in the real-time data based on the exact matching, fuzzy matching, semantic understanding, etc. of the keyword, and the matched information can be analyzed or inferred to extract the target answer that is most relevant to the evaluation keyword.

[0038] Step S104: Determine the detection result of the model to be evaluated based on the reference answer and the target answer.

[0039] Compare the reference answer with the target answer to determine whether there is information deviation between the reference answer and the target answer. When there is no problem with the model's output, the semantics of the reference answer and the target answer are consistent. Therefore, by comparing the reference answer and the target answer, it can be determined whether the model has an inappropriate answer.

[0040] In the above implementation, the model evaluation process is divided into multiple subtasks such as collecting evaluation data sets, extracting evaluation keywords, and matching target answers, which helps to improve the confidence of each link, thereby improving the evaluation accuracy of the model. At the same time, when an abnormality occurs, it can be located to the corresponding subtask for timely resolution. In this way, the automated evaluation of the model can be achieved in combination with the workflow.

[0041] The evaluation method of the model provided in this embodiment introduces real-time data with high timeliness into the evaluation data set, extracts evaluation keywords according to prompt information and reference answers, and retrieves real-time data according to the evaluation keywords to determine the target answers that match the evaluation keywords, so that it can be determined whether the model has inappropriate answers in combination with the reference answers and the target answers. Therefore, by constructing a problem evaluation data set with high timeliness and introducing real-time data in the security evaluation process of the model, it is possible to identify and screen problems with high timeliness, making up for the problem of insufficient timeliness in model evaluation, enabling the evaluation to adapt to and respond to new content and information more quickly, and enhancing the model's ability to process timely content, thereby ensuring that the output of the model is accurate at any given moment. By reducing the lag in information reception and processing during the evaluation process, the response speed and accuracy of the model in emergency situations are effectively improved, and the evaluation loopholes caused by data lag are reduced.

[0042] In this embodiment, a model evaluation method is provided, which can be used for computer equipment, such as computers, servers, etc. Figure 4 is a flow chart of a method for evaluating a model according to an embodiment of the present disclosure, such as Figure 4 As shown, the process includes the following steps:

[0043] Step S201, obtain the evaluation data set of the model to be evaluated. The evaluation data set includes prompt information, reference answers and real-time data, and the real-time data is obtained based on real-time retrieval. For details, please refer to Figure 3 Step S101 of the illustrated embodiment will not be described in detail here.

[0044] Step S202: extracting evaluation keywords based on the prompt information and the reference answers.

[0045] Specifically, the above step S202 includes:

[0046] Step S2021, structure the prompt information and reference answers to generate structured data consisting of the prompt information and reference answers.

[0047] After determining the prompt information and reference answers from the evaluation data set, first determine the structured data format for storing the prompt information and reference answers, which can be a data frame, table, JSON object or other suitable data structure. Then, organize and normalize the prompt information and reference answers to ensure the consistency and accuracy of the data, for example, convert the prompt information and reference answers into a unified format, remove special characters or spaces, etc. Then, use the data structure operation in the programming language to organize and store the organized prompt information and reference answers according to the pre-set data structure.

[0048] Taking JSON object as an example, the prompt information and reference answer are extracted as question and expected answer respectively, and the prompt information and reference answer are processed into a string containing keys and values. The string constitutes the structured data of the prompt information and reference answer, thereby realizing the structured processing of the prompt information and reference answer.

[0049] Step S2022: extracting evaluation keywords from the structured data.

[0050] Analyze the structured data composed of prompt information and reference answers to determine the position or field where the evaluation keyword may appear in the structured data. For example, if the structured data is a table, the evaluation keyword may appear in a specific column; if it is JSON data, the evaluation keyword may appear in a specific key.

[0051] Then, the evaluation keywords are extracted according to the format of the structured data. In some specific implementations, if the evaluation keywords have specific patterns or rules, regular expressions can be used to match and extract them; string methods and functions can also be used to extract evaluation keywords, such as string operations such as search, split, and replace; for specific data formats, such as XML, JSON, etc., corresponding parsing libraries can also be used to extract evaluation keywords; for structured data in text form, natural language processing technology can also be used to extract keywords. After the evaluation keywords are extracted, the evaluation keywords are verified and cleaned to ensure their accuracy and consistency, such as removing duplicate keywords and processing special characters.

[0052] In some optional implementations, the above method further includes:

[0053] Step a1, checking whether the evaluation keywords are extracted successfully.

[0054] Step a2: when the evaluation keyword extraction fails, the evaluation keyword is re-extracted from the structured data based on a preset retry strategy.

[0055] The preset retry strategy is a pre-set keyword extraction strategy, which is activated when the evaluation keyword output fails. After the evaluation keyword extraction is completed, the evaluation keyword extraction result can be analyzed to determine whether the evaluation keyword extraction is successful. If the evaluation keyword extraction fails, the evaluation keyword can be re-extracted according to the retry strategy.

[0056] It should be noted that if an error occurs during the evaluation keyword extraction process, an error message will be printed, and the evaluation keyword will be re-extracted according to the number of retries and delay time set by the retry strategy. If the number of retries reaches the upper limit, an empty output result will be returned, that is, the extraction result output of the evaluation keyword is empty.

[0057] In the above implementation, when the evaluation keyword extraction fails, re-extraction is performed to overcome the instability of the model, thereby avoiding the influence of the evaluation keyword extraction failure on the model output.

[0058] In some optional implementations, the above method further includes: filling the extracted evaluation keywords into the structured data.

[0059] When the evaluation keywords are extracted successfully, the extracted keywords are filled into the structured data formed by the prompt information and the reference answers.

[0060] In the above implementation, by filling keywords in the structured data having prompt information and reference answers, the prompt information and reference answers can be effectively combined to retrieve the real-time data, so as to achieve effective processing and analysis of the real-time data.

[0061] Step S203: Retrieve real-time data according to the evaluation keyword to determine the target answer that matches the evaluation keyword. Figure 3 Step S103 of the illustrated embodiment will not be described in detail here.

[0062] Step S204: Determine the detection result of the model to be evaluated based on the reference answer and the target answer. Figure 3 Step S104 of the illustrated embodiment will not be described in detail here.

[0063] The model evaluation method provided in this embodiment performs structured processing on prompt information and reference answers to extract evaluation keywords from structured data, thereby facilitating more accurate extraction of evaluation keywords and helping to improve the evaluation accuracy of the model.

[0064] In this embodiment, a model evaluation method is provided, which can be used for computer equipment, such as computers, servers, etc. Figure 5 is a flow chart of a method for evaluating a model according to an embodiment of the present disclosure, such as Figure 5 As shown, the process includes the following steps:

[0065] Step S301, obtain the evaluation data set of the model to be evaluated. The evaluation data set includes prompt information, reference answers and real-time data, and the real-time data is obtained based on real-time retrieval. For details, please refer to Figure 4 Step S201 of the illustrated embodiment will not be described in detail here.

[0066] Step S302: Extract evaluation keywords based on prompt information and reference answers. Figure 4 Step S202 of the illustrated embodiment will not be described in detail here.

[0067] Step S303: Retrieve real-time data according to the evaluation keyword to determine a target answer that matches the evaluation keyword.

[0068] Specifically, the above step S303 includes:

[0069] Step S3031, querying real-time data according to the evaluation keyword to determine candidate data containing the evaluation keyword.

[0070] On the basis of determining the evaluation keywords, the real-time data is filtered according to the evaluation keywords to eliminate the real-time data irrelevant to the evaluation keywords, thereby obtaining filtered data, and determining the filtered data as candidate data.

[0071] In some specific implementations, if the evaluation data set is stored in a database, a SQL query statement can be used to filter out real-time data containing evaluation keywords from the evaluation data set. For example, a fuzzy match is performed using the `LIKE` operator. Of course, string matching, regular expressions, etc. can also be used to filter real-time data containing evaluation keywords, which is not specifically limited here and can be determined by those skilled in the art according to actual needs.

[0072] Step S3032, associate and match the evaluation keyword with the candidate data to obtain a target answer that matches the evaluation keyword.

[0073] The evaluation keyword is compared with the text in the candidate data to determine the correlation between the two, and the candidate data with a higher correlation is determined as the target answer corresponding to the evaluation keyword.

[0074] Specifically, a string matching algorithm such as exact matching, fuzzy matching or regular expression matching may be used here to compare the evaluation keyword with the text in the candidate data to determine a target answer that matches the evaluation keyword.

[0075] Specifically, natural language processing technology is used to perform semantic analysis and understanding (such as lexical analysis, syntactic analysis, semantic role labeling, etc.) on the evaluation keywords and candidate data to determine the semantic relationship between the evaluation keywords and the candidate data, and determine the target answer that matches the evaluation keywords based on the semantic relationship. This may include determining the semantic relationship between them.

[0076] Specifically, a classification model or a matching model can be pre-built, such as using labeled training data to learn the association pattern between evaluation keywords and target answers. Then, after the model training is completed, the evaluation keywords can be input into the pre-trained model, and the model outputs the target answer that matches the evaluation keywords.

[0077] Specifically, a knowledge graph can be constructed to associate evaluation keywords and candidate data with entities and relationships in the knowledge graph. By querying and reasoning about the knowledge graph, target answers related to the evaluation keywords can be found in the candidate data.

[0078] Of course, other methods may be used to determine the target answer that matches the evaluation keyword, which is not specifically limited here and can be determined by those skilled in the art according to actual needs.

[0079] Step S304: Determine the detection result of the model to be evaluated based on the reference answer and the target answer.

[0080] Specifically, the above step S304 includes:

[0081] Step S3041: When the semantic expressions of the reference answer and the target answer are consistent, it is determined that the model to be evaluated has passed the detection.

[0082] When the semantic expression of the reference answer is consistent with that of the target answer, it means that the semantics expressed by the reference answer and the target answer are the same. At this time, it can be temporarily determined that the model does not have any inappropriate answers, that is, the model to be evaluated has passed the test. Specifically, when the semantics expressed by the reference answer and the target answer are the same, it means that the data used to train and evaluate the model is accurate, complete and reliable. If the data is biased, wrong or incomplete, it may cause the model to produce inaccurate results.

[0083] Since the security of the model is a dynamic issue that requires continuous monitoring and updating, as new data or new situations emerge, the model may need to be adjusted and improved to ensure that the reference answer has the same semantics as the target answer to ensure its security.

[0084] Step S3042: When the semantic expressions of the reference answer and the target answer are different, it is determined that the model to be evaluated fails the detection.

[0085] When the semantic expression of the reference answer is different from that of the target answer, that is, there is a deviation in the semantics expressed by the reference answer and the target answer, it means that the data used to train and evaluate the model is biased, wrong, or incomplete. At this time, it can be indicated that the model to be evaluated has an inappropriate answer problem, and further investigation and analysis are needed to determine the root cause and impact of the problem.

[0086] Specifically, when it is determined that there is a deviation between the semantics expressed by the reference answer and the target answer, it is necessary to check whether there are errors or deviations in the data used to train the model to ensure the quality and accuracy of the data and to eliminate inconsistencies caused by data problems. At the same time, compare the performance of the model to be evaluated in different data sets or scenarios to determine whether there are anomalies under specific conditions, and carefully analyze the possible causes of the deviation by reviewing the input data, the model's prediction process and results.

[0087] In some optional implementations, the above method further includes: adjusting the model to be evaluated based on the detection results, and retraining the adjusted model to be evaluated.

[0088] Analyze the test results of the model to be evaluated to locate the location where the model to be evaluated has inappropriate answers. Adjust the parameters, training data, feature engineering, model architecture, etc. of the model to be evaluated based on the test results to improve the performance and security of the model.

[0089] Retraining is performed using the adjusted model to help the model to be evaluated learn new knowledge and patterns, thereby improving the performance of identifying security issues. After retraining, the model to be evaluated is monitored to ensure its reliability in practical applications. At the same time, it is evaluated to determine whether the retrained model to be evaluated still has security issues.

[0090] In the above implementation, the model is adjusted and trained in combination with the detection results of the model to improve the problem identification and processing capabilities of the model.

[0091] In some optional implementations, the above method further includes:

[0092] Step c1, based on the detection results, determine the feedback information of the model to be evaluated.

[0093] The feedback information is used to provide feedback on the answer effect of the model to be evaluated on the questions raised. Specifically, the feedback information may include problem description, input data, expected output data, etc. The feedback information can be obtained through error reporting, log analysis, system monitoring, etc.

[0094] After obtaining the test results, we can identify the feedback information currently existing in the model to be evaluated, such as data leakage, model attacks, misclassification, etc., and evaluate the severity of the problems existing in the model. At the same time, we can determine the scope of the impact of the current problem on the model, such as whether it is a local problem or an overall problem, and the degree of impact on specific users or data sets. Then, we analyze the potential risks that may be brought about by the current problem, such as infringement of user privacy, threats to system stability, and the consequences of wrong decisions.

[0095] Step c2: Update the feedback information to the evaluation data set, and use the updated evaluation data set to adjust the model to be evaluated.

[0096] The collected feedback information is collated and annotated to ensure data quality and consistency, and the feedback data in the feedback information is converted into a format suitable for model training and evaluation. The annotated feedback data is added to the existing evaluation data set by merging data sets or performing incremental learning. Subsequently, the model is adjusted and improved using the updated evaluation data set, including adjusting model parameters, adding training data, improving feature engineering, etc.

[0097] Use the updated evaluation dataset to evaluate the retrained model to be evaluated, and compare the performance of the retrained model to be evaluated on new problems and scenarios with the previous one to determine whether the model has been optimized.

[0098] Deploy the optimized model to be evaluated to the actual scenario and continuously monitor its performance. Repeat the above steps after collecting new feedback data to achieve continuous optimization. By establishing a feedback loop, the model can continuously learn and improve from actual applications, adapt to new problems and scenarios, and improve its performance and reliability. At the same time, timely processing of feedback data and making adjustments can ensure that the model always maintains good performance.

[0099] In the above implementation, the feedback information of the model is determined in combination with the detection results of the model on security issues, so that new problems and scenarios collected from actual applications can be systematically added to the training and evaluation data sets, so as to facilitate continuous optimization of the performance of the model.

[0100] The model evaluation method provided in this embodiment uses the evaluation keyword to query the real-time data to retrieve the real-time information associated with the evaluation keyword from the real-time data, thereby enhancing the model's ability to accurately process content that is highly time-sensitive and changes rapidly, thereby enhancing the user's trust in the model in terms of security and reliability. Combined with the semantics expressed by the reference answer and the target answer, it is determined whether the model has problems, so that when it is determined that the model has problems, the model can be adjusted and optimized in a timely manner.

[0101] In the present embodiment, a model evaluation device is also provided, which is used to implement the above-mentioned embodiments and preferred embodiments, and the descriptions thereof will not be repeated. As used below, the term "module" can implement a combination of software and / or hardware of a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, the implementation of hardware, or a combination of software and hardware is also possible and contemplated.

[0102] This embodiment provides a model evaluation device, such as Figure 6 As shown, including:

[0103] The acquisition module 501 is used to acquire the evaluation data set of the model to be evaluated. The evaluation data set includes prompt information, reference answers and real-time data. The real-time data is obtained based on real-time retrieval.

[0104] The extraction module 502 is used to extract evaluation keywords based on the prompt information and the reference answers.

[0105] The search module 503 is used to search the real-time data according to the evaluation keyword and determine the target answer that matches the evaluation keyword.

[0106] The security assessment module 504 is used to determine the detection result of the model to be assessed based on the reference answer and the target answer.

[0107] In some optional implementations, the extraction module 502 includes:

[0108] The structured processing unit is used to perform structured processing on the prompt information and the reference answer to generate structured data consisting of the prompt information and the reference answer.

[0109] The extraction unit is used to extract evaluation keywords from structured data.

[0110] In some optional implementations, the extraction module 502 further includes:

[0111] The detection unit is used to detect whether the evaluation keywords are extracted successfully.

[0112] The retry unit is used to re-extract the evaluation keywords from the structured data based on a preset retry strategy when the evaluation keyword extraction fails.

[0113] In some optional implementations, the extraction module 502 further includes:

[0114] The filling unit is used to fill the extracted evaluation keywords into the structured data.

[0115] In some optional implementations, the retrieval module 503 includes:

[0116] The query unit is used to query the real-time data according to the evaluation keyword and determine the candidate data containing the evaluation keyword.

[0117] The association unit is used to associate and match the evaluation keywords with the candidate data to obtain a target answer that matches the evaluation keywords.

[0118] In some optional implementations, the security assessment module 504 includes:

[0119] The first determination unit is used to determine that the model to be evaluated has passed the detection when the semantic expressions of the reference answer and the target answer are consistent.

[0120] The second determination unit is used to determine that the model to be evaluated fails the detection when the semantic expressions of the reference answer and the target answer are different.

[0121] In some optional embodiments, the above device further includes:

[0122] The first adjustment module is used to adjust the model to be evaluated based on the detection results of the security issues, and retrain the adjusted model to be evaluated.

[0123] In some optional embodiments, the above device further includes:

[0124] The feedback module is used to determine feedback information of the model to be evaluated based on the detection results of the security issues.

[0125] The second adjustment module is used to update the feedback information into the evaluation data set and adjust the model to be evaluated using the updated evaluation data set.

[0126] The further functional description of each of the above modules and units is the same as that of the above corresponding embodiments and will not be repeated here.

[0127] The evaluation device of the model in this embodiment is presented in the form of a functional unit, where the unit refers to an ASIC (Application Specific Integrated Circuit), a processor and memory that executes one or more software or fixed programs, and / or other devices that can provide the above functions.

[0128] The evaluation device of the model provided in this embodiment introduces real-time data with high timeliness into the evaluation data set, extracts evaluation keywords according to prompt information and reference answers, and retrieves real-time data according to the evaluation keywords to determine the target answers that match the evaluation keywords, so that the model detection results can be determined in combination with the reference answers and the target answers. Therefore, by constructing an evaluation data set with high-timeliness security issues and introducing real-time data in the evaluation process of the model, it is possible to identify and screen issues with high timeliness, making up for the problem of insufficient timeliness in the model evaluation process, so that the evaluation can adapt to and respond to new content and information more quickly, and enhance the model's ability to process timely content, thereby ensuring that the output of the model is accurate at any given moment. By reducing the lag in information reception and processing during the evaluation process, the response speed and accuracy of the model in emergency situations are effectively improved, and the evaluation loopholes caused by data lag are reduced.

[0129] The present disclosure also provides a computer device having Figure 6 Evaluation device of the model shown.

[0130] See also Figure 7 , Figure 7 is a schematic diagram of a computer device provided by an optional embodiment of the present disclosure, such as Figure 7As shown, the computer device includes: one or more processors 10, a memory 20, and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. Various components are connected to each other using different buses for communication, and can be installed on a common mainboard or installed in other ways as needed. The processor can process the instructions executed in the computer device, including instructions stored in or on the memory to display the graphical information of the GUI on an external input / output device (such as, a display device coupled to the interface). In some optional embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories and multiple memories. Similarly, multiple computer devices can be connected, and each device provides some necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system). Figure 7 A processor 10 is taken as an example.

[0131] The processor 10 may be a central processing unit, a network processor or a combination thereof. The processor 10 may further include a hardware chip. The hardware chip may be a dedicated integrated circuit, a programmable logic device or a combination thereof. The programmable logic device may be a complex programmable logic device, a field programmable gate array, a general purpose array logic or any combination thereof.

[0132] The memory 20 stores instructions executable by at least one processor 10, so that the at least one processor 10 executes the method shown in the above embodiment.

[0133] The memory 20 may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created according to the use of the computer device, etc. In addition, the memory 20 may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some optional embodiments, the memory 20 may optionally include a memory remotely arranged relative to the processor 10, and these remote memories may be connected to the computer device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0134] The memory 20 may include a volatile memory, such as a random access memory; the memory may also include a non-volatile memory, such as a flash memory, a hard disk or a solid state drive; the memory 20 may also include a combination of the above types of memory.

[0135] The computer device also includes an input device 30 and an output device 40. The processor 10, the memory 20, the input device 30 and the output device 40 may be connected via a bus or other means. Figure 7 The example of connecting through bus is taken in the following.

[0136] The input device 30 can receive input digital or character information, and generate key signal input related to the user settings and function control of the computer device, such as a touch screen, a keypad, a mouse, a track pad, a touch pad, an indicator bar, one or more mouse buttons, a trackball, a joystick, etc. The output device 40 may include a display device, an auxiliary lighting device (e.g., an LED) and a tactile feedback device (e.g., a vibration motor), etc. The above-mentioned display device includes but is not limited to a liquid crystal display, a light emitting diode, a display and a plasma display. In some optional embodiments, the display device can be a touch screen.

[0137] The computer device also includes a communication interface for the computer device to communicate with other devices or a communication network.

[0138] The embodiments of the present disclosure also provide a computer-readable storage medium. The above-mentioned method according to the embodiments of the present disclosure can be implemented in hardware, firmware, or can be implemented as a computer code that can be recorded in a storage medium, or can be implemented as a computer code that is originally stored in a remote storage medium or a non-temporary machine-readable storage medium and will be stored in a local storage medium and downloaded through a network, so that the method described herein can be stored in such software processing on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. Among them, the storage medium can be a magnetic disk, an optical disk, a read-only storage memory, a random access memory, a flash memory, a hard disk or a solid-state drive, etc.; further, the storage medium can also include a combination of the above-mentioned types of memory. It can be understood that a computer, a processor, a microprocessor controller, or programmable hardware includes a storage component that can store or receive software or computer code. When the software or computer code is accessed and executed by a computer, a processor, or hardware, the method shown in the above embodiment is implemented.

[0139] A part of the present disclosure may be applied as a computer program product, such as a computer program instruction, which, when executed by a computer, can call or provide the method and / or technical solution according to the present disclosure through the operation of the computer. Those skilled in the art should understand that the existence of computer program instructions in computer-readable media includes, but is not limited to, source files, executable files, installation package files, etc., and accordingly, the way in which computer program instructions are executed by a computer includes, but is not limited to: the computer directly executes the instruction, or the computer compiles the instruction and then executes the corresponding compiled program, or the computer reads and executes the instruction, or the computer reads and installs the instruction and then executes the corresponding installed program. Here, the computer-readable medium can be any available computer-readable storage medium or communication medium accessible to the computer.

[0140] Although the embodiments of the present disclosure have been described in conjunction with the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present disclosure, and such modifications and variations are all within the scope defined by the appended claims.

Claims

1. A model evaluation method, characterized in that: The method comprises: Obtaining an evaluation data set of the model to be evaluated, wherein the evaluation data set includes prompt information, reference answers, and real-time data, wherein the real-time data is obtained based on real-time retrieval; Extracting evaluation keywords based on the prompt information and reference answers; Retrieving real-time data according to the evaluation keyword to determine a target answer that matches the evaluation keyword; Based on the reference answer and the target answer, a detection result of the model to be evaluated is determined.

2. The method according to claim 1, characterized in that The step of extracting evaluation keywords based on the prompt information and the reference answer includes: Performing structural processing on the prompt information and the reference answer to generate structured data consisting of the prompt information and the reference answer; The evaluation keywords are extracted from the structured data.

3. The method according to claim 2, characterized in that Also includes: Detecting whether the evaluation keyword is extracted successfully; When the evaluation keyword extraction fails, the evaluation keyword is re-extracted from the structured data based on a preset retry strategy.

4. The method according to claim 2 or 3, characterized in that: Also includes: Fill the extracted evaluation keywords into the structured data.

5. The method according to claim 1, characterized in that The step of retrieving real-time data according to the evaluation keyword and determining a target answer matching the evaluation keyword includes: Querying the real-time data according to the evaluation keyword to determine candidate data containing the evaluation keyword; The keyword is associated and matched with the candidate data to obtain a target answer that matches the evaluation keyword.

6. The method according to claim 1, characterized in that The step of determining the detection result of the model to be evaluated based on the reference answer and the target answer includes: When the semantic expression of the reference answer is consistent with that of the target answer, it is determined that the model to be evaluated has passed the detection; When the semantic expressions of the reference answer and the target answer are different, it is determined that the model to be evaluated fails detection.

7. The method according to claim 1, characterized in that Also includes: Based on the detection result, the model to be evaluated is adjusted, and the adjusted model to be evaluated is retrained.

8. The method according to claim 1, characterized in that Also includes: Based on the detection result, determining feedback information of the model to be evaluated; The feedback information is updated to the evaluation data set, and the model to be evaluated is adjusted using the updated evaluation data set.

9. A model evaluation device, characterized in that: The device comprises: An acquisition module is used to acquire an evaluation data set of a model to be evaluated, wherein the evaluation data set includes prompt information, reference answers, and real-time data, wherein the real-time data is obtained based on real-time retrieval; An extraction module, used for extracting evaluation keywords based on the prompt information and reference answers; A retrieval module, used to retrieve real-time data according to the evaluation keyword and determine a target answer that matches the evaluation keyword; A security assessment module is used to determine the detection result of the model to be evaluated based on the reference answer and the target answer.

10. A computer device, characterized in that: include: A memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the evaluation method of the model described in any one of claims 1 to 8 by executing the computer instructions.

11. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a computer to execute the model evaluation method according to any one of claims 1 to 8.

12. A computer program product, characterized in that The method comprises computer instructions for causing a computer to execute the model evaluation method according to any one of claims 1 to 8.

Images