Model evaluation method, device, equipment, storage medium and program product

Abstract

Embodiments of the present disclosure provide a model evaluation method and device, equipment, a storage medium and a program product. The method comprises: inputting a test sample into a to-be-tested model, analyzing the test sample according to a target task by the to-be-tested model, and obtaining an execution result of the target task; based on the execution result of the target task, screening a plurality of target word units from the word units of the test sample; based on the plurality of target word units and the association relationship between the plurality of target word units, establishing a model inference chain output by the to-be-tested model for the test sample; and based on the alignment degree between the labeled inference chain and the model inference chain, obtaining an evaluation result of the to-be-tested model. The present disclosure screens the word units of the test sample based on the execution result of the target task to establish the model inference chain, realizes deep analysis of the inference logic of the to-be-tested model, changes the internal decision of the to-be-tested model into an interpretable model inference chain, and then facilitates accurate positioning of the fault point.

Description

Technical Field

[0001] This disclosure belongs to the field of text comprehension ability assessment technology, and in particular relates to a model assessment method, device, equipment, storage medium and program product. Background Technology

[0002] In existing technologies, text understanding evaluation schemes mainly rely on static benchmarks. This involves constructing static, standardized text datasets and using predefined evaluation metrics, such as ROUGE-1 and ROUGE-2, to quantitatively analyze the model's output. However, this evaluation scheme cannot effectively assess the model's performance in higher-order comprehension and logical reasoning abilities, exhibiting significant limitations and failing to provide a comprehensive and accurate evaluation of the model. Summary of the Invention

[0003] This disclosure provides a model evaluation method, apparatus, device, storage medium, and program product, which can deeply analyze the text understanding ability of the model under test and achieve comprehensive and accurate model evaluation.

[0004] In a first aspect, embodiments of this disclosure provide a model evaluation method, the method comprising: The test sample is input into the model under test, and the model under test analyzes the test sample according to the target task to obtain the execution result of the target task; Based on the execution results of the target task, multiple target lexical units are selected from the lexical units of the test sample; Based on multiple target words and the relationships between them, a model inference chain is established for the output of the model under test to the test sample. The evaluation results of the model under test are obtained based on the alignment between the labeled inference chain corresponding to the test sample and the model inference chain corresponding to the test sample.

[0005] In one feasible implementation, before inputting the test sample into the model under test, analyzing the test sample according to the target task through the model under test, and obtaining the execution result of the target task, the method further includes: Select the target perturbation strategy corresponding to the target perturbation level from multiple perturbation strategies; Data augmentation is performed on the original samples based on the target perturbation strategy and the semantic features of the original samples to obtain the initial test samples; If the response matching degree of the model under test to the initial test sample is less than the preset response matching degree threshold, the initial test sample is used as the test sample; where the response matching degree is used to characterize the degree of adaptation between the processing logic of the model under test and the initial test sample. If the response matching degree of the model under test to the initial test sample is greater than or equal to the preset response matching degree threshold, the next perturbation degree adjacent to the target perturbation degree is taken as the new target perturbation degree; based on the new target perturbation degree, the step of selecting the target perturbation strategy corresponding to the target perturbation degree from multiple perturbation strategies is returned.

[0006] In one feasible implementation, before using the initial test sample as the test sample when the response matching degree of the model under test to the initial test sample is less than a preset response matching degree threshold, the method further includes: The initial test sample is input into the model under test. The model under test analyzes the initial test sample according to the first objective task and obtains the execution result of the first objective task. The execution result includes the reasoning answer output by the model under test for the first objective task. Based on the semantic similarity between the initial test sample and the original sample, and the semantic similarity between the inferred answer and the standard answer corresponding to the original sample, the response matching degree of the model under test to the initial test sample is determined.

[0007] In one feasible implementation, the evaluation result of the model under test is obtained based on the alignment between the labeled inference chain corresponding to the test sample and the model inference chain corresponding to the test sample, including: Based on the number of historical initial test samples corresponding to the degree of disturbance of each historical target and the response matching degree of each historical initial test sample, the model robustness index of the model under test is determined. The evaluation results of the model under test are obtained based on the model robustness index of the model under test, the alignment degree between the labeled inference chain corresponding to the test sample and the model inference chain corresponding to the test sample.

[0008] In one feasible implementation, based on the execution result of the target task, multiple target lexical units are selected from the lexical units of the test sample, including: For each current word element in the test sample, determine the query vector based on the current word element, and determine the key vectors corresponding to the query vector based on all other words elements in the test sample except the current word element. For each query vector, calculate the degree of matching between the query vector and the corresponding key vectors to obtain the attention score; Based on the attention scores corresponding to each query vector, the tokens of the test samples are sorted to obtain the sorting results; Based on the ranking results and the execution results of the target task, multiple target lexical units are selected from the lexical units of the test sample.

[0009] In one feasible implementation, before obtaining the evaluation result of the model under test based on the alignment between the labeled inference chain corresponding to the test sample and the model inference chain corresponding to the test sample, the method further includes: Calculate the edit distance similarity between the labeled inference chain corresponding to the test sample and the model inference chain corresponding to the test sample; Calculate the semantic similarity between the labeled inference chain corresponding to the test sample and the model inference chain corresponding to the test sample; We calculate the alignment between the edit distance similarity and semantic similarity by weighted summation.

[0010] Secondly, embodiments of this disclosure provide a model evaluation apparatus, the apparatus comprising: The execution module is used to input test samples into the model under test, analyze the test samples according to the target task, and obtain the execution results of the target task. The filtering module is used to filter out multiple target words from the words in the test samples based on the execution results of the target task. A module is established to build a model inference chain for the output of the model under test for the test sample based on multiple target words and the relationships between them. The evaluation module is used to obtain the evaluation results of the model under test based on the alignment between the labeled inference chain corresponding to the test sample and the model inference chain corresponding to the test sample.

[0011] Thirdly, this disclosure provides a model evaluation device, which includes a processor and a memory storing computer program instructions; the processor reads and executes the computer program instructions to implement any of the model evaluation methods in the above embodiments.

[0012] Fourthly, this disclosure provides a computer storage medium storing computer program instructions, which, when executed by a processor, implement any of the model evaluation methods described in the above embodiments.

[0013] Fifthly, this disclosure provides a computer program product, including a computer program that, when executed by a processor, implements any of the model evaluation methods described in the above embodiments.

[0014] The model evaluation method, apparatus, device, storage medium, and program product disclosed herein can deeply analyze the text understanding ability of the model under test, achieving comprehensive and accurate model evaluation. This disclosure filters the lexical units of the test samples based on the execution results of the target task to establish a model inference chain during the process of the model under test executing the target task using the test samples. Compared to methods that calculate evaluation indicators based on model output results, this disclosure achieves a deep analysis of the inference logic of the model under test, transforming the internal decisions of the model under test into an interpretable model inference chain, thereby facilitating accurate location of fault points. Furthermore, by assessing the alignment between the model inference chain and the manually pre-annotated labeled inference chain, this disclosure can better quantify the performance changes of the model under test after fine-tuning, preventing the phenomenon of correct results but incorrect inference processes, and enhancing evaluation accuracy. Attached Figure Description

[0015] To more clearly illustrate the technical solutions of the embodiments of this disclosure, the accompanying drawings used in the embodiments of this disclosure will be briefly introduced below. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0016] Figure 1 This is a schematic flowchart of a model evaluation method provided in an embodiment of this disclosure; Figure 2 This is a schematic diagram of the structure of a model evaluation device provided in an embodiment of this disclosure; Figure 3 This is a schematic diagram of the structure of a model evaluation device provided in an embodiment of this disclosure. Detailed Implementation

[0017] The features and exemplary embodiments of various aspects of this disclosure will now be described in detail. To make the objectives, technical solutions, and advantages of this disclosure clearer, the following detailed description, in conjunction with the accompanying drawings and specific embodiments, will provide a further detailed description. It should be understood that the specific embodiments described herein are intended only to explain this disclosure and not to limit it. For those skilled in the art, this disclosure can be implemented without some of these specific details. The following description of the embodiments is merely to provide a better understanding of this disclosure by illustrating examples.

[0018] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising..." does not exclude the presence of additional identical elements in the process, method, article, or apparatus that includes said element.

[0019] The acquisition, storage, use, and processing of data in this application embodiment all comply with the relevant provisions of national laws and regulations.

[0020] In the embodiments of this application, certain software, components, models and other existing solutions in the industry may be mentioned. These should be regarded as exemplary and are only intended to illustrate the feasibility of implementing the technical solution of this application. However, they do not mean that the applicant has used or necessarily used the solution.

[0021] With the development of artificial intelligence technology, Large Language Models (LLMs) are playing an increasingly important role in fields such as intelligent customer service, medical diagnosis, and legal reasoning. Constructing a scientific and effective evaluation system for model understanding capabilities has become a core element in ensuring system reliability. Current mainstream text understanding evaluation schemes primarily construct static standardized text datasets (such as the LongBench long text understanding test set and the LCSTS Chinese short text summarization dataset) and use predefined evaluation metrics to quantitatively analyze the model's output. For example, test samples are first input into the model, and then the lexical overlap between the model-generated text and the standard answer is calculated based on metrics such as ROUGE-1 and ROUGE-2, thereby measuring the model's ability to capture semantic information.

[0022] However, existing technologies still have shortcomings in practical implementation. First, standardized text datasets are constructed using manually annotated clean text, and their corpus distribution exhibits strong structured characteristics, failing to simulate common noise interference in real-world scenarios (such as colloquial expressions, mixed use of technical terms, and missing information). This idealized data environment makes it difficult for evaluation results to reflect the robustness of the model in practical applications, especially in scenarios with low tolerance for error, such as medical consultations and legal advice. Existing evaluation systems cannot effectively identify the model's stability under noise, leading to a one-sided assessment of model robustness. Second, evaluation mechanisms based on ROUGE-1 / ROUGE-2 have inherent limitations. These metrics perform superficial matching by statistically analyzing word frequency overlap, ignoring deep semantic connections within the text. For example, in legal reasoning scenarios, a model might obtain a high ROUGE score (Recall-Oriented Understudy for Gisting Evaluation) through keyword stuffing, but fail to construct the correct logical connections between legal clauses. This evaluation method cannot quantify the model's higher-order understanding capabilities, such as causal reasoning and implicit connections, and is also ill-suited to the new evaluation requirements of multimodal information fusion.

[0023] To address the problems of existing technologies, embodiments of this disclosure provide a model evaluation method, apparatus, device, storage medium, and program product. First, this disclosure evaluates the model based on the matching between the model's reasoning logic for the evaluation text and human cognition, utilizing the alignment between the model's reasoning chain and the labeled reasoning chain to enhance the interpretability, comprehensiveness, and accuracy of the model's evaluation capabilities. Second, to more comprehensively evaluate the model, this disclosure also dynamically generates test samples based on the model's reasoning feedback to test samples by simulating phenomena such as noise, ambiguity, or logical inversion in real-world scenarios, thereby quantifying the model's semantic understanding ability in complex scenarios.

[0024] This model evaluation method can be applied to scenarios where the text understanding or processing capabilities of a model are evaluated. The entity executing this evaluation method can be a model capable of analyzing input text. For example, it can be a large language model.

[0025] The model evaluation method provided in the embodiments of this disclosure will be introduced first.

[0026] Figure 1 A schematic flowchart of a model evaluation method provided in one embodiment of this disclosure is shown. Figure 1 As shown, the method may include the following steps: S110, Input the test sample into the model under test, and use the model under test to analyze the test sample according to the target task and obtain the execution result of the target task; S120, Based on the execution results of the target task, select multiple target lexical units from the lexical units of the test sample; S130, Based on multiple target words and the relationships between them, establish the model inference chain of the model to be tested for the test sample output; S140, based on the alignment between the labeled inference chain corresponding to the test sample and the model inference chain corresponding to the test sample, the evaluation result of the model to be tested is obtained.

[0027] Here, the test sample can be in the form of text.

[0028] It should be noted that the test samples can be used as analysis materials for the target task. The model under test analyzes the test samples according to the target task, and uses the analysis results to obtain the execution result of the target task. At this point, the target task has been completed.

[0029] It should be noted that the test samples include multiple tokens. A token can be the smallest basic unit of the test sample that the model under test processes and represents, and can be a word, a sub-word character, or a string segment cut according to other rules.

[0030] Thus, this disclosure filters the lexical units of the test samples based on the execution results of the target task to establish the model inference chain during the process of the model under test performing the target task using the test samples. Compared with the method of calculating evaluation indicators based on the model output results, this disclosure achieves a deep analysis of the inference logic of the model under test, transforming the internal decisions of the model under test into an interpretable model inference chain, thereby facilitating the accurate location of fault points. On this basis, this disclosure, by measuring the alignment between the model inference chain and the manually pre-annotated labeled inference chain, can better quantify the performance changes of the model under test after fine-tuning, preventing the phenomenon of correct results but incorrect inference processes, and enhancing the accuracy of evaluation.

[0031] The steps described above are explained in detail below.

[0032] Regarding S110, in this embodiment of the application, the test sample is dynamically generated and determined by the perturbation degree of each perturbation strategy in the perturbation strategy set and the inference performance of the model under test on the initial test sample.

[0033] In an optional implementation, before inputting the test sample into the model under test, analyzing the test sample according to the target task through the model under test, and obtaining the execution result of the target task, the method further includes: Select the target perturbation strategy corresponding to the target perturbation level from multiple perturbation strategies; Data augmentation is performed on the original samples based on the target perturbation strategy and the semantic features of the original samples to obtain the initial test samples; If the response matching degree of the model under test to the initial test sample is less than the preset response matching degree threshold, the initial test sample is used as the test sample; where the response matching degree is used to characterize the degree of adaptation between the processing logic of the model under test and the initial test sample. If the response matching degree of the model under test to the initial test sample is greater than or equal to the preset response matching degree threshold, the next perturbation degree adjacent to the target perturbation degree is taken as the new target perturbation degree; based on the new target perturbation degree, the step of selecting the target perturbation strategy corresponding to the target perturbation degree from multiple perturbation strategies is returned.

[0034] Here, the degree of perturbation is used to represent the semantic complexity that the perturbation strategy brings to the original sample.

[0035] For example, the semantic features of the original text can be entity distribution, syntactic structure, etc. The perturbation strategy can be the insertion of ambiguous words, syntactic recombination, logical reversal, etc., with changes in expression being the lowest level of perturbation, changes in structure being the medium level of perturbation, and changes in logical relationships being the highest level of perturbation.

[0036] In a specific example, the perturbation level can be set to three levels (K=3), with semantic complexity increasing progressively. The perturbation levels are arranged from smallest to largest as follows: Level 1: slight perturbation by inserting synonyms; Level 2: medium perturbation by syntactic recombination; and Level 3: strong perturbation by logical inversion.

[0037] When the original sample includes multiple original subsamples, the initial data augmentation of the first original subsample uses a perturbation level of Level 1 adversarial, applying only a slight perturbation. If the response matching degree of the first original subsample is greater than or equal to a preset response matching degree threshold... If the level of difficulty is high enough, the test sample is upgraded to the next level of adversarial sample, with the perturbation level targeting level two; otherwise, the current level is maintained. In this way, the adversarial difficulty of subsequent test samples is dynamically adjusted based on the response of the model under test to the test sample corresponding to the current level.

[0038] For example, the preset response matching threshold can be 0.7.

[0039] Furthermore, in a specific example, data augmentation is achieved through reinforcement learning (RL) to obtain test samples with semantic complexity and controllable adversarial intensity. The semantic features of the original samples are used as the state space, and the set of perturbation policies is used as the action space. In RL adversarial text generation, the attack success and semantic similarity are balanced, and the attack success reward, semantic similarity reward, etc., are combined into a total reward function. And a multi-index linearly weighted reward function is established through the following formula (1): (1) in, This is the original sample; For testing purposes; To reinforce the immediate reward of learning, a larger value indicates that the current learning reward is more substantial. The more it aligns with the goal; Score the semantic similarity. Assuming the attack was successful; , These are the weighting coefficients.

[0040] For example, when the original sample is "the user applied for a refund due to a system malfunction", and the perturbation strategy is to insert ambiguity and reverse logic, the ambiguity "claim" is inserted into the original sample, and the logic is reversed to "actually a mistake". The resulting initial test sample can be "the user claimed that 'system malfunction' caused the refund application, but it was actually a mistake", and the final test sample can be "the user claimed that 'system malfunction', but it was actually a mistake".

[0041] Thus, this disclosure sets difficulty levels for the perturbation strategy based on the degree of perturbation. A response matching degree higher than a preset threshold indicates that the model under test can easily cope with the perturbation at the current difficulty level. Therefore, the difficulty is increased, and stronger perturbations are applied, ensuring that each test sample falls precisely within the model's capability boundary. This improves the relevance and effectiveness of the test, avoiding wasting testing resources on patterns the model has already fully mastered. Furthermore, by progressively increasing the perturbation level, this disclosure only fully tests test samples that are suitable for the model, avoiding the inability to effectively utilize the effective information from high-difficulty test samples. It effectively distinguishes subtle differences between models, avoids useless computations caused by difficulty mismatches, and improves evaluation efficiency. Finally, dynamic perturbation is used to filter out test samples that match the processing capabilities of the model under test. Then, inference chain alignment evaluation is performed on the test samples, thereby accurately locating the breakpoints in the model's inference chain under adversarial pressure, forming a complete evaluation loop. This more realistically reflects the model's robust performance in actual deployment when facing distributed off-inputs, providing high-quality test material for subsequent inference chain analysis and error localization.

[0042] Furthermore, this disclosure determines the degree of matching of the model under test to the initial test sample based on the actual output, actual input, standard output, and standard input of the model under test.

[0043] In an optional implementation, before using the initial test sample as the test sample when the response matching degree of the model under test to the initial test sample is less than a preset response matching degree threshold, the method further includes: The initial test sample is input into the model under test. The model under test analyzes the initial test sample according to the first objective task and obtains the execution result of the first objective task. The execution result includes the reasoning answer output by the model under test for the first objective task. Based on the semantic similarity between the initial test sample and the original sample, and the semantic similarity between the inferred answer and the standard answer corresponding to the original sample, the response matching degree of the model under test to the initial test sample is determined.

[0044] Here, semantic similarity can be calculated using cosine similarity.

[0045] It's important to note that the primary objective task here can be the same as the target task, or it can be a different task. It can be a task specifically for testing, or it can be a task that the model under test can perform when applied.

[0046] In a specific example, the response matching index is calculated using the following formula (2): (2) The model output, or inference answer, is the actual output obtained by the model after the initial test samples are input into it and the model performs the first objective task based on the initial test samples. The standard answer can be the correct answer for the first objective task determined by a human based on the original samples. The initial test samples are obtained by data augmentation of the original samples, and therefore can also be called adversarial samples.

[0047] Thus, by simultaneously checking whether the semantics of the samples in the input model are reasonable and whether the model output is similar to the correct answer, this disclosure can effectively evaluate the initial test samples and thus effectively filter out valid samples that are semantically reasonable but difficult for the model to process.

[0048] Based on this, this disclosure further optimizes the evaluation results of the model under test based on the data augmentation process, and determines the robustness of the model under test by considering the degree of target perturbation that the model under test can adapt to and the condition of the initial sample.

[0049] In one optional implementation, the evaluation result of the model under test is obtained based on the alignment between the labeled inference chain corresponding to the test sample and the model inference chain corresponding to the test sample, including: Based on the number of historical initial test samples corresponding to the degree of disturbance of each historical target and the response matching degree of each historical initial test sample, the model robustness index of the model under test is determined. The evaluation results of the model under test are obtained based on the model robustness index of the model under test, the alignment degree between the labeled inference chain corresponding to the test sample and the model inference chain corresponding to the test sample.

[0050] Here, the historical target perturbation level refers to the degree of multiple target perturbations calculated during the process from receiving the first initial test sample to finally obtaining the test sample corresponding to the model under test. The historical initial test sample refers to the multiple initial test samples (including the final test sample) calculated during the process from receiving the first initial test sample to finally obtaining the test sample corresponding to the model under test.

[0051] In a specific example, to characterize the comprehensive robustness of the model under test on test samples of different intensity / difficulty levels, this disclosure proposes a hierarchical weighted robustness index calculation method. In this embodiment, each target perturbation level is assigned a corresponding level. The average matching degree of each level is weighted and summarized using formula (3): (3) in, Indicates the robustness index of the model; Indicates the model under test in the th... Cumulative value of response matching degree on the test samples; Indicates the first Total number of test samples at each level; Indicates the first Intensity weighting coefficient of level, ,in It is the difficulty penalty coefficient, which can be adjusted. Control the rate of weight growth; higher-level test samples (such as k=3) pose a greater challenge to the robustness of the model under test and need to be given higher weights, so they can account for a larger proportion in the final index.

[0052] For example, if γ = 0.5, then =1.0, =1.5, =2.0; if γ=1.0, then =1.0, =2.0, =3.0.

[0053] Thus, after dynamically generating adversarial examples by adjusting the test intensity step by step, this disclosure further quantifies the level of stable processing capability of the model under test when facing input perturbations of different intensities or types through a model robustness index, thereby deeply exploring the model's text processing capabilities. Then, the model robustness index is jointly considered with the alignment degree of the inference chain of the test samples to obtain a comprehensive evaluation result. This allows for simultaneous evaluation of the model's stable robustness and inference logic, resulting in a more comprehensive and refined assessment that enhances the accuracy of the evaluation.

[0054] Regarding S120, this disclosure uses attention mechanism analysis to extract key word elements involved in the text understanding process of the model under test in order to obtain the key decision path.

[0055] In one optional implementation, based on the execution result of the target task, multiple target lexical units are selected from the lexical units of the test sample, including: For each current word element in the test sample, determine the query vector based on the current word element, and determine the key vectors corresponding to the query vector based on all other words elements in the test sample except the current word element. For each query vector, calculate the degree of matching between the query vector and the corresponding key vectors to obtain the attention score; Based on the attention scores corresponding to each query vector, the tokens of the test samples are sorted to obtain the sorting results; Based on the ranking results and the execution results of the target task, multiple target lexical units are selected from the lexical units of the test sample.

[0056] Here, a lexical unit refers to the smallest basic unit that the model under test processes and represents text. For example, it can be a word, a sub-word character, or a string segment cut according to other rules.

[0057] In a specific example, the decision path parsing based on the attention mechanism is as follows. The attention matrix of the Transformer model is used to capture the key dependencies of the model under test when processing test samples. The attention weight matrix is ​​calculated using the following formula (4), where the attention weight matrix represents the attention score corresponding to each query vector: (4) in, This represents a query matrix, which includes multiple query vectors; it also represents a key matrix, which includes multiple key vectors. The feature dimension is [value], and the scaling factor is [value]. .

[0058] The attention weight matrix can identify query vectors with large attention weights, indicating a strong correlation between the query vector and its corresponding key vector. Thus, the words or phrases that the model under test focuses on during inference can be identified through the global ranking results.

[0059] For example, when the result of the target task is a refund refusal, the test sample identifies the term "refund application" as strongly related to refund refusal and "system failure" as strongly related to "refund application." This allows for the identification of the association between "system failure" and "refund application." The target terms involved can be system failure, misoperation, and refund application.

[0060] Therefore, this disclosure considers that the model under test will use an attention matrix when processing text internally. Thus, it also uses attention scores to simulate the focus of the model when processing test samples. This is combined with the execution results of the target task for selection, ensuring that the final selected target words not only have high attention weights but also have a causal relationship with the model output, naturally possessing interpretability and providing a possibility for establishing the model's inference chain. Furthermore, this disclosure ranks the words in the test samples based on the attention scores of each word, avoiding information blind spots caused by focusing only on a subset of words. The ranking results are also more representative, facilitating the capture of words with low frequency but playing a crucial bridging role in the model's decision-making.

[0061] Regarding S130, after obtaining the target lexical units through S120, it is necessary to reconstruct the model's inference chain by considering the relationships between the target lexical units or their positions in the test samples. That is, to reconstruct how the model under test obtains the execution result of the target task based on the relationships between the target lexical units.

[0062] In a specific example, key decision nodes are extracted and generated based on attention weights to form the model's inference chain. Here, key decision nodes can represent the role or stage of the target lexical in the entire inference chain.

[0063] Based on the target lexical units obtained above, the reasoning logic between the related target lexical units is inferred. Logical connectors are set for the target lexical units with the strongest relationships, forming a complete model reasoning chain. That is, starting from the system failure and sequentially going through each target lexical unit, the reasoning chain explains why a refund application can be obtained.

[0064] For example, when the test sample is "the user claims that a 'system malfunction' caused the refund request, but it was actually a mistake," the determined model inference chain is "identify 'system malfunction' → verify 'mistake' → infer the real reason is 'user mistake' → conclusion: refuse refund."

[0065] Regarding S140, to address the problem that traditional assessments are like black boxes, making it difficult to understand the internal reasoning process of the model, this disclosure analyzes the reasoning path of the model under test through step S130, compares the reasoning path with the standard reasoning chain labeled by human experts, to determine the deviation of the model under test in the reasoning process, and thus verify whether the model's logic conforms to human cognitive patterns. Taking a legal judgment task as an example, the model needs to generate a reasoning chain containing three stages: legal citation → fact matching → logical deduction.

[0066] In an optional implementation, before obtaining the evaluation result of the model under test based on the alignment between the labeled inference chain corresponding to the test sample and the model inference chain corresponding to the test sample, the method further includes: Calculate the edit distance similarity between the labeled inference chain corresponding to the test sample and the model inference chain corresponding to the test sample; Calculate the semantic similarity between the labeled inference chain corresponding to the test sample and the model inference chain corresponding to the test sample; We calculate the alignment between the edit distance similarity and semantic similarity by weighted summation.

[0067] It should be noted that the labeled inference chain refers to the inference chain labeled in advance by human experts, which is used as a benchmark for alignment evaluation.

[0068] In a specific example, a standard reasoning chain database implementation can be constructed. The text and questions are provided, requiring experts to generate step-by-step reasoning chains. For example, in a legal application scenario, the reasoning chain could be legal citation → fact matching → logical deduction. Correspondingly, each stage of the reasoning chain needs to include key decision nodes, such as entity identification → logic verification → conclusion generation. Finally, the reasoning chain is represented based on structured storage, and the expert reasoning chain is represented as a sequence or graph structure as shown in equation (5): (5) in, Let i represent the i-th decision node. The decision node can be "verify whether 'misoperation' exists".

[0069] Here, alignment is used to characterize the similarity or matching between the labeled inference chain and the model inference chain corresponding to the same test sample.

[0070] In a specific example, the alignment degree of the inference chain is quantified through the following process: That is, the alignment degree is quantified based on edit distance similarity and semantic similarity. The smaller the edit distance, the higher the similarity. For example, when the model inference chain is completely consistent with the labeled inference chain, the edit distance similarity is 1. The edit distance similarity characterizes the structural similarity and is calculated using formula (6): (6) in, Represents the model inference chain; This represents the labeled reasoning chain established by experts.

[0071] Meanwhile, to measure the semantic consistency between the model inference chain and the labeled inference chain, BERTScore's token-level greedy matching similarity is used to calculate semantic similarity. Its basic form is shown in formula (7): (7) in, This represents the number of nodes in the model's inference chain. The number of nodes in the inference chain is used to indicate the number of nodes. It is the first inference chain of the model The context vector representation of the text of each decision node from the bidirectional encoder representations from the transformers (BERT); It is the first inference chain labeled The BERT context vector representation of the text of each node. It is the vector cosine similarity; Indicates semantic similarity.

[0072] Based on this, in order to simultaneously consider the structural similarity and semantic similarity of the reasoning chain, a linearly weighted fusion comprehensive alignment index is constructed, as shown in Equation (8): (8) in, The weighting coefficient (e.g., λ=0.6) balances structural and semantic similarity; To edit distance similarity, normalized Levenshtein similarity can be used; Indicates the degree of alignment.

[0073] Thus, by extracting the model's inference chain and comparing it with inference chains annotated by human experts, the alignment between the model's inference logic and human cognition can be quantified. This approach balances textual structure preservation and semantic fidelity, making model evaluation interpretable and facilitating the identification of the causes of inference errors in the model under test. Furthermore, this disclosure is applicable to situations where detection results are consistent but the inference process is inconsistent; even under this premise, it can still identify phenomena of illogical model logic. In addition, this disclosure evaluates the model under test based on the completeness and rationality of the inference process, resulting in more diversified evaluation dimensions.

[0074] Furthermore, this disclosure proposes an error type localization and optimization strategy to locate specific defects in the model's inference chain based on the analysis results of alignment. For example, defects may include missing key nodes, logical order errors, or semantic shifts.

[0075] Among them, missing key nodes refer to the failure to identify core decision points (such as ignoring the verification of "misoperation"); incorrect logical order refers to the unreasonable order of reasoning steps (such as drawing conclusions before verification); semantic deviation refers to the excessive deviation between the semantics of nodes and expert annotations (such as misinterpreting "misoperation" as "system vulnerability").

[0076] Accordingly, to address the deficiency of missing key nodes, the training of the model under test on specific domain knowledge (such as legal clauses or medical terminology) can be enhanced; to address logical sequence errors, logical constraint modules (such as causal reasoning networks) can be introduced.

[0077] Furthermore, the original text can exist in a multimodal form, simultaneously containing text, layers, audio, and other multimodal data. In this case, cross-modal consistency evaluation metrics can be designed to quantify the model's semantic understanding and reasoning capabilities under different modal inputs, such as logical derivation in text-image matching tasks. Moreover, by combining multimodal adversarial example generation techniques, complex real-world scenarios can be simulated, such as noisy voice commands or blurred image descriptions, thereby improving the model's robustness in multimodal interactive applications. This extension can be widely applied in fields such as intelligent customer service, medical image analysis, and content moderation, significantly enhancing the system's comprehensive understanding capabilities.

[0078] Furthermore, while current evaluation schemes primarily focus on general text understanding capabilities, a dynamic weight adaptation mechanism based on domain knowledge can be introduced to optimize model performance in specific application scenarios. For example, in the field of medical diagnosis, the weight of the logical reasoning layer can be strengthened to highlight causal deduction capabilities; in the field of legal reasoning, the weight of the contextual association layer can be increased to capture complex clause citations and factual relationships. By combining domain knowledge graphs and task requirements, and dynamically adjusting the weight allocation strategy at each level, the evaluation system can flexibly adapt to the key capability requirements of different domains, thereby providing developers with more precise optimization guidance. This avoids the problem of insufficient evaluation resolution caused by fixed samples in traditional methods, significantly reducing the time and resource investment required for model optimization.

[0079] Furthermore, a real-time feedback-driven adaptive evaluation mechanism can be established. By adding a real-time feedback-driven module to the existing evaluation framework, performance data of the model in real-world scenarios can be continuously collected, dynamically adjusting evaluation criteria and adversarial example generation strategies. For example, user interaction logs or error case analysis can be used to identify weaknesses in the model for specific tasks, and targeted adversarial examples can be generated for re-evaluation. This self-evolutionary mechanism not only helps the model quickly adapt to dynamically changing task requirements but also continuously improves the scientific rigor and practicality of the evaluation system over long-term operation, making it particularly suitable for industrial applications requiring high-frequency iterative optimization.

[0080] In summary, this disclosure first proposes a method for dynamic test sample generation and progressively increasing difficulty testing. Test samples are dynamically generated using reinforcement learning, and a perturbation-based adjustment mechanism is combined to quantify the model's robustness in complex semantic scenarios. Specifically, reinforcement learning-based data augmentation ensures the semantic similarity and adversarial nature of the generated samples; the closed-loop logic for dynamically adjusting the test sample intensity allows for real-time upgrading or maintenance of sample difficulty based on the model's response; and the graded weighted adversarial robustness index calculation method quantifies the model's overall performance on test samples of different intensities through weighted accuracy. Thus, by using reinforcement learning to generate semantically complex and controllable difficulty test samples, problems such as noise, ambiguity, and logical inversion in real-world scenarios can be simulated, comprehensively exposing the model's weaknesses in complex semantic scenarios. This significantly improves the accuracy of model robustness evaluation and scenario adaptability, avoids the need for manually annotating large-scale static datasets, and reduces the cost of repeated testing.

[0081] Secondly, this disclosure also establishes a method for analyzing cognitive reasoning chains and evaluating their alignment with human cognition. By extracting the model's reasoning chain and comparing it with reasoning chains annotated by human experts, the degree of alignment between the model's reasoning logic and human cognition is quantified. Specifically, based on the attention mechanism, key decision paths of the model in text comprehension tasks are extracted to generate reasoning chains; a comprehensive alignment index combining edit distance similarity and semantic similarity is designed to comprehensively evaluate the rationality of the model's reasoning logic. This is the first time that a deep-level analysis and precise diagnosis of the model's reasoning logic has been achieved, which is conducive to clearly pointing out specific defects in the model's reasoning chain and enhancing the interpretability of the evaluation results.

[0082] Figure 2 This is a schematic diagram of the structure of a model evaluation device provided in an embodiment of this disclosure. Figure 2 As shown, the model evaluation device 200 may include an execution module 210, a screening module 220, a creation module 230, and an evaluation module 240.

[0083] The execution module 210 is used to input test samples into the model under test, analyze the test samples according to the target task through the model under test, and obtain the execution result of the target task. The filtering module 220 is used to filter out multiple target lexical units from the lexical units of the test sample based on the execution results of the target task. Module 230 is established to build a model inference chain for the test model outputting test samples based on multiple target words and the relationships between them. The evaluation module 240 is used to obtain the evaluation results of the model under test based on the alignment degree between the labeled inference chain corresponding to the test sample and the model inference chain corresponding to the test sample.

[0084] Optionally, before inputting the test sample into the model under test, analyzing the test sample according to the target task through the model under test, and obtaining the execution result of the target task, the execution module 210 is further configured to: Select the target perturbation strategy corresponding to the target perturbation level from multiple perturbation strategies; Data augmentation is performed on the original samples based on the target perturbation strategy and the semantic features of the original samples to obtain the initial test samples; If the response matching degree of the model under test to the initial test sample is less than the preset response matching degree threshold, the initial test sample is used as the test sample; where the response matching degree is used to characterize the degree of adaptation between the processing logic of the model under test and the initial test sample. If the response matching degree of the model under test to the initial test sample is greater than or equal to the preset response matching degree threshold, the next perturbation degree adjacent to the target perturbation degree is taken as the new target perturbation degree; based on the new target perturbation degree, the step of selecting the target perturbation strategy corresponding to the target perturbation degree from multiple perturbation strategies is returned.

[0085] Optionally, before using the initial test sample as the test sample, if the response matching degree of the model under test to the initial test sample is less than a preset response matching degree threshold, the execution module 210 is further configured to: The initial test sample is input into the model under test. The model under test analyzes the initial test sample according to the first objective task and obtains the execution result of the first objective task. The execution result includes the reasoning answer output by the model under test for the first objective task. Based on the semantic similarity between the initial test sample and the original sample, and the semantic similarity between the inferred answer and the standard answer corresponding to the original sample, the response matching degree of the model under test to the initial test sample is determined.

[0086] Optionally, the evaluation module 240 is also used for: Based on the number of historical initial test samples corresponding to the degree of disturbance of each historical target and the response matching degree of each historical initial test sample, the model robustness index of the model under test is determined. The evaluation results of the model under test are obtained based on the model robustness index of the model under test, the alignment degree between the labeled inference chain corresponding to the test sample and the model inference chain corresponding to the test sample.

[0087] Optionally, module 230 is also used for: For each current word element in the test sample, determine the query vector based on the current word element, and determine the key vectors corresponding to the query vector based on all other words elements in the test sample except the current word element. For each query vector, calculate the degree of matching between the query vector and the corresponding key vectors to obtain the attention score; Based on the attention scores corresponding to each query vector, the tokens of the test samples are sorted to obtain the sorting results; Based on the ranking results and the execution results of the target task, multiple target lexical units are selected from the lexical units of the test sample.

[0088] Optionally, before obtaining the evaluation result of the model under test based on the alignment between the labeled inference chain corresponding to the test sample and the model inference chain corresponding to the test sample, the evaluation module 240 is also used for: Calculate the edit distance similarity between the labeled inference chain corresponding to the test sample and the model inference chain corresponding to the test sample; Calculate the semantic similarity between the labeled inference chain corresponding to the test sample and the model inference chain corresponding to the test sample; We calculate the alignment between the edit distance similarity and semantic similarity by weighted summation.

[0089] Figure 3 A schematic diagram of the hardware structure of the model evaluation device provided in this embodiment is shown.

[0090] The model evaluation device may include a processor 301 and a memory 302 storing computer program instructions.

[0091] Specifically, the processor 301 may include a central processing unit (CPU), an application specific integrated circuit (ASIC), or one or more integrated circuits that can be configured to implement the embodiments of this disclosure.

[0092] Memory 302 may include mass storage for data or instructions. For example, and not limitingly, memory 302 may include a hard disk drive (HDD), floppy disk drive, flash memory, optical disk, magneto-optical disk, magnetic tape, or Universal Serial Bus (USB) drive, or a combination of two or more of these. In one instance, memory 302 may include removable or non-removable (or fixed) media, or memory 302 may be non-volatile solid-state memory. Memory 302 may be internal or external to the integrated gateway disaster recovery device.

[0093] In one instance, memory 302 may be read-only memory (ROM). In one instance, the ROM may be a mask-programmed ROM, a programmable ROM (PROM), an erasable PROM (EPROM), an electrically erasable PROM (EEPROM), an electrically rewritable ROM (EAROM), or flash memory, or a combination of two or more of these.

[0094] Memory 302 may include read-only memory (ROM), random access memory (RAM), disk storage media device, optical storage media device, flash memory device, electrical, optical, or other physical / tangible memory storage device. Therefore, generally, memory includes one or more tangible (non-transitory) computer-readable storage media (e.g., memory devices) encoded with software including computer-executable instructions, and when the software is executed (e.g., by one or more processors), it is operable to perform the operations described with reference to the method according to one aspect of this disclosure.

[0095] The processor 301 reads and executes computer program instructions stored in the memory 302 to achieve... Figure 1 The model evaluation method in the illustrated embodiment.

[0096] In one example, the model evaluation device may also include a communication interface 303 and a bus 304. For example, Figure 3 As shown, the processor 301, memory 302, and communication interface 303 are connected through bus 304 and complete communication with each other.

[0097] The communication interface 303 is mainly used to realize communication between various modules, devices, units and / or equipment in the embodiments of this disclosure.

[0098] Bus 304 includes hardware, software, or both, that couples components of a model evaluation device together. For example, and not as a limitation, the bus may include an Accelerated Graphics Port (AGP) or other graphics bus, an Extended Industry Standard Architecture (EISA) bus, a Front Side Bus (FSB), a Hyper Transport (HT) interconnect, an Industry Standard Architecture (ISA) bus, an Infinite Bandwidth Interconnect, a Low Pin Count (LPC) bus, a memory bus, a Microchannel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCI-X) bus, a Serial Advanced Technology Attachment (SATA) bus, a Video Electronics Standards Association Local (VLB) bus, or other suitable buses, or combinations of two or more of these. Where appropriate, bus 304 may include one or more buses. Although specific buses are described and illustrated in embodiments of this disclosure, this disclosure contemplates any suitable bus or interconnect.

[0099] Furthermore, in conjunction with the model evaluation methods in the above embodiments, this disclosure can provide a computer storage medium for implementation. The computer storage medium stores computer program instructions; when these computer program instructions are executed by a processor, they implement any of the model evaluation methods in the above embodiments.

[0100] This application also provides a computer program product, including a computer program, which, when executed by a processor, implements any of the model evaluation methods described in the above embodiments.

[0101] It should be clarified that this disclosure is not limited to the specific configurations and processes described above and shown in the figures. For the sake of brevity, detailed descriptions of known methods are omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of this disclosure is not limited to the specific steps described and shown, and those skilled in the art can make various changes, modifications, and additions, or change the order of steps, after understanding the spirit of this disclosure.

[0102] The functional blocks shown in the above-described block diagram can be implemented as hardware, software, firmware, or a combination thereof. When implemented in hardware, they can be, for example, electronic circuits, application-specific integrated circuits (ASICs), appropriate firmware, plug-ins, function cards, etc. When implemented in software, the elements of this disclosure are programs or code segments used to perform the required tasks. Programs or code segments can be stored on a machine-readable medium or transmitted over a transmission medium or communication link via data signals carried on a carrier wave. "Machine-readable medium" can include any medium capable of storing or transmitting information. Examples of machine-readable media include electronic circuits, semiconductor memory devices, read-only memory (ROM), flash memory, erasable read-only memory (EROM), floppy disks, compact disc read-only memory (CD-ROM), optical disks, hard disks, fiber optic media, radio frequency (RF) links, etc. Code segments can be downloaded via computer networks such as the Internet, intranets, etc.

[0103] It should also be noted that the exemplary embodiments mentioned in this disclosure describe methods or systems based on a series of steps or apparatus. However, this disclosure is not limited to the order of the above steps; that is, the steps can be performed in the order mentioned in the embodiments, or in a different order, or several steps can be performed simultaneously.

[0104] The aspects of this disclosure have been described above with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this disclosure. It should be understood that each block in the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that these instructions, executable via the processor of the computer or other programmable data processing apparatus, enable the implementation of the functions / actions specified in one or more blocks of the flowchart illustrations and / or block diagrams. Such a processor can be, but is not limited to, a general-purpose processor, a special-purpose processor, a special application processor, or a field-programmable logic circuit. It is also understood that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can also be implemented by special-purpose hardware performing the specified functions or actions, or can be implemented by a combination of special-purpose hardware and computer instructions.

[0105] The above description is merely a specific embodiment of this disclosure. Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, modules, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here. It should be understood that the protection scope of this disclosure is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in this disclosure, and these modifications or substitutions should all be covered within the protection scope of this disclosure.

Claims

1. A model evaluation method, characterized in that, The method includes: The test sample is input into the model under test, and the model under test analyzes the test sample according to the target task to obtain the execution result of the target task; Based on the execution results of the target task, multiple target lexical units are selected from the lexical units of the test sample; Based on the multiple target words and the relationships between them, a model inference chain is established for the test model outputting the test sample. The evaluation result of the model under test is obtained based on the alignment between the labeled inference chain corresponding to the test sample and the model inference chain corresponding to the test sample.

2. The model evaluation method according to claim 1, characterized in that, Before inputting the test sample into the model under test, analyzing the test sample according to the target task through the model under test, and obtaining the execution result of the target task, the method further includes: Select the target perturbation strategy corresponding to the target perturbation level from multiple perturbation strategies; Based on the target perturbation strategy and the semantic features of the original samples, the original samples are augmented to obtain the initial test samples; If the response matching degree of the model under test to the initial test sample is less than a preset response matching degree threshold, the initial test sample is used as the test sample; wherein, the response matching degree is used to characterize the degree of adaptation between the processing logic of the model under test and the initial test sample; If the response matching degree of the model under test to the initial test sample is greater than or equal to the preset response matching degree threshold, the next perturbation degree adjacent to the target perturbation degree is taken as the new target perturbation degree; based on the new target perturbation degree, the step of selecting the target perturbation strategy corresponding to the target perturbation degree from multiple perturbation strategies is returned to execute.

3. The model evaluation method according to claim 2, characterized in that, Before using the initial test sample as the test sample when the response matching degree of the model under test to the initial test sample is less than a preset response matching degree threshold, the method further includes: The initial test sample is input into the model to be tested, and the model to be tested analyzes the initial test sample according to the first target task to obtain the execution result of the first target task; the execution result includes the reasoning answer output by the model to be tested for the first target task; Based on the semantic similarity between the initial test sample and the original sample, and the semantic similarity between the inferred answer and the standard answer corresponding to the original sample, the response matching degree of the model to be tested to the initial test sample is determined.

4. The model evaluation method according to claim 2, characterized in that, The evaluation result of the model under test is obtained based on the alignment degree between the labeled inference chain corresponding to the test sample and the model inference chain corresponding to the test sample, including: Based on the number of historical initial test samples corresponding to the degree of disturbance of each historical target and the response matching degree of each historical initial test sample, the model robustness index of the model under test is determined. The evaluation result of the model under test is obtained based on the model robustness index of the model under test, the alignment degree between the labeled inference chain corresponding to the test sample and the model inference chain corresponding to the test sample.

5. The model evaluation method according to claim 1, characterized in that, Based on the execution result of the target task, multiple target lexical units are selected from the lexical units of the test sample, including: For each current word element in the test sample, a query vector is determined based on the current word element, and key vectors corresponding to the query vector are determined based on all other word elements in the test sample except for the current word element. For each query vector, the degree of matching between the query vector and each key vector corresponding to the query vector is calculated to obtain the attention score; Based on the attention scores corresponding to each query vector, the tokens of the test sample are sorted to obtain the sorting results; Based on the sorting results and the execution results of the target task, multiple target lexical units are selected from the lexical units of the test sample.

6. The model evaluation method according to claim 1, characterized in that, Before obtaining the evaluation result of the model under test based on the alignment between the labeled inference chain corresponding to the test sample and the model inference chain corresponding to the test sample, the method further includes: Calculate the edit distance similarity between the labeled inference chain corresponding to the test sample and the model inference chain corresponding to the test sample; Calculate the semantic similarity between the labeled inference chain corresponding to the test sample and the model inference chain corresponding to the test sample; The alignment degree between the edit distance similarity and the semantic similarity is obtained by weighted summation of the labeled inference chain corresponding to the test sample and the model inference chain corresponding to the test sample.

7. A model evaluation device, characterized in that, The device includes: The execution module is used to input test samples into the model under test, analyze the test samples according to the target task through the model under test, and obtain the execution result of the target task. The filtering module is used to filter out multiple target words from the words in the test sample based on the execution result of the target task; A module is established to build a model inference chain for the test model output by the model under test for the test sample, based on the multiple target words and the association between the multiple target words. The evaluation module is used to obtain the evaluation result of the model under test based on the alignment degree between the labeled inference chain corresponding to the test sample and the model inference chain corresponding to the test sample.

8. A model evaluation device, characterized in that, The device includes: a processor and a memory storing computer program instructions; the processor reads and executes the computer program instructions to implement the model evaluation method as described in any one of claims 1-6.

9. A computer storage medium, characterized in that, The computer storage medium stores computer program instructions, which, when executed by a processor, implement the model evaluation method as described in any one of claims 1-6.

10. A computer program product, characterized in that, It includes a computer program that, when executed by a processor, implements the model evaluation method as described in any one of claims 1-6.

Images