Risk Testing Method and Apparatus for Deep Learning Models Based on Metamorphosis Testing

Abstract

This invention provides a method and apparatus for risk testing of deep learning models based on metamorphic testing, belonging to the field of artificial intelligence technology. The method includes: acquiring source test cases; selecting a target-specific metamorphic relationship pattern from a dedicated metamorphic relationship pattern library, which includes various dedicated metamorphic relationship patterns for multi-dimensional risk testing; transforming the source test cases based on the target-specific metamorphic relationship pattern to generate derived test cases; inputting the source test cases into a deep learning model to obtain source output, and inputting the derived test cases into the deep learning model to obtain derived output; verifying whether the source output and derived output satisfy the expected relationship to determine the risk status of the deep learning model; updating the risk feature library and optimizing the dedicated metamorphic relationship pattern library based on the risk status, and dynamically adjusting the testing strategy of the deep learning model. This invention employs multiple testing modes, improving the accuracy and comprehensiveness of the testing.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and in particular to a method and apparatus for risk testing of deep learning models based on metamorphosis testing. Background Technology

[0002] Deep learning models, as a core technology in the field of artificial intelligence, have been widely applied in many key areas. Deep learning models exhibit high sensitivity to input data, their decision-making processes are difficult to interpret and verify, and their performance is highly dependent on the quality, quantity, and distribution of the training data. The inherent complexity and data-driven nature of deep learning models make it difficult to directly and effectively apply traditional software testing methods to their risk assessment.

[0003] Currently, existing methods for risk testing of deep learning models often employ general and fragmented metamorphic relationships, lacking specificity and affecting the accuracy and comprehensiveness of the tests. Summary of the Invention

[0004] This invention provides a method and apparatus for risk testing of deep learning models based on metamorphic testing, which addresses the shortcomings of existing deep learning model risk testing methods in terms of accuracy and comprehensiveness.

[0005] This invention provides a risk testing method for deep learning models based on metamorphosis testing, comprising: Obtain the source test cases for the deep learning model to be tested; Select at least one target exclusive transformation relationship pattern from a preset exclusive transformation relationship pattern library, wherein the exclusive transformation relationship pattern library includes a variety of exclusive transformation relationship patterns for multi-dimensional risk testing; Based on the target-specific metamorphic relationship pattern, the source test cases are transformed to generate derivative test cases; The source test cases are input into the deep learning model to obtain the source output of the deep learning model, and the derived test cases are input into the deep learning model to obtain the derived output of the deep learning model. Verify whether the source output and the derived output satisfy the expected relationship, obtain the verification result, and determine the risk status of the deep learning model based on the verification result; Based on the risk profile of the deep learning model, the pre-built risk feature library is updated, the exclusive metamorphosis relationship pattern library is optimized, and the testing strategy of the deep learning model is dynamically adjusted.

[0006] In some embodiments, the dedicated metamorphic relationship pattern library includes: a disturbance-resistant robust consistency pattern, an environmental adaptation reliability pattern, a semantic structure consistency pattern, and a feature contribution stability pattern; the disturbance-resistant robust consistency pattern is used to evaluate the stability of the deep learning model when facing adversarial disturbances or backdoor triggering; the environmental adaptation reliability pattern is used to evaluate the reliability of the deep learning model when facing environmental degradation; the semantic structure consistency pattern is used to evaluate the accuracy of the deep learning model in semantic understanding and logical reasoning of input information; and the feature contribution stability pattern is used to evaluate the rationality of the deep learning model's reliance on key features of input information during the decision-making process.

[0007] In some embodiments, the target-specific metamorphic relationship pattern is defined by a transformation function, a relationship verification function, and constraints; the transformation function is used to define the specific operations for generating the derived test cases based on the source test cases; the relationship verification function is used to define the expected relationship that should be satisfied between the source output and the derived output; and the constraints are used to define the scope of application of the transformation function.

[0008] In some embodiments, when the target-specific metamorphic relationship mode is a disturbance-resistant robust consistency mode, the transformation of the source test cases to generate derived test cases includes: Under the constraints, based on the transformation function, a perturbation is applied to the source test case, or a backdoor trigger is implanted at a preset position in the source test case to generate a derivative test case; The constraints include at least one of the following: The amplitude of the disturbance is within a preset amplitude threshold range; The triggering mode of the backdoor trigger conforms to the preset backdoor triggering mode.

[0009] In some embodiments, when the target-specific metamorphic relationship pattern is an environmental adaptation reliability pattern, the transformation of the source test cases to generate derived test cases includes: Under the constraints, based on the transformation function, environmental degradation simulation operations are performed on the source test cases to generate derived test cases. The environmental degradation simulation operations include at least one of adding noise, blurring, changing brightness, and simulating weather effects. The constraints include: the parameters used to simulate environmental degradation are within a preset threshold range.

[0010] In some embodiments, when the target-specific metamorphic relationship pattern is a semantically consistent pattern, the transformation of the source test case to generate derived test cases includes: Under the constraints, based on the transformation function, the source test cases are semantically rewritten, logical words are modified, or structures are reorganized to generate derived test cases; The constraints include at least one of the following: The core semantics of the source test cases and the derived test cases are consistent. The derived test cases are logically clear and unambiguous; When the core task objective of the source test case does not depend on the specific order of the elements to be reorganized, the elements to be reorganized are structurally reorganized, and the elements to be reorganized are the elements in the source test case.

[0011] In some embodiments, when the target-specific metamorphic relationship pattern is a feature contribution stability pattern, transforming the source test cases to generate derived test cases includes: Under the constraints, based on the transformation function, key features in the source test cases are removed, or key features are introduced into the source test cases, or irrelevant features in the source test cases are modified to generate derivative test cases. The constraints include at least one of the following: The operation on the key features and / or irrelevant features is clear, complete, and controlled; Based on the domain knowledge corresponding to the source test cases, the key features and irrelevant features are determined.

[0012] In some embodiments, after generating the derived test cases, the method further includes: Verify whether the derived test cases satisfy the constraints. If the derived test case does not meet the constraints, the derived test case is regenerated.

[0013] In some embodiments, determining the risk status of the deep learning model based on the verification results includes: Based on the verification results corresponding to the specific transformation relationship patterns of each target, the risk index of the deep learning model under each specific transformation relationship pattern is calculated. Based on the risk indicators of the deep learning model under the specific transformation relationship mode of each target, the risk status of the deep learning model is determined. In some embodiments, after determining the risk status of the deep learning model, the method further includes: Based on the risk profile of the deep learning model, the pre-built risk feature library is updated, the exclusive metamorphosis relationship pattern library is optimized, and the testing strategy of the deep learning model is dynamically adjusted.

[0014] This invention also provides a risk testing device for deep learning models based on metamorphosis testing, comprising: The acquisition unit is used to acquire the source test cases of the deep learning model to be tested. The selection unit is used to select at least one target exclusive transformation relationship pattern from a preset exclusive transformation relationship pattern library, wherein the exclusive transformation relationship pattern library includes a variety of exclusive transformation relationship patterns for multi-dimensional risk testing. The first generation unit is used to transform the source test cases based on the target-specific metamorphic relationship pattern to generate derived test cases; The output unit is used to input the source test cases into the deep learning model to obtain the source output of the deep learning model, and to input the derived test cases into the deep learning model to obtain the derived output of the deep learning model. A verification unit is used to verify whether the source output and the derived output satisfy the expected relationship, obtain the verification result, and determine the risk status of the deep learning model based on the verification result. The adjustment unit is used to update the pre-built risk feature library, optimize the exclusive metamorphosis relationship pattern library, and dynamically adjust the testing strategy of the deep learning model based on the risk status of the deep learning model.

[0015] The present invention provides a method and apparatus for risk testing of deep learning models based on metamorphic testing. This method involves: acquiring source test cases for the deep learning model to be tested; selecting at least one target-specific metamorphic relationship pattern from a pre-defined library of specific metamorphic relationship patterns; transforming the source test cases in a targeted manner based on the target-specific metamorphic relationship pattern to generate derived test cases; inputting the source test cases into the deep learning model to obtain source output; inputting the derived test cases into the deep learning model to obtain derived output; verifying whether the expected relationship is satisfied between the source output and the derived output; determining the risk status of the deep learning model based on the verification results; updating the risk feature library and optimizing the library of specific metamorphic relationship patterns based on the risk status; and dynamically adjusting the testing strategy of the deep learning model, thereby improving the accuracy and comprehensiveness of risk testing for deep learning models. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0017] Figure 1This is a flowchart illustrating the risk testing method for deep learning models based on metamorphosis testing provided in this embodiment of the invention.

[0018] Figure 2 This is a flowchart illustrating the process of determining the risk status of a deep learning model provided in an embodiment of the present invention.

[0019] Figure 3 This is a schematic diagram of the structure of the deep learning model risk testing device based on metamorphosis testing provided in an embodiment of the present invention.

[0020] Figure 4 This is a schematic diagram of the structure of the electronic device provided in an embodiment of the present invention. Detailed Implementation

[0021] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0022] The terms "first," "second," etc., used in this invention are used to distinguish similar objects, not to describe a specific order or sequence. It should be understood that such terms can be used interchangeably where appropriate so that embodiments of the invention can be implemented in orders other than those illustrated or described herein, and the objects distinguished by "first" and "second" are generally of the same class, not limited in number; for example, the first object can be one or more. Furthermore, in this invention, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship.

[0023] Currently, existing methods for risk assessment of deep learning models have the following shortcomings: First, existing risk assessment methods suffer from the lack of test oracles. In complex scenarios, providing accurate and predictable outputs for massive amounts of deep learning model inputs is extremely costly and impractical. Traditional testing methods that rely on such oracles struggle to handle large-scale data, resulting in insufficient test coverage.

[0024] Secondly, the inherent black-box nature of deep learning models makes it difficult to analyze the root causes when the model exhibits abnormal or erroneous behavior, and it is also difficult to proactively discover the hidden structural defects or potential biases within it, thus limiting the accuracy of risk identification.

[0025] Furthermore, current risk assessment methods are significantly insufficient in terms of the comprehensiveness and specificity of risk coverage. Many methods may only focus on the general performance indicators of the model or only target a single type of risk, failing to systematically cover the series of core risk dimensions unique to deep learning models, leading to misjudgments of the overall risk status of the model.

[0026] Furthermore, existing testing strategies often lack sufficient dynamism and adaptability, and lack closed-loop feedback mechanisms. Faced with the rapid iteration of deep learning models and the constant emergence of new attack methods and unknown risk scenarios, pre-set, static test case sets and testing methods are insufficient to meet the challenges. At the same time, existing technologies lack unified and effective standards and tools for objectively and comparablely quantifying the discovered risks, making it difficult to accurately measure the severity and priority of different risks.

[0027] To address this, embodiments of the present invention provide a method and apparatus for risk testing of deep learning models based on metamorphic testing. The method involves: acquiring source test cases for the deep learning model to be tested; selecting at least one target-specific metamorphic relationship pattern from a pre-defined library of specific metamorphic relationship patterns; transforming the source test cases in a targeted manner based on the target-specific metamorphic relationship pattern to generate derived test cases; inputting the source test cases into the deep learning model to obtain source output; inputting the derived test cases into the deep learning model to obtain derived output; verifying whether the source output and derived output satisfy the expected relationship; determining the risk status of the deep learning model based on the verification results; updating the risk feature library and optimizing the library of specific metamorphic relationship patterns based on the risk status; and dynamically adjusting the testing strategy of the deep learning model, thereby improving the accuracy and comprehensiveness of risk testing for deep learning models.

[0028] Figure 1 This is a flowchart illustrating the risk testing method for deep learning models based on metamorphosis testing provided in an embodiment of the present invention. Figure 1 As shown, a risk testing method for deep learning models based on metamorphosis testing is provided, including the following steps: steps 110 to 160. This method's steps are merely one possible implementation of the present invention.

[0029] Step 110: Obtain the source test cases for the deep learning model to be tested; Deep learning models are complex artificial intelligence models that learn patterns and rules from large amounts of data by simulating the structure of the human brain's neural networks. Common examples include models for image recognition, natural language processing, or speech recognition.

[0030] The source test cases include test inputs, such as an image, a piece of text, or a voice command.

[0031] Step 120: Select at least one target-specific transformation relationship pattern from the preset exclusive transformation relationship pattern library. The exclusive transformation relationship pattern library includes a variety of exclusive transformation relationship patterns for multi-dimensional risk testing. The core theoretical basis of this invention is metamorphic testing. Metamorphic testing does not rely on test oracles; for each test case, it is not necessary to predetermine its precise and unique correct output. When the expected output of a certain input is difficult to determine, metamorphic testing can be performed using certain inherent relationships that should be followed between the input and the output.

[0032] In some embodiments, the dedicated metamorphic relation pattern library includes: a disturbance-resistant robust consistency pattern, an environmental adaptation reliability pattern, a semantic structure consistency pattern, and a feature contribution stability pattern; the disturbance-resistant robust consistency pattern is used to evaluate the stability of a deep learning model when facing adversarial perturbations or backdoor triggering; the environmental adaptation reliability pattern is used to evaluate the reliability of a deep learning model when facing environmental degradation; the semantic structure consistency pattern is used to evaluate the accuracy of a deep learning model in semantic understanding and logical reasoning of input information; and the feature contribution stability pattern is used to evaluate the rationality of a deep learning model's reliance on key features of input information during the decision-making process.

[0033] It should be noted that each of the above models focuses on one or a class of core risks of deep learning models, providing a standardized and operational set of testing logic and judgment criteria. By applying multiple specialized models and comprehensively considering multiple risk dimensions, the risks of deep learning models can be assessed more accurately and in greater depth, achieving a comprehensive review of model risks.

[0034] Optionally, one or more of the following modes can be selected from the dedicated metamorphic relation mode library: disturbance-resistant robust consistency mode, environmental adaptation reliability mode, semantic structure consistency mode, and feature contribution stability mode.

[0035] In some embodiments, the target-specific metamorphic relationship pattern is defined by a transformation function, a relationship verification function, and constraints; the transformation function is used to define the specific operations for generating derived test cases based on source test cases; the relationship verification function is used to define the expected relationship that should be satisfied between the source output and the derived output; and the constraints are used to define the scope of application of the transformation function.

[0036] It should be noted that the exclusive metamorphosis relationship pattern library uses standardized...<T,R,C> The triplet format provides a reusable test logic template with high flexibility and scalability. T represents the transformation function, R represents the relation verification function, and C represents the constraint condition. The testing process involves instantiating these triples, performing the transformation, obtaining the output, and verifying the expected relationship under the constraint condition. Test cases for specific risks can be quickly configured and generated based on these templates, improving testing efficiency.

[0037] A transformation function T defines how to generate a new, logically related derivative test case from a known source test case through a series of systematic, controlled modifications closely related to a specific risk to be assessed. The transformation function T determines what types of model behavior and potential risks can be detected.

[0038] The relation validation function R defines the expected relation that should be satisfied between the source output and the derived output. For example, the relation validation function R requires that the classification result of the deep learning model remains unchanged under a certain transformation, such as classification after image rotation; or that the output of the deep learning model should change according to a certain expected pattern, such as the sentiment polarity reversing after adding a negative word in text sentiment analysis; or that certain key performance indicators of the deep learning model should remain within an acceptable range, such as the recognition accuracy under environmental perturbations. The validation result of the relation validation function R is usually either satisfied or not satisfied.

[0039] Constraint C specifies in detail the applicable scope, parameter boundaries, and characteristics that derived test cases should satisfy for the transformation function T. It also defines the preconditions for the validity of the relation verification function R. Subsequent judgments based on the relation verification function R are only meaningful when constraint C is satisfied. By setting constraint C, meaningless or misleading tests can be avoided, ensuring the effectiveness, rationality, and reliability of metamorphic testing. For example, when applying adversarial perturbations, constraint C limits the magnitude of the perturbation to ensure that the input after perturbation still belongs to the same category or retains its original semantics for a human observer; when simulating environmental noise, constraint C ensures that the type and intensity of the noise are realistically possible.

[0040] Step 130: Based on the target-specific metamorphic relationship pattern, transform the source test cases to generate derived test cases; Optionally, based on the target-specific metamorphic relationship pattern, a transformation function is determined, and based on the transformation function, each source test case is transformed once or multiple times to generate one or more derived test cases.

[0041] In some embodiments, after generating derived test cases, the method further includes: Verify whether the derived test cases meet the constraints; If it is determined that the derived test cases do not meet the constraints, the derived test cases are regenerated.

[0042] Optionally, if it is determined that the derived test cases meet the constraints, verify whether the derived test cases are high-quality test cases; if it is determined that the derived test cases are not high-quality test cases, regenerate the derived test cases.

[0043] Step 140: Input the source test cases into the deep learning model to obtain the source output of the deep learning model; input the derived test cases into the deep learning model to obtain the derived output of the deep learning model. Optionally, the association between source test cases and derived test cases is determined, and the logical relationship between source output and derived output is determined.

[0044] Step 150: Verify whether the source output and the derived output meet the expected relationship, obtain the verification results, and determine the risk status of the deep learning model based on the verification results.

[0045] Optionally, a relationship verification function is determined based on the target-specific metamorphic relationship pattern, and the expected relationship is determined based on the relationship verification function.

[0046] Understandably, evaluating the behavior of deep learning models by verifying the logical relationship between source outputs and derived outputs effectively eliminates the reliance on precise test oracles, allowing test results to focus more on the consistency and rationality of the model's intrinsic behavior, thereby improving the reliability of risk assessment and the credibility of test results.

[0047] Figure 2 This is a flowchart illustrating the process of determining the risk status of a deep learning model according to an embodiment of the present invention. In some embodiments, step 150, based on the verification results, determines the risk status of the deep learning model, including: Step 151: Based on the verification results corresponding to the transformation relationship patterns of each target, calculate the risk index of the deep learning model under each target-specific transformation relationship pattern. Step 152: Determine the risk status of the deep learning model based on the risk indicators of the deep learning model under the specific transformation relationship pattern of each target.

[0048] Optionally, the proportion of each type of metamorphic relation that is satisfied is calculated, i.e., the metamorphic relation (MR) satisfaction rate. Based on the MR satisfaction rate of each type of metamorphic relation and the weight of each type of metamorphic relation, a risk score for the deep learning model is calculated to determine the risk level of the deep learning model.

[0049] Step 160: Based on the risk status of the deep learning model, update the pre-built risk feature library, optimize the exclusive metamorphosis relationship pattern library, and dynamically adjust the testing strategy of the deep learning model.

[0050] Understandably, by updating the pre-built risk feature library and continuously improving the risk profile, the transformation test mode can be automatically or semi-automatically adjusted to better adapt to new risks and threats. By optimizing the exclusive transformation relationship pattern library, the testing strategy of the deep learning model can be dynamically adjusted, thus achieving continuous optimization of the testing strategy and the exclusive transformation relationship pattern library.

[0051] In this embodiment of the invention, source test cases of the deep learning model to be tested are obtained; at least one target-specific metamorphic relationship pattern is selected from a preset library of specific metamorphic relationship patterns; based on the target-specific metamorphic relationship pattern, the source test cases are transformed in a targeted manner to generate derived test cases; the source test cases are input into the deep learning model to obtain the source output, and the derived test cases are input into the deep learning model to obtain the derived output; it is verified whether the source output and the derived output meet the expected relationship, and based on the verification results, the risk status of the deep learning model is determined; based on the risk status, the risk feature library is updated, the library of specific metamorphic relationship patterns is optimized, and the testing strategy of the deep learning model is dynamically adjusted, thereby improving the accuracy and comprehensiveness of the risk testing of the deep learning model.

[0052] In some embodiments, when the target-specific metamorphic relationship pattern is a disturbance-resistant robust consistency pattern, the source test cases are transformed to generate derived test cases, including: Under constraints, based on the transformation function, a perturbation is applied to the source test case, or a backdoor trigger is implanted at a preset position in the source test case to generate a derivative test case; The constraints include at least one of the following: The amplitude of the disturbance is within the preset amplitude threshold range; The backdoor trigger's triggering mode conforms to the preset backdoor triggering mode.

[0053] The core idea of ​​the robust consistency model lies in simulating various potential malicious attack scenarios. This involves designing a specific transformation function T to transform the source test case X, generating derived test cases X' containing malicious perturbations or triggering conditions. The model then verifies whether the derived output P(X') of the deep learning model, when processing X', maintains the expected consistency with the source output P(X), or whether it can successfully resist pre-set malicious behavior. If, under these malicious transformations, the key output behavior of the deep learning model changes unexpectedly, it indicates insufficient robustness and a risk of being attacked or manipulated.

[0054] Design of the transformation function T for the disturbance-resistant robust consistency mode: For adversarial perturbations: T can be instantiated for various known or novel adversarial example generation algorithms. For example, for image classification models, small perturbations can be added to the original image based on the transformation function T; for text data, synonym replacements or character-level perturbations can be added to the original text based on the transformation function T.

[0055] For backdoor attack triggers: T can be a predefined trigger superimposed at a specific location of the source test case X, such as adding a small pixel block of a specific color in a corner of an image, or inserting a specific phrase or special symbol into a text sequence, thereby generating a derivative test case X' containing a backdoor trigger.

[0056] Definition of the relation verification function R for robust consistency patterns: For classification tasks in adversarial perturbation scenarios: R requires that the predicted class labels of deep learning models remain unchanged. This means that even if the input is slightly perturbed, the correct classification result of the model should not change. For regression tasks in adversarial perturbation scenarios: R requires that the difference between the source output P(X) and the derived output P(X') be within a very small acceptable range. For testing the resistance of deep learning models to backdoor triggering: R requires that when the input contains triggers, the output of the deep learning model should not be the malicious output expected by the attacker.

[0057] Setting the constraint C for the disturbance-resistant robust consistency mode: Adversarial perturbation magnitude limitation: C strictly limits the minimization of adversarial perturbations. For example, for images, a certain norm distance between the original image and the adversarial sample can be required to be less than a preset distance threshold. For text, metrics such as edit distance can be used to limit the degree of modification, ensuring that it does not change the core semantics.

[0058] Explicit backdoor triggering mode: C explicitly defines the specific mode of the trigger used, such as the shape, size, color, content, position in the input, etc. of the trigger, as well as the specific conditions for backdoor activation.

[0059] In this embodiment of the invention, by applying the anti-disturbance robust consistency mode, the stability and security of deep learning models in the face of various simulated malicious attacks can be systematically and quantitatively evaluated, revealing their vulnerability in resisting disturbances and potential backdoors.

[0060] In some embodiments, when the target-specific metamorphic relationship pattern is an environment-adaptive reliability pattern, the source test cases are transformed to generate derived test cases, including: Under constraints, based on the transformation function, environmental degradation simulation operations are performed on the source test cases to generate derived test cases. The environmental degradation simulation operations include at least one of adding noise, blurring, changing brightness, and simulating weather effects. The constraints include: the parameters used to simulate environmental degradation are within a preset threshold range.

[0061] The core idea of ​​the environmental adaptability reliability model is to simulate a series of common environmental factors that may lead to a deterioration in the quality or alteration of the input data. Specifically, a specific transformation function T is designed to generate derived test cases X' that simulate these environmental effects. Then, the key outputs or core performance metrics of the deep learning model when processing X' are verified to ensure they remain at an acceptable level of stability. If the deep learning model experiences a sharp decline in performance or significant deviations in key outputs under these common, non-malicious environmental changes, it indicates insufficient environmental adaptability and a risk to its reliability.

[0062] Design of the transformation function T for environmental adaptability reliability mode: For image data: T can include various operations that simulate image quality degradation. For example: Image quality degradation simulation: Applying Gaussian blur algorithm to simulate image defocusing or motion blur effects; adding salt and pepper noise or Gaussian white noise to simulate sensor noise; changing the brightness, contrast, and saturation of the image to simulate different lighting conditions; simulating lossy transmission; environmental noise superposition: directly superimposing background noise on the original clear image; weather condition simulation: using image synthesis technology or specific algorithms to simulate the impact of different weather conditions such as rain, snow, fog, and haze on the image.

[0063] For text data, T can be transformed as follows: Noise addition to speech signals: For speech input, various background noises can be mixed into the original clear speech signal, and the signal-to-noise ratio can be controlled; this can simulate recognition errors that may occur during optical character recognition, or simulate random keystroke errors that may occur when users use keyboard input. For example, randomly replacing, inserting, or deleting certain characters in text with a low probability; similarly, corresponding environmental degradation simulation methods can be designed according to specific data types and application scenarios, such as adding drift or noise to sensor data, or simulating partial data loss or distortion in time-series data.

[0064] Definition of the relationship verification function R for environmental adaptation reliability patterns: Classification / recognition results remain unchanged: For scenarios with mild environmental degradation, it is expected that the predicted class labels of the deep learning model should not change; Key performance indicators above the threshold: When environmental degradation is significant, it is expected that the key performance indicators of the deep learning model can be maintained above a preset acceptable minimum level. Output values ​​fluctuate within an acceptable range: For regression tasks, it is expected that the difference between the source output P(X) and the derived output P(X') will be within a preset threshold range under environmental disturbances.

[0065] Setting the constraint C for the environmental adaptability reliability mode: Reasonable environmental simulation parameters: The parameters used to simulate environmental degradation should be set within the range of parameters that may occur in real life and should be representative. Avoid setting overly extreme or unrealistic degradation parameters.

[0066] The applied environmental degradation transformation should not be so severe that key information is completely lost, rendering it incomprehensible even to human experts: the applied transformation should not cause the complete loss of all identifiable information in the input, to the point that even domain-savvy human experts cannot make a valid judgment. For example, in simulating dense fog, the outlines of objects in the image should at least be partially visible.

[0067] In this embodiment of the invention, by applying an environment-adaptive reliability model, the performance stability and reliability of deep learning models under various simulated, real-world application-like environmental conditions can be effectively evaluated, thereby revealing their potential risks in terms of generalization ability and coping with the complexities of the real world.

[0068] In some embodiments, when the target-specific metamorphic relation pattern is a semantically consistent pattern, the source test cases are transformed to generate derived test cases, including: Under constraints, based on transformation functions, source test cases are semantically rewritten, logical words are modified, or structures are reorganized to generate derived test cases. The constraints include at least one of the following: The core semantics of the source test cases and the derived test cases should remain consistent. The logic of the derived test cases is clear and unambiguous; When the core task objective of the source test case does not depend on the specific order of the elements to be reorganized, the elements to be reorganized are restructured, and the elements to be reorganized are the elements in the source test case.

[0069] The core idea of ​​the semantic structure consistency pattern is to generate derived test cases X' by applying a series of transformations to the source test cases, aiming to change their expression or local structure while preserving their core semantics. Then, it verifies whether the derived output P(X') obtained by the deep learning model after processing X' is semantically consistent with the source output P(X), or whether it correctly reflects the logical changes introduced by the transformations. If the deep learning model exhibits a bias in understanding equivalent semantics or logical processing errors under these transformations, it indicates a risk in its semantic understanding and logical reasoning.

[0070] Design of the transformation function T for semantically consistent patterns: For text data, the following transformations can be performed: Semantic equivalence rewriting involves replacing words with synonyms or near-synonyms, transforming sentence structures, adjusting word order, and providing explanatory restatements to the original text. The goal is to change the surface form of the text while striving to preserve its core semantic information.

[0071] Logical word modification: This involves adding, deleting, or modifying key logical connectors, negations, or modal words in the original text. This operation directly alters the logical relationships or emotional tone of the sentence, and it is expected that the output of the deep learning model will reflect these logical changes accordingly.

[0072] Reordering: For some tasks, such as document topic classification or keyword extraction, the order of sentences or paragraphs in the input text has little impact on the final result. In this case, the order of sentences or paragraphs can be shuffled.

[0073] For image data, the following transformations can be performed: Object list rearrangement: Primarily used for tasks such as scene description generation and multi-object counting. If the input to a deep learning model is a list of objects detected in an image, such as a list containing "cat, dog, and table," the order of this list can be shuffled, such as becoming "dog, table, and cat."

[0074] Image set order rearrangement: Suitable for tasks that process image sequences or sets of images, such as video summarization or image set classification. If the overall result of the task does not depend on a specific arrangement of the input images, then the order of these input images is shuffled.

[0075] Definition of the relation verification function R for semantic structural consistency patterns: Output invariance under semantic equivalence: When the transformation belongs to the semantic equivalence rewriting type, it is expected that the output of the deep learning model remains unchanged. For example, the core intent or event identified by the model should be the same for an active sentence and its corresponding passive sentence.

[0076] Expected output changes under logical operations: When a transformation introduces explicit logical word modifications, the output of the expected model should reflect this logical change accordingly. For example, in sentiment analysis, if a sentence that was originally a positive evaluation is transformed into a negative one after applying a negation transformation, the expected model's sentiment judgment should change accordingly.

[0077] Output invariance under structural reorganization: When the transformation involves reordering the input elements and the task itself is insensitive to this order, the output of the expected model remains unchanged.

[0078] The constraint C for the semantic structure consistency pattern is set as follows: Semantic Preservation / Logical Clarity: For semantically equivalent rewriting operations, it must be ensured that the core semantics before and after the transformation are highly consistent, and ambiguity or alteration of the original meaning should not be introduced due to the rewriting. For logical operations, the introduced logical meaning must be clear and unambiguous; Applicable Scenarios: Especially for structural reorganization operations, it is only applicable to scenarios where the core task objective does not depend on the specific order of the reorganized elements; Avoid Introducing Additional Confusion: Transformation should not introduce additional irrelevant semantic information or ambiguity that may confuse the model's judgment.

[0079] In this embodiment of the invention, by applying the semantic structure consistency pattern, it is possible to conduct an in-depth evaluation of the deep learning model and reveal the potential risks to its semantic understanding.

[0080] In some embodiments, when the target-specific metamorphic relationship pattern is a feature-contributing stability pattern, the source test cases are transformed to generate derived test cases, including: Under constraints, based on the transformation function, key features in the source test cases are removed, key features are introduced into the source test cases, or irrelevant features in the source test cases are modified to generate derivative test cases. The constraints include at least one of the following: Operations on key features and / or irrelevant features are clear, complete, and controlled; Based on the domain knowledge corresponding to the source test cases, key features and irrelevant features are identified.

[0081] The core idea of ​​the feature contribution stability model is to determine whether the model's decisions reasonably depend on specific local features by systematically modifying them in the input data and observing the corresponding changes in the model's output. These features can be key or irrelevant features known from prior knowledge, or features that have a significant or minor impact on the model's decisions, as initially identified through interpretability analysis tools. If the model shows a logically significant response to changes in key features but is largely insensitive to changes in irrelevant features, its feature contribution stability is considered good; otherwise, there may be a risk of feature dependency bias.

[0082] Design of the transformation function T for the characteristic contribution stability mode: For image data, the following transformations can be performed: Key object / feature removal: Removing a key small target or local feature region from an image. For example, in face recognition tasks, key areas of the face, such as the eyes and nose, can be occluded. Key object / feature introduction: Clearly adding a key object to a scene image that originally did not contain it, through image compositing or pasting. For example, adding a red light to an image that originally did not contain traffic lights in an autonomous driving scenario. Irrelevant object / feature modification: Modifying local features in an image that are considered irrelevant. For example, changing the color of a small object in the background, which usually has little impact on the judgment of the main task.

[0083] For text data, the following transformations can be performed: Keyword removal: Delete keywords from text containing specific strong indicative words; Keyword introduction: Add the keyword or phrase to text that originally did not contain the specific strong indicative word; Irrelevant word / sentence modification: Make minor modifications to words or sentences in the text that are considered irrelevant and do not affect the core semantics. For example, replacing an insignificant adjective with its synonym should, in theory, not affect the model's judgment of core information.

[0084] Definition of the relationship verification function R for the characteristic contribution stability mode: Expected response after introducing a key positive feature: When a key positive feature that can guide the model to make a specific correct judgment is introduced into the input, it is expected that the output category of the model can accurately become the category indicated by this new feature, and the model's confidence in this judgment should be higher than a preset threshold.

[0085] Expected response after removing a key positive feature: When a key positive feature is removed from the input, it is expected that the output category of the model will no longer be the original category, or the confidence of the model in judging the original category will drop significantly below a certain low threshold.

[0086] Output stability after modifying irrelevant features: When irrelevant features in the input are modified, it is expected that the derived output of the model will remain essentially consistent with the source output. Specifically, for classification tasks, it is expected that the derived output category after modifying irrelevant features will be the same as the source output category; for regression tasks, it is expected that the difference between the derived output after modifying irrelevant features and the source output will be less than a very small threshold.

[0087] Setting the constraint C for the characteristic contribution stability mode: Clarity and controllability of feature modification: Operations on target features must be clear, complete, and controlled; Reasonable assumptions about feature criticality: When conducting feature contribution stability tests, key features and irrelevant features should be reasonably set based on domain knowledge, common sense judgment, or preliminary interpretability analysis results; Minimize side effects: When modifying target features, unexpected significant impacts on other parts of the input should be avoided as much as possible.

[0088] In this embodiment of the invention, by systematically applying the feature contribution stability model, it is possible to delve into the internal logic of deep learning model decision-making, assess whether it depends on truly meaningful input features, and whether its response to changes in key information is stable and in line with expectations, thereby effectively identifying the risks caused by feature dependency bias.

[0089] The following describes the deep learning model risk testing device based on metamorphosis testing provided in the embodiments of the present invention. The deep learning model risk testing device based on metamorphosis testing described below and the deep learning model risk testing method based on metamorphosis testing described above can be referred to and correspond to each other.

[0090] Figure 3 This is a schematic diagram of the structure of the deep learning model risk testing device based on metamorphosis testing provided in an embodiment of the present invention, as shown below. Figure 3 As shown, the deep learning model risk testing device 300 based on metamorphosis testing includes: Acquisition unit 310 is used to acquire source test cases for the deep learning model to be tested; Selection unit 320 is used to select at least one target exclusive transformation relationship pattern from a preset exclusive transformation relationship pattern library. The exclusive transformation relationship pattern library includes a variety of exclusive transformation relationship patterns for multi-dimensional risk testing. The first generation unit 330 is used to transform the source test cases based on the target-specific metamorphic relationship pattern to generate derived test cases; Output unit 340 is used to input source test cases into the deep learning model to obtain the source output of the deep learning model, and to input derived test cases into the deep learning model to obtain the derived output of the deep learning model. The verification unit 350 is used to verify whether the expected relationship is met between the source output and the derived output, obtain the verification result, and determine the risk status of the deep learning model based on the verification result.

[0091] The adjustment unit 360 is used to update the pre-built risk feature library, optimize the exclusive metamorphosis relationship pattern library, and dynamically adjust the testing strategy of the deep learning model based on the risk status of the deep learning model.

[0092] Optionally, the target-specific metamorphic relationship pattern is defined by a transformation function, a relationship verification function, and constraints; the transformation function is used to define the specific operations for generating derived test cases based on source test cases; the relationship verification function is used to define the expected relationship that should be satisfied between the source output and the derived output; and the constraints are used to define the scope of application of the transformation function.

[0093] Optionally, the dedicated metamorphic relationship pattern library includes: a disturbance-resistant robust consistency pattern, an environmental adaptation reliability pattern, a semantic structure consistency pattern, and a feature contribution stability pattern; the disturbance-resistant robust consistency pattern is used to evaluate the stability of deep learning models when facing adversarial disturbances or backdoor triggering; the environmental adaptation reliability pattern is used to evaluate the reliability of deep learning models when facing environmental degradation; the semantic structure consistency pattern is used to evaluate the accuracy of deep learning models in semantic understanding and logical reasoning of input information; and the feature contribution stability pattern is used to evaluate the rationality of deep learning models' reliance on key features of input information during the decision-making process.

[0094] Optionally, the target-specific metamorphic relationship pattern is defined by a transformation function, a relationship verification function, and constraints; the transformation function is used to define the specific operations for generating derived test cases based on source test cases; the relationship verification function is used to define the expected relationship that should be satisfied between the source output and the derived output; and the constraints are used to define the scope of application of the transformation function.

[0095] Optionally, when the target-specific metamorphic relationship pattern is a disturbance-resistant robust consistency pattern, the source test cases are transformed to generate derived test cases, including: Under constraints, based on the transformation function, a perturbation is applied to the source test case, or a backdoor trigger is implanted at a preset position in the source test case to generate a derivative test case; The constraints include at least one of the following: The amplitude of the disturbance is within the preset amplitude threshold range; The backdoor trigger's triggering mode conforms to the preset backdoor triggering mode.

[0096] Optionally, when the target-specific metamorphic relationship pattern is an environment-adaptive reliability pattern, the source test cases are transformed to generate derived test cases, including: Under constraints, based on the transformation function, environmental degradation simulation operations are performed on the source test cases to generate derived test cases. The environmental degradation simulation operations include at least one of adding noise, blurring, changing brightness, and simulating weather effects. The constraints include: the parameters used to simulate environmental degradation are within a preset threshold range.

[0097] Optionally, if the target-specific metamorphic relation pattern is a semantically consistent pattern, the source test cases are transformed to generate derived test cases, including: Under constraints, based on transformation functions, source test cases are semantically rewritten, logical words are modified, or structures are reorganized to generate derived test cases. The constraints include at least one of the following: The core semantics of the source test cases and the derived test cases should remain consistent. The logic of the derived test cases is clear and unambiguous; When the core task objective of the source test case does not depend on the specific order of the elements to be reorganized, the elements to be reorganized are restructured, and the elements to be reorganized are the elements in the source test case.

[0098] Optionally, when the target-specific metamorphic relationship pattern is a feature contribution stability pattern, the source test cases are transformed to generate derived test cases, including: Under constraints, based on the transformation function, key features in the source test cases are removed, key features are introduced into the source test cases, or irrelevant features in the source test cases are modified to generate derivative test cases. The constraints include at least one of the following: Operations on key features and / or irrelevant features are clear, complete, and controlled; Based on the domain knowledge corresponding to the source test cases, key features and irrelevant features are identified.

[0099] Optionally, the deep learning model risk testing device based on metamorphosis testing also includes: The verification unit is used to verify whether the derived test cases meet the constraints. The second generation unit is used to regenerate derivative test cases when the derivative test cases do not meet the constraints.

[0100] Optionally, based on the validation results, the risk profile of the deep learning model is determined, including: Based on the verification results corresponding to the specific transformation relationship patterns of each target, the risk indicators of the deep learning model under the specific transformation relationship patterns of each target are calculated. Based on the risk indicators of deep learning models under the specific transformation relationship patterns of each objective, the risk status of deep learning models is determined. It should be noted that the deep learning model risk testing device based on metamorphosis testing provided in this embodiment of the invention can implement all the method steps implemented in the above-mentioned deep learning model risk testing method embodiment based on metamorphosis testing, and can achieve the same technical effect. Here, the parts that are the same as those in the method embodiment and the beneficial effects will not be described in detail.

[0101] Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention, such as... Figure 4 As shown, the electronic device may include: a processor 410, a communications interface 420, a memory 430, and a communications bus 440, wherein the processor 410, the communications interface 420, and the memory 430 communicate with each other through the communications bus 440. The processor 410 can call logical instructions in the memory 430 to execute a deep learning model risk testing method based on metamorphic testing. This method includes: acquiring source test cases for the deep learning model to be tested; selecting at least one target-specific metamorphic relationship pattern from a pre-defined library of specific metamorphic relationship patterns, the library including multiple specific metamorphic relationship patterns for multi-dimensional risk testing; transforming the source test cases based on the target-specific metamorphic relationship pattern to generate derived test cases; inputting the source test cases into the deep learning model to obtain the source output of the deep learning model, and inputting the derived test cases into the deep learning model to obtain the derived output of the deep learning model; verifying whether the source output and the derived output satisfy the expected relationship, obtaining the verification result; determining the risk status of the deep learning model based on the verification result; and updating the pre-built risk feature library and optimizing the specific metamorphic relationship pattern library based on the risk status of the deep learning model, dynamically adjusting the testing strategy of the deep learning model.

[0102] Furthermore, the logical instructions in the aforementioned memory 430 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0103] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0104] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0105] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A risk testing method for deep learning models based on metamorphosis testing, characterized in that, include: Obtain the source test cases for the deep learning model to be tested; Select at least one target exclusive transformation relationship pattern from a preset exclusive transformation relationship pattern library, wherein the exclusive transformation relationship pattern library includes a variety of exclusive transformation relationship patterns for multi-dimensional risk testing; Based on the target-specific metamorphic relationship pattern, the source test cases are transformed to generate derivative test cases; The source test cases are input into the deep learning model to obtain the source output of the deep learning model, and the derived test cases are input into the deep learning model to obtain the derived output of the deep learning model. Verify whether the source output and the derived output satisfy the expected relationship, obtain the verification result, and determine the risk status of the deep learning model based on the verification result; Based on the risk profile of the deep learning model, the pre-built risk feature library is updated, the exclusive metamorphosis relationship pattern library is optimized, and the testing strategy of the deep learning model is dynamically adjusted.

2. The risk testing method for deep learning models based on metamorphosis testing according to claim 1, characterized in that, The dedicated metamorphic relationship pattern library includes: a disturbance-resistant robust consistency pattern, an environmental adaptation reliability pattern, a semantic structure consistency pattern, and a feature contribution stability pattern. The disturbance-resistant robust consistency pattern is used to evaluate the stability of the deep learning model when facing adversarial disturbances or backdoor triggering. The environmental adaptation reliability pattern is used to evaluate the reliability of the deep learning model when facing environmental degradation. The semantic structure consistency pattern is used to evaluate the accuracy of the deep learning model's semantic understanding and logical reasoning of input information. The feature contribution stability pattern is used to evaluate the rationality of the deep learning model's reliance on key features of input information during the decision-making process.

3. The risk testing method for deep learning models based on metamorphosis testing according to claim 2, characterized in that, The target-specific metamorphic relationship pattern is defined by a transformation function, a relationship verification function, and constraints; the transformation function is used to define the specific operations for generating the derived test cases based on the source test cases; the relationship verification function is used to define the expected relationship that should be satisfied between the source output and the derived output. The constraints are used to define the applicable range of the transformation function.

4. The risk testing method for deep learning models based on metamorphosis testing according to claim 3, characterized in that, When the target-specific metamorphic relationship pattern is a disturbance-resistant robust consistency pattern, the transformation of the source test cases to generate derived test cases includes: Under the constraints, based on the transformation function, a perturbation is applied to the source test case, or a backdoor trigger is implanted at a preset position in the source test case to generate a derivative test case; The constraints include at least one of the following: The amplitude of the disturbance is within a preset amplitude threshold range; The triggering mode of the backdoor trigger conforms to the preset backdoor triggering mode.

5. The risk testing method for deep learning models based on metamorphosis testing according to claim 3, characterized in that, When the target-specific metamorphic relationship pattern is an environmental adaptation reliability pattern, the transformation of the source test cases to generate derived test cases includes: Under the constraints, based on the transformation function, environmental degradation simulation operations are performed on the source test cases to generate derived test cases. The environmental degradation simulation operations include at least one of adding noise, blurring, changing brightness, and simulating weather effects. The constraints include: the parameters used to simulate environmental degradation are within a preset threshold range.

6. The risk testing method for deep learning models based on metamorphosis testing according to claim 3, characterized in that, When the target-specific metamorphic relationship pattern is a semantically consistent pattern, the transformation of the source test case to generate derived test cases includes: Under the constraints, based on the transformation function, the source test cases are semantically rewritten, logical words are modified, or structures are reorganized to generate derived test cases; The constraints include at least one of the following: The core semantics of the source test cases and the derived test cases are consistent. The derived test cases are logically clear and unambiguous; When the core task objective of the source test case does not depend on the specific order of the elements to be reorganized, the elements to be reorganized are structurally reorganized, and the elements to be reorganized are the elements in the source test case.

7. The risk testing method for deep learning models based on metamorphosis testing according to claim 3, characterized in that, When the target-specific metamorphic relationship pattern is a feature contribution stability pattern, the transformation of the source test cases to generate derived test cases includes: Under the constraints, based on the transformation function, key features in the source test cases are removed, or key features are introduced into the source test cases, or irrelevant features in the source test cases are modified to generate derivative test cases. The constraints include at least one of the following: The operation on the key features and / or irrelevant features is clear, complete, and controlled; Based on the domain knowledge corresponding to the source test cases, the key features and irrelevant features are determined.

8. The risk testing method for deep learning models based on metamorphosis testing according to claim 3, characterized in that, After generating the derived test cases, the process also includes: Verify whether the derived test cases satisfy the constraints. If the derived test case does not meet the constraints, the derived test case is regenerated.

9. The risk testing method for deep learning models based on metamorphosis testing according to claim 1, characterized in that, Determining the risk status of the deep learning model based on the verification results includes: Based on the verification results corresponding to the specific transformation relationship patterns of each target, the risk index of the deep learning model under each specific transformation relationship pattern is calculated. Based on the risk indicators of the deep learning model under the specific transformation relationship patterns of each target, the risk status of the deep learning model is determined.

10. A risk testing device for deep learning models based on metamorphosis testing, characterized in that, include: The acquisition unit is used to acquire the source test cases of the deep learning model to be tested. The selection unit is used to select at least one target exclusive transformation relationship pattern from a preset exclusive transformation relationship pattern library, wherein the exclusive transformation relationship pattern library includes a variety of exclusive transformation relationship patterns for multi-dimensional risk testing. The first generation unit is used to transform the source test cases based on the target-specific metamorphic relationship pattern to generate derived test cases; The output unit is used to input the source test cases into the deep learning model to obtain the source output of the deep learning model, and to input the derived test cases into the deep learning model to obtain the derived output of the deep learning model. A verification unit is used to verify whether the source output and the derived output satisfy the expected relationship, obtain the verification result, and determine the risk status of the deep learning model based on the verification result. The adjustment unit is used to update the pre-built risk feature library, optimize the exclusive metamorphosis relationship pattern library, and dynamically adjust the testing strategy of the deep learning model based on the risk status of the deep learning model.

Images