Image classification credibility evaluation method and system based on activation mode

Abstract

The invention discloses an image classification credibility evaluation method and system based on activation modes, and the method comprises the steps: carrying out the forward propagation of training samples of a training data set, recording the activation mode of each training sample in an internal activation layer of a model, and storing the activation modes as the standard activation modes of the training samples; when the model receives a new input sample in actual operation, performing path similarity comparison on a real-time activation mode of the input sample and a standard activation mode to evaluate credibility; executing one of the following operations according to the credibility obtained by evaluation: if the credibility of the input sample exceeds the distribution boundary of the standard activation mode in the training data set, marking as needing manual confirmation, giving an alarm and triggering a rejection response mechanism; otherwise, outputting a result and attaching a credible label. According to the invention, the interpretability of the model decision process can be improved, and abnormal behaviors can be accurately identified.

Description

Technical Field

[0001] This invention relates to the fields of artificial intelligence and image recognition technology, specifically to an image classification credibility evaluation method and system based on activation patterns. Background Technology

[0002] In recent years, with the rapid development of deep learning technology, neural networks have become a core driving force for the development of artificial intelligence, achieving widespread application and significant results in fields such as computer vision, speech recognition, and natural language processing. Especially in tasks such as image recognition, object detection, and image segmentation, deep models represented by convolutional neural networks (CNNs) have demonstrated outstanding performance and have been deployed in practical scenarios such as autonomous driving, industrial inspection, medical image diagnosis, and security monitoring. Representative models such as Faster R-CNN, the YOLO series, and ResNet have achieved recognition accuracy approaching or even surpassing human levels on public datasets such as COCO and ImageNet.

[0003] While deep neural networks possess powerful representational and decision-making capabilities, their highly complex nonlinear structure makes their internal reasoning mechanisms difficult for humans to directly understand, creating the so-called "black box problem." The model's decision-making path, key activation units, and class determination criteria are often opaque and difficult to trace. For applications with extremely high safety and reliability requirements, such as autonomous driving systems or medical diagnostic systems, relying solely on the model's output class labels or confidence values ​​cannot adequately support high-confidence decision-making. Therefore, in recent years, "Explainable AI (XAI)" and "Model Confidence Evaluation" have gradually become research focuses in academia and industry.

[0004] In the field of interpretability research, scholars have proposed various methods to improve the understandability of models, such as Grad-CAM, LIME, SHAP, and Layer-wise Relevance Propagation (LRP). These methods often reveal the model's attention areas or discrimination criteria for input samples through visualization or surrogate modeling. Although the above methods enhance the visual interpretation of model outputs to some extent, they rely on the model's posterior response and are usually focused on the period after the decision is made, using saliency icons to indicate "where the model paid attention." They lack systematic modeling of the internal neuronal activation paths and representation space evolution processes, and cannot reveal the differences in the model's behavioral structure under different samples or categories.

[0005] Meanwhile, credibility assessment, as a crucial technical approach to ensuring model security, has also been extensively studied. Common methods include threshold determination based on output probability distribution, Bayesian uncertainty modeling implemented through Monte Carlo Dropout, robustness assessment under adversarial perturbations, and distance metrics in the input space. These methods attempt to determine whether the model is "confident" in its output and are used to reject uncertain samples or detect anomalous behavior. However, most of these methods rely on external modifications or proxy models, lacking observation and constraints on the model's internal behavior. Furthermore, these methods often struggle to accurately distinguish between two types of high-risk samples: boundary samples with low confidence but semantically correct; and pseudo-correct samples with high confidence but identifying abnormal paths. This results in a significant risk of misjudgment in the practical deployment of existing credibility assessment methods.

[0006] In summary, existing interpretability and credibility assessment methods still have significant shortcomings in the following aspects: 1. Insufficient ability to systematically model the activation structure of neurons within the model. Current mainstream interpretation methods (CN119418132A) focus on the mapping relationship between input and output, emphasizing the visual basis for inferring the output category from the input image. However, there is insufficient research on behavioral characteristics such as activation paths and substructure collaboration patterns among multiple neurons within deep neural networks. This makes it difficult to identify whether the model uses a consistent discrimination mechanism under different inputs, and thus it is difficult to determine whether the model is "thinking normally".

[0007] 2. The credibility assessment lacks behavioral structure support, making it prone to misjudging complex or anomalous samples. Existing confidence methods typically start from output layer features, ignoring the different internal paths that the model may activate during processing. When the model faces semantic drift, out-of-distribution samples, or adversarial samples, its output may appear reasonable but the activation paths are abnormal, which is a phenomenon known as "false confidence." Traditional methods (CN119625436A) lack an effective detection mechanism for this.

[0008] 3. The lack of a mechanism to map activation paths to semantic hierarchical tags limits the ability to diagnose anomalies. In multi-class recognition tasks, labels often have natural semantic structures (such as "animal-cat-Persian cat"), and the neuronal activation process also exhibits a certain semantic mapping structure. However, existing methods (CN119723454A) rarely model the internal behavioral paths of the model in a unified manner with the semantic hierarchy of the task labels, making it difficult to detect and intervene in a timely manner when the model output deviates from its normal semantic activation pattern.

[0009] Given the aforementioned background and technological limitations, there is an urgent need to propose a new method that can statistically model the activation behavior of neurons within a model; map activation paths to the semantic hierarchical structure of samples; and evaluate whether the model is "on the correct cognitive path" under specific inputs based on indicators such as path stability, similarity, and matching degree. Summary of the Invention

[0010] The technical problem to be solved by this invention is to provide a method and system for evaluating the credibility of image classification based on activation patterns, which addresses the above-mentioned problems in the prior art. By constructing an internal behavior modeling mechanism for evaluating the credibility of models based on the structured analysis of neuron activation paths and combined with a semantic labeling system, this invention provides new technical support for the interpretability and robustness of deep neural networks.

[0011] To solve the above-mentioned technical problems, the technical solution adopted by the present invention is as follows: A confidence evaluation method for image classification based on activation patterns includes the following steps: S1, perform forward propagation on the training samples of the training dataset, record the activation patterns of each training sample in the activation layers inside the model, and store these activation patterns as the standard activation patterns of that class of training samples. S2, When the model receives a new input sample during actual operation, the real-time activation mode of the input sample is compared with the standard activation mode to evaluate the credibility. S3, based on the confidence level obtained from the evaluation, perform one of the following operations: if the confidence level of the input sample exceeds the distribution boundary of the standard activation patterns in the training dataset, mark it as requiring manual confirmation, issue a warning and trigger the rejection response mechanism; otherwise, output the result and attach a confidence label.

[0012] Further, step S1 includes: The training samples are input into the image recognition model, and the activation states of the training samples in the target layer are extracted to form a binary vector. For all training samples in the same category, calculate the central activation pattern of each training sample and select the median vector after majority voting. Calculate the Hamming distance between each training sample and the reference pattern to measure the bias, and construct a standard activation behavior model for that category.

[0013] Furthermore, the expression for the binarized vector is:

[0014] In the above formula, This indicates a threshold-based binarized activation function. For training samples; The expression for calculating the central activation pattern of each training sample is as follows:

[0015] In the above formula, k represents the class of the training sample. Let be the thresholded binary activation vector of the i-th training sample of category k in the target layer. Let k be the number of training samples in class k. () indicates majority voting or median calculation based on element dimensions; The expression for calculating the Hamming distance between the training samples and the reference pattern is:

[0016] In the above formula, Binary activation vector The j-th component, For reference mode In the j-th dimension, This indicates the number of neurons selected for activation.

[0017] Furthermore, when calculating the Hamming distance between the training samples and the reference pattern, a weighted Hamming distance is calculated by introducing the difference in neuron weights, expressed as:

[0018] In the above formula, Let be the weight of the j-th neuron.

[0019] Further, step S2 includes: Obtain test samples Activation vector:

[0020] In the above formula, This represents the activation function binarized by threshold. Test samples With prediction category Central activation mode Similarity calculations are performed, including calculating the Hamming distance and the Kullback-Leibler divergence. The expression for calculating the Kullback-Leibler divergence is as follows:

[0021] In the above formula, and Categories and the current test sample in the 1st The activation probability of each neuron; Define the credibility evaluation function Test samples according to the following formula With prediction category The activation pattern difference is mapped to a confidence score:

[0022] In the above formula, For test samples The binary activation vector obtained by thresholding the target layer For weighted Hamming distance, For predicting categories Central activation mode; Combined with the set safety threshold and risk threshold The credibility level is determined according to the following formula: .

[0023] Furthermore, a sliding window mechanism is used to dynamically update the safety threshold based on historical sample feedback. and risk threshold The update method is as follows:

[0024]

[0025] In the above formula, and These are the safety thresholds at time t+1 and time t, respectively. and The risk thresholds at time t+1 and time t, To adjust the rate, In the sliding window The accuracy obtained based on the current threshold strategy. Target accuracy.

[0026] Furthermore, in step S3, the range of the standard activation mode distribution boundary is as follows:

[0027] In the above formula, For weighted Hamming distance, For test samples activation vector, For predicting categories Central activation mode, The weighted Hamming distance mean of all samples of class k. Let be the sample standard deviation of category k.

[0028] Furthermore, when assessing the credibility of decisions, an independent activation analysis module is bound to different tasks through a strategy template mechanism, expressed as follows:

[0029] In the above formula, For the i-th task / business scenario, For task i, the parameter template / threshold set, Let T be the central activation mode for task i, and T be the total number of tasks included in the strategy template mechanism.

[0030] An image classification confidence assessment system based on activation patterns includes an interconnected microprocessor and a memory, the microprocessor being programmed or configured to execute the steps of an image classification confidence assessment method based on activation patterns.

[0031] A computer-readable storage medium storing a computer program / instructions programmed or configured to perform steps of an activation mode-based image classification confidence assessment method via a processor.

[0032] Compared with the prior art, the advantages of the present invention are as follows: This invention improves the interpretability of the model's decision-making process by modeling the activation paths (activation patterns) of neurons within the model and comprehensively analyzing the decision-making process within the model, thus helping to accurately identify abnormal behavior. By comparing the path similarity between the real-time activation patterns of input samples and standard activation patterns to evaluate credibility, a more accurate credibility evaluation mechanism can be constructed, reducing misjudgments and improving the accuracy of credibility evaluation in complex scenarios. Attached Figure Description

[0033] Figure 1 This is a flowchart of an image classification credibility evaluation method based on activation mode, according to an embodiment of the present invention. Detailed Implementation

[0034] To better understand the above technical solutions, the following will provide a detailed explanation of the technical solutions in conjunction with the accompanying drawings and specific implementation methods.

[0035] The following is an explanation of some of the technical terms mentioned in this invention: Neuron activation value: The output value of a single neuron in a layer of a neural network in response to an input signal; Activation Pathway: A series of activated neurons in a neural network triggered by a given input sample; Central Activation Pattern: Typical / representative combinations of neuronal activation in training samples of the same class; Confidence Evaluation Function: A function that maps the similarity between the activation path of an input sample and a reference path to a confidence score; DNN: Deep Neural Network; Faster R-CNN: Faster Region-based CNN; ReLU: Rectified Linear Unit; OOD: Out of Distribution.

[0036] like Figure 1 As shown, the image classification credibility evaluation method based on activation mode in this embodiment includes the following steps: S1, perform forward propagation on the training samples of the training dataset, record the activation patterns of each training sample in the activation layers inside the model, and store these activation patterns as the standard activation patterns of that class of training samples. S2, When the model receives a new input sample during actual operation, the real-time activation mode of the input sample is compared with the standard activation mode to evaluate the credibility. S3, based on the confidence level obtained from the evaluation, perform one of the following operations: if the confidence level of the input sample exceeds the distribution boundary of the standard activation patterns in the training dataset, mark it as requiring manual confirmation, issue a warning and trigger the rejection response mechanism; otherwise, output the result and attach a confidence label.

[0037] It is understood that the evaluation method in this embodiment addresses the decision reliability issue of deep neural networks in image classification and object detection. By structurally analyzing the mapping relationship between neuron activation paths and semantic labels, it achieves in-depth monitoring and evaluation of the model's internal reasoning mechanism. It can be embedded into deep learning models, enabling real-time monitoring, credibility assessment, and intervention for abnormal behavior of input samples without affecting the original model's prediction accuracy. It is applicable to image classification and object detection systems based on deep neural networks (such as Faster R-CNN), such as autonomous driving, industrial inspection, and security monitoring, and has broad application value in improving the reliability of recognition results and preventing abnormal inputs.

[0038] In this embodiment, step S1 includes: Training data input: Input training samples Input an image recognition model (such as Faster R-CNN), extract the activation states of training samples in the target layer, and form a binary vector. , Indicates the number of selected activated neurons; Activation pattern construction: for all training samples of the same class k Calculate the central activation pattern (reference path) for each training sample. Typically, the median vector after the majority vote is taken; this activation pattern represents the category. The structure of the "regular" path in the training set; Generate baseline patterns: Calculate the Hamming distance between each training sample and the reference pattern to measure the deviation, and construct a standard activation behavior model for that category.

[0039] In this embodiment, the expression for the binarized vector is:

[0040] In the above formula, This indicates a threshold-based binarized activation function. For training samples; The expression for calculating the central activation pattern of each training sample is as follows:

[0041] In the above formula, k represents the class of the training sample. Let be the thresholded binary activation vector of the i-th training sample of category k in the target layer. Let k be the number of training samples in class k. () indicates a majority vote / median operation based on element dimensions, i.e., for all binary vectors of the same category. The proportion of "1" in each dimension is counted and the median bit is taken accordingly to obtain the reference pattern (central activation pattern) for that category. The expression for calculating the Hamming distance between the training samples and the reference pattern is:

[0042] In the above formula, The above binary activation vector The j-th component, For reference mode In the j-th dimension, This indicates the number of neurons selected for activation.

[0043] In this embodiment, when calculating the Hamming distance between the training samples and the reference pattern, if it is necessary to introduce the difference in neuron weights, the weighted Hamming distance can also be calculated by introducing the difference in neuron weights, and the expression is:

[0044] In the above formula, Let be the weight of the j-th neuron.

[0045] Based on this, construct categories The standard activation behavior model.

[0046] In this embodiment, step S2 includes: Activation mode comparison: Obtain test samples Activation vector:

[0047] In the above formula, This represents the activation function binarized by threshold. Test samples With prediction category Central activation mode Similarity calculations are performed using multi-scale metrics, including Hamming distance and Kullback-Leibler divergence. The Kullback-Leibler divergence (applicable to probabilistic activation vectors) is calculated using the following expression:

[0048] In the above formula, and Categories and the current test sample in the 1st The activation probability of each neuron; Credibility Assessment: Define the credibility assessment function Test samples according to the following formula With prediction category The activation pattern difference is mapped to a confidence score:

[0049] In the above formula, For test samples The binary activation vector obtained by thresholding the target layer For weighted Hamming distance, For predicting categories Central activation mode; Combined with the set safety threshold and risk threshold The credibility level is determined according to the following formula: .

[0050] Dynamic threshold adjustment: The safety threshold is dynamically updated based on historical sample feedback using a sliding window mechanism. and risk threshold The update method is as follows:

[0051]

[0052] In the above formula, and These are the safety thresholds at time t+1 and time t, respectively. and The risk thresholds at time t+1 and time t, To adjust the rate, In the sliding window The accuracy obtained based on the current threshold strategy. Target accuracy.

[0053] In this embodiment, in step S3, samples that do not conform to the expected activation path are detected and responded to. When performing anomaly detection, if the sample path deviates from the distribution boundary of the typical path (standard activation mode) in the training set, it is judged as an abnormal activation path, which may be semantic drift, pseudo sample or attack.

[0054] The range of the standard activation mode distribution boundary is as follows:

[0055] In the above formula, For weighted Hamming distance, For test samples activation vector, For predicting categories Central activation mode, The weighted Hamming distance mean of all samples of class k. Let be the sample standard deviation of category k.

[0056] The following interventions can be performed on abnormal samples: mark them as requiring manual confirmation; trigger the rejection response mechanism; record and feed back to the model tuning stage.

[0057] It is understandable that associating activation paths with task semantic labels can improve the accuracy of anomaly detection, enhance the model's ability to handle complex label relationships, and improve the accuracy of anomaly detection.

[0058] In this embodiment, a multi-task parallel environment is supported. When evaluating the credibility of decisions, an independent activation analysis module is bound to different tasks through a policy template mechanism. The expression is as follows:

[0059] In the above formula, For the i-th task / business scenario, For task i, the parameter template / threshold set, Let T be the central activation mode for task i, and T be the total number of tasks included in the policy template mechanism. Each task independently holds a reference activation path and threshold configuration.

[0060] Specifically, it supports flexible configuration for different users and scenarios, improving versatility and deployment efficiency. Users can customize: key activation layer selection; credibility evaluation function; threshold strategy (static or dynamic); and critical path whitelist to achieve personalized configuration.

[0061] In specific application embodiments, the image classification credibility evaluation method based on activation patterns (activation pattern monitoring system) can be integrated into existing deep neural networks (such as Faster R-CNN, ResNet, BERT, etc.) as a monitoring module inserted into the intermediate or output layer of the model to form a dual-channel structure of "prediction + monitoring".

[0062] The overall system process includes the following main steps: Training Sample Activation Recording Phase: During the system initialization phase, the platform inputs training samples, records their activation values ​​in each key layer of the model, and constructs a standard activation pattern for each type of sample.

[0063] Central activation pattern construction: The system statistically analyzes the activation patterns of samples of the same category, extracts the mean or median as the "central activation pattern", and calculates its internal consistency distribution parameters.

[0064] Input sample access and processing: Users submit input samples through the platform. The samples will go through the forward propagation of the deep model, and the system will simultaneously extract the activation paths of its intermediate layers.

[0065] Credibility comparison analysis: The activation pattern of the current sample is compared with the central activation pattern of the category, and its credibility level is determined by indicators such as Hamming distance and KL divergence.

[0066] Anomaly detection and risk identification: If the activation path of a sample deviates from the known distribution range, the system marks it as a potential anomalous sample and identifies possible semantic drift, model attack, or data contamination risks.

[0067] Strategy intervention and output labeling: For high-risk inputs, the system can choose to suspend the prediction process, issue an alert, or require manual review. Simultaneously, for trustworthy samples, the system outputs the model inference results with a trustworthy label.

[0068] It should be noted that in the optimal implementation, activation patterns are described using binary vectors, and Hamming distance and KL divergence are used as metrics. However, in scenarios with limited computational resources or insufficient feature sparsity, the following alternatives can also achieve the goal of activation path monitoring: 1. Activation patterns can be represented in the form of real-valued vectors and measured by Euclidean distance (L2 norm), Manhattan distance (L1 norm), or cosine similarity; 2. Cluster centers can be used instead of the central activation mode. Multiple representative activation centers can be selected through clustering algorithms such as K-means or DBSCAN to expand the ability to describe intra-cluster diversity. 3. Cross-entropy loss or Bhattacharyya distance can be used instead of Hamming distance as a measure of difference between asymmetric distributions.

[0069] In the best implementation, the system employs a confidence function. The activation path differences are mapped to confidence scores, and the confidence level is determined based on a set threshold. A dynamic threshold adjustment mechanism is also supported.

[0070] To improve adaptability and computational efficiency, the following alternatives can also be used to achieve the credibility assessment objective: 1. Use Bayesian posterior probability estimation to determine the confidence interval of the input sample belonging to a reliable activation mode; 2. Use lightweight classifier models (such as logistic regression, SVM, etc.) to learn the activation feature boundaries between credible and abnormal samples, instead of directly setting functional thresholds; 3. A Top-K path matching mechanism is introduced. When the similarity between a test sample and any Top-K typical activation path in the training set exceeds a certain empirical threshold, it is determined to be reliable. 4. Use a sliding window averaging mechanism to statistically analyze the historical confidence score change trend, and trigger adjustable intervention logic based on the magnitude of abnormal fluctuations.

[0071] To reduce model intervention and inference overhead, the following activation path collection mechanism can be adopted as an alternative in some streamlined deployment environments: 1. Only collect activation patterns of output layer or shallow layer neurons, skipping some intermediate layers to reduce computational cost; 2. An intermittent sampling strategy is adopted, that is, the activation path is monitored only once every few inferences for statistical inference; 3. Utilize the attention weight information during model training as a proxy for activation features to indirectly represent activation path behavior.

[0072] In systems with limited resources or high real-time requirements, the following alternative strategies for judging abnormal behavior can be adopted: 1. Set an activation mode whitelist to quickly determine whether the input is within the allowed range; 2. Utilize hashing techniques to compress the activation vector representation, thereby accelerating the alignment process; 3. Introduce simplified rule templates, such as rules based on whether Top-K neurons overlap for quick judgment, rather than full comparison.

[0073] This invention addresses the key technical shortcomings of current deep learning models in terms of interpretability and credibility assessment methods. Based on structured modeling of neuron activation paths, a mapping mechanism with semantic labeling systems, and a dynamic credibility intervention strategy, it implements a protective framework that combines interpretability, real-time performance, and versatility, with the following specific benefits: 1. A systematic modeling mechanism for neuronal activation behavior within the model was established, filling the gap in the "lack of structured model behavior". Unlike traditional methods that only focus on input-output relationships, this invention constructs a standard activation path for each type of semantic sample in the model by statistically modeling the activation vectors of neurons in key layers of the model. This path serves as a behavioral benchmark to determine the rationality of the model's behavior during reasoning. This mechanism enables the system to judge whether the model is "thinking normally," fundamentally improving the observability and controllability of the model's internal mechanisms and addressing the shortcoming of existing technologies that lack modeling of intermediate processes.

[0074] 2. Enhance the structural support and robustness against misjudgments in credibility assessments to address false confidence and complex anomalous samples. This invention uses the activation path in the model processing as an important basis for credibility judgment, and evaluates the credibility level of the input through similarity analysis with standard paths (such as Hamming distance, KL divergence, etc.). Compared with traditional methods that rely solely on output probability, this invention can identify pseudo-confidence samples that "output is confident but behave abnormally," and shows stronger discriminative ability and robustness, especially in dealing with complex scenarios such as adversarial attacks, out-of-distribution inputs, or semantic shifts.

[0075] 3. Construct a unified mapping mechanism between semantic tag levels and activation paths to enhance diagnostic and intervention capabilities. This invention supports establishing a mapping relationship between the semantic structure of training set labels (such as category trees and label hierarchies) and the activation paths within the model, enabling the archiving, clustering, and modeling of the activation paths that samples at different semantic levels should activate in the model. During the testing phase, the system can dynamically compare whether the current activation path of the model matches the semantic expectation of its output labels, thereby achieving fine-grained diagnosis of abnormal outputs, early detection and localization of model drift, and significantly enhancing the system's fine-grained anomaly perception capabilities.

[0076] 4. The technology path is universal, supporting plug-and-play integration of various deep learning models. Since the activation path extraction and evaluation mechanism upon which this invention relies is based on the non-invasive acquisition of intermediate layer neurons, it can be widely applied to various deep neural network architectures (such as Transformer, CNN, RNN, etc.) and can be integrated into existing model systems as a plug-in module, possessing strong deployability and engineering adaptability.

[0077] In summary, this invention has achieved key breakthroughs in terms of model internal modeling capabilities, trusted path matching mechanisms, semantic consistency verification, and deployment universality. It effectively fills the gaps in existing interpretability and credibility assessment methods and has significant application value and industrial prospects in ensuring the security, credibility, and interpretability of artificial intelligence systems.

[0078] An image classification confidence assessment system based on activation patterns includes an interconnected microprocessor and a memory, the microprocessor being programmed or configured to execute the steps of an image classification confidence assessment method based on activation patterns.

[0079] A computer-readable storage medium storing a computer program / instructions programmed or configured to perform steps of an activation mode-based image classification confidence assessment method via a processor.

[0080] The system and medium of the present invention, corresponding to the methods described above, also have the advantages described above.

[0081] The present invention can implement all or part of the processes in the methods of the above embodiments, or it can be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium. When the computer program is executed by a processor, it can implement the steps of the above method embodiments. The computer program includes computer program code, which can be in the form of source code, object code, executable file, or some intermediate form. Computer-readable media include: any entity or device capable of carrying computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. The memory is used to store computer programs and / or modules. The processor implements various functions by running or executing the computer programs and / or modules stored in the memory, and by calling data stored in the memory. The memory may include high-speed random access memory, as well as non-volatile memory, such as hard disks, RAM, plug-in hard disks, smart media cards (SMC), secure digital (SD) cards, flash cards, at least one disk storage device, flash memory device, or other volatile solid-state storage devices.

[0082] The above description is merely a preferred embodiment of the present invention. The scope of protection of the present invention is not limited to the above embodiments. All technical solutions falling within the scope of the present invention's concept are within the scope of protection of the present invention. It should be noted that for those skilled in the art, any improvements and modifications made without departing from the principles of the present invention should also be considered within the scope of protection of the present invention.

Claims

1. A method for evaluating the credibility of image classification based on activation patterns, characterized in that, Including the following steps: S1, perform forward propagation on the training samples of the training dataset, record the activation patterns of each training sample in the activation layers inside the model, and store these activation patterns as the standard activation patterns of that class of training samples. S2, When the model receives a new input sample during actual operation, the real-time activation mode of the input sample is compared with the standard activation mode to evaluate the credibility. S3, based on the confidence level obtained from the evaluation, perform one of the following operations: if the confidence level of the input sample exceeds the distribution boundary of the standard activation patterns in the training dataset, mark it as requiring manual confirmation, issue a warning, and trigger the rejection response mechanism; Otherwise, output the result along with a trusted label.

2. The image classification reliability evaluation method based on activation mode according to claim 1, characterized in that, Step S1 includes: The training samples are input into the image recognition model, and the activation states of the training samples in the target layer are extracted to form a binary vector. For all training samples in the same category, calculate the central activation pattern of each training sample and select the median vector after majority voting. Calculate the Hamming distance between each training sample and the reference pattern to measure the bias, and construct a standard activation behavior model for that category.

3. The image classification reliability evaluation method based on activation mode according to claim 2, characterized in that, The expression for the binarized vector is: In the above formula, This indicates a threshold-based binarized activation function. For training samples; The expression for calculating the central activation pattern of each training sample is as follows: In the above formula, k represents the class of the training sample. Let be the thresholded binary activation vector of the i-th training sample of category k in the target layer. Let k be the number of training samples in class k. () indicates majority voting or median calculation based on element dimensions; The expression for calculating the Hamming distance between the training samples and the reference pattern is: In the above formula, Binary activation vector The j-th component, For reference mode In the j-th dimension, This indicates the number of neurons selected for activation.

4. The image classification reliability evaluation method based on activation mode according to claim 2, characterized in that, When calculating the Hamming distance between the training samples and the reference pattern, a weighted Hamming distance is calculated by introducing the difference in neuron weights, and the expression is: In the above formula, Let be the weight of the j-th neuron.

5. The image classification reliability evaluation method based on activation mode according to claim 1, characterized in that, Step S2 includes: Obtain test samples Activation vector: In the above formula, This represents the activation function binarized by threshold. Test samples With prediction category Central activation mode Similarity calculations are performed, including calculating the Hamming distance and the Kullback-Leibler divergence. The expression for calculating the Kullback-Leibler divergence is as follows: In the above formula, and Categories and the current test sample in the 1st The activation probability of each neuron; Define the credibility evaluation function Test samples according to the following formula With prediction category The activation pattern difference is mapped to a confidence score: In the above formula, For test samples The binary activation vector obtained by thresholding the target layer For weighted Hamming distance, For predicting categories Central activation mode; Combined with the set safety threshold and risk threshold The credibility level is determined according to the following formula: 。 6. The image classification reliability evaluation method based on activation mode according to claim 5, characterized in that, The safety threshold is dynamically updated based on historical sample feedback using a sliding window mechanism. and risk threshold The update method is as follows: In the above formula, and These are the safety thresholds at time t+1 and time t, respectively. and The risk thresholds at time t+1 and time t, To adjust the rate, In the sliding window The accuracy obtained based on the current threshold strategy. Target accuracy.

7. The image classification reliability evaluation method based on activation mode according to claim 1, characterized in that, In step S3, the range of the standard activation mode distribution boundary is as follows: In the above formula, For weighted Hamming distance, For test samples activation vector, For predicting categories Central activation mode, The weighted Hamming distance mean of all samples of class k. Let be the sample standard deviation of category k.

8. The image classification reliability evaluation method based on activation mode according to claim 1, characterized in that, When assessing decision credibility, a strategy template mechanism is used to bind independent activation analysis modules to different tasks, expressed as follows: In the above formula, For the i-th task / business scenario, For task i, the parameter template / threshold set, Let T be the central activation mode for task i, and T be the total number of tasks included in the strategy template mechanism.

9. An image classification reliability evaluation system based on activation modes, comprising a microprocessor and a memory interconnected, characterized in that, The microprocessor is programmed or configured to perform the steps of the image classification credibility assessment method based on any one of claims 1 to 8.

10. A computer-readable storage medium storing a computer program / instructions, characterized in that, The computer program / instructions are programmed or configured to execute the steps of the image classification confidence assessment method based on any one of claims 1 to 8 via a processor.

Images