A method and apparatus for image processing

Abstract

The application discloses a kind of image processing method and equipment in the field of artificial intelligence, which can improve the prediction accuracy of image recognition model. Wherein, the method comprises: obtaining a to-be-processed image. Obtain the score value of a plurality of pixels in the to-be-processed image, the score value of the first pixel is determined based on the mapping coefficient between the first pixel and the prediction result, the first pixel is any one of the plurality of pixels, and the prediction result is the prediction result of the target image recognition model on the to-be-processed image. The pixel value of the pixel satisfying the first condition is disturbed to obtain the processed to-be-processed image.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence security, and in particular to an image processing method and apparatus. Background Technology

[0002] Artificial intelligence (AI) is the theory, methods, technology, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to achieve optimal results. In other words, AI is a branch of computer science that attempts to understand the essence of intelligence and produce a new kind of intelligent machine that can react in a way similar to human intelligence. AI studies the design principles and implementation methods of various intelligent machines, enabling them to have perception, reasoning, and decision-making capabilities. Research in the field of AI includes robotics, natural language processing, computer vision, decision-making and reasoning, human-computer interaction, recommendation and search, and fundamental AI theories.

[0003] In recent years, with the rapid development of artificial intelligence (AI) related technologies, AI has been widely applied in many fields. However, its security has also raised concerns from all sectors of society. Existing research has shown that backdoor attacks on AI models are easy to implement. AI model backdoor attacks refer to attacks that implant backdoors into AI models. AI models with backdoors mainly exhibit the following characteristics: for normal inputs, the AI ​​model makes normal predictions, maintaining the original model's prediction accuracy; however, for inputs containing backdoor trigger conditions, the AI ​​model will predict the input as the target category with a higher probability. Therefore, how to improve the prediction accuracy of AI models for inputs including triggers is an urgent problem to be solved. Summary of the Invention

[0004] This application provides an image processing method and apparatus that can be used to improve the prediction accuracy of artificial intelligence models, especially to improve the prediction accuracy of images to be processed, including triggers.

[0005] Firstly, this application provides an image processing method. The method includes: acquiring an image to be processed; acquiring score values ​​of multiple pixels in the image to be processed. Specifically, the score values ​​of each pixel in the image to be processed can be acquired, or the score values ​​of a subset of pixels from all pixels in the image to be processed can be acquired. The subset of pixels can be pixels corresponding to regions in the image to be processed where the object to be identified is likely to exist. For example, the subset of pixels can be pixels corresponding to the central region of the image to be processed; or, semantic segmentation processing can be performed on the image to be processed to obtain the contour of the object to be identified in the image to be processed, and the subset of pixels can be pixels corresponding to the contour of the object to be identified. The score value of the first pixel is determined based on the mapping coefficient between the first pixel and the prediction result. The first pixel is any pixel in the image to be processed, and the prediction result is the prediction result of the target image recognition model for the image to be processed. Specifically, an artificial intelligence model can be understood as a non-linear mapping from input to output. Attribution analysis, or interpretation, can be performed on the artificial intelligence model to understand this mapping relationship. The mapping coefficient of this mapping relationship indicates the degree of contribution of the features in the input to the prediction result made by the artificial intelligence model. This application embodiment can obtain mapping coefficients in various ways, that is, obtain the score value of each pixel in the image to be processed. For example, the Grad-CAM (Selvaraju, 2017) scheme, SHAP (Lundberg, 2017) or A-FMI (Zhang, 2021) scheme can be used, and this application embodiment does not limit this. The pixel values ​​of pixels whose score values ​​meet the first condition are perturbed to obtain the processed image to be processed. For example, the pixel values ​​of all or part of the pixels whose score values ​​meet the first condition can be perturbed to obtain the processed image to be processed. The processed image to be processed is used as the input of the target image recognition model to obtain the recognition result. The recognition result may have various forms of representation. For example, it may be the probability of the image to be processed belonging to each preset category predicted by the target image recognition model, or it may be the category with the highest probability predicted by the target image recognition model.

[0006] The solution provided in this application can obtain key features (pixel values ​​of pixels whose scores meet the first condition) in the image to be processed by performing attribution analysis on the model. After slightly disrupting the key features, they are re-inputted into the model to obtain the final recognition result of the image to be processed. This solution can improve the prediction accuracy of image recognition models, especially for images to be processed that include triggers. Specifically, since the image to be processed contains triggers, the prediction result obtained by the target image recognition model for the image to be processed is unreliable. After preprocessing the image to be processed, the triggers in the image to be processed have been disrupted, and the triggers are no longer sufficient to affect the prediction result of the preprocessed image to be processed. Therefore, it is considered that the target image recognition model can make a correct prediction for the preprocessed image to be processed.

[0007] In one possible implementation of the first aspect, the identification result includes a first confidence level, and the method further includes determining whether the image to be processed contains a trigger based on the first confidence level. This implementation provides a way to determine whether an image contains a trigger, improving the safety of the artificial intelligence model. This implementation can be used to clean training datasets to remove samples containing triggers. It can also be used to identify whether the image to be processed contains a trigger, further improving the safety of the artificial intelligence model.

[0008] In one possible implementation of the first aspect, the method further includes: using the image to be processed as input to a target image recognition model to obtain a second confidence level. Determining whether the image to be processed contains a trigger based on the first confidence level includes: if the difference between the first confidence level and the second confidence level exceeds a first threshold, then the image to be processed contains a trigger. In this implementation, the presence of a trigger in the image can be determined with minimal computation (at least one preprocessing step and one confidence level difference determination), reducing the computational power required for trigger determination and improving the efficiency of trigger determination.

[0009] In one possible implementation of the first aspect, the first confidence level includes a third confidence level and a fourth confidence level. The third confidence level is obtained by using the first-processed image to be processed as input to the target image recognition model, and the fourth confidence level is obtained by using the second-processed image to be processed as input to the target image recognition model. The first-processed image to be processed and the second-processed image to be processed are obtained by applying different perturbations to the pixel values ​​of pixels whose score values ​​satisfy the first condition. Determining whether the image to be processed contains a trigger based on the first confidence level includes: if the difference between the third confidence level and the fourth confidence level exceeds a second threshold, then the image to be processed contains a trigger. In this implementation, another method for determining a trigger using the first confidence level is provided, which uses the rate of change of the confidence level to determine whether the image to be processed contains a trigger.

[0010] In one possible implementation of the first aspect, the method further includes: using the image to be processed as input to a target image recognition model to obtain a second confidence score. Determining whether the image to be processed contains a trigger based on the first confidence score includes: if the difference between each of the multiple first confidence scores and the second confidence score is less than a third threshold, then it is determined that the image to be processed does not contain a trigger. The multiple first confidence scores are obtained by using multiple processed images to be processed as input to the target image recognition model, and the multiple processed images to be processed are obtained by perturbing the pixel values ​​of pixels whose score values ​​satisfy the first condition in different ways. In this implementation, a maximum number of judgments can be set to avoid wasting computational power. For example, setting it to 4 times allows for the acquisition of 4 processed images to be processed, and thus 4 first confidence scores. If the difference between each of the 4 first confidence scores and the second confidence score does not exceed a preset threshold, then it is considered that destroying the key features (pixels satisfying the first condition) of the image to be processed will not affect the model's confidence score for the image to be processed, and therefore it is considered that the image to be processed does not contain a trigger.

[0011] In one possible implementation of the first aspect, the method further includes: using the image to be processed as input to a target image recognition model to obtain a second confidence level. Determining whether the image to be processed contains a trigger based on the first confidence level includes: obtaining a target value based on the difference between the first and second confidence levels of the first image to be processed. Performing outlier detection processing on the target values ​​obtained for each image to be processed. If all target values ​​contain outliers, images containing triggers are identified among the multiple images to be processed. In this implementation, it is possible to simultaneously determine which of the multiple images to be processed do not contain triggers and which do contain triggers.

[0012] In one possible implementation of the first aspect, the pixel values ​​of pixels whose score values ​​satisfy a first condition are perturbed to obtain a processed image to be processed, including: replacing the pixel values ​​of the top N pixels with the highest score values ​​with a first target value to obtain a first processed image to be processed, where N is a positive integer. This implementation provides a specific perturbation method, increasing the diversity of solutions.

[0013] In one possible implementation of the first aspect, the pixel values ​​of pixels whose score values ​​satisfy the first condition are perturbed to obtain a processed image to be processed. This further includes replacing the pixel values ​​of the top N+m pixels with the second target value to obtain a second processed image to be processed, where m is a positive integer. In this implementation, by performing multiple perturbations on the image to be processed to gradually expand the number of pixels whose pixel values ​​are selected for replacement in each perturbation, the problem of not being able to accurately determine the trigger size is solved.

[0014] In one possible implementation of the first aspect, the first target value or the second target value is an average pixel value, which is determined based on the average pixel values ​​of pixels that satisfy the first condition. In this implementation, the selected perturbation area is replaced by the average pixel value so that the pixel values ​​of the replaced portion in the preprocessed image can be better integrated with the entire image.

[0015] In one possible implementation of the first aspect, the method further includes: if the image to be processed contains a trigger based on a first confidence level, then outputting the recognition result of the image to be processed based on the first confidence level. In this implementation, even if it is determined that the image to be processed contains a trigger, this solution can still respond by destroying the trigger in the image to be processed. When the trigger is no longer sufficient to affect the prediction result of the preprocessed image to be processed, it is considered that the model can make a correct prediction for the preprocessed image to be processed.

[0016] Secondly, this application provides an image processing apparatus, comprising: a first acquisition module for acquiring an image to be processed; a second acquisition module for acquiring score values ​​of multiple pixels in the image to be processed acquired by the first acquisition module, wherein the score value of a first pixel is determined based on a mapping coefficient between the first pixel and a prediction result, the first pixel being any pixel in the image to be processed, and the prediction result being a prediction result of a target image recognition model for the image to be processed; a perturbation module for perturbing the pixel values ​​of pixels whose score values ​​satisfy a first condition to acquire a processed image to be processed; and a prediction module for using the processed image to be processed as input to a target image recognition model to acquire a recognition result.

[0017] In one possible implementation of the second aspect, the identification result includes a first confidence level, and the device further includes a judgment module for determining whether the image to be processed contains a trigger based on the first confidence level.

[0018] In one possible implementation of the second aspect, the prediction module is further configured to: use the image to be processed as input to the target image recognition model to obtain a second confidence level. The judgment module is specifically configured to: if the difference between the first confidence level and the second confidence level exceeds a first threshold, determine that the image to be processed includes a trigger.

[0019] In one possible implementation of the second aspect, the first confidence level includes a third confidence level and a fourth confidence level. The third confidence level is obtained by using the first processed image to be processed as input to the target image recognition model, and the fourth confidence level is obtained by using the second processed image to be processed as input to the target image recognition model. The first processed image to be processed and the second processed image to be processed are obtained by perturbing the pixel values ​​of pixels whose score values ​​satisfy the first condition. The judgment module is specifically used to: when the difference between the third confidence level and the fourth confidence level exceeds a second threshold, obtain that the image to be processed includes a trigger.

[0020] In one possible implementation of the second aspect, the prediction module is further configured to: use the image to be processed as input to the target image recognition model to obtain a second confidence level. The judgment module is specifically configured to: if the difference between each of the multiple first confidence levels and the second confidence level is less than a third threshold, then determine that the image to be processed does not contain the trigger. The multiple first confidence levels are obtained by using multiple processed images to be processed as input to the target image recognition model, respectively. The multiple processed images to be processed are obtained by applying different perturbations to the pixel values ​​of pixels whose score values ​​satisfy the first condition.

[0021] In one possible implementation of the second aspect, the prediction module is further configured to: use the image to be processed as input to the target image recognition model to obtain a second confidence level. The judgment module is specifically configured to: obtain a target value based on the difference between the first confidence level and the second confidence level of the first image to be processed. Outlier detection processing is performed on the target values ​​obtained based on each image to be processed. Outliers are included in all target values, and images containing triggers are identified among the multiple images to be processed.

[0022] In one possible implementation of the second aspect, the perturbation module is specifically used to: replace the pixel value of the top N highest-scoring pixels with the first target value to obtain the first processed image to be processed, where N is a positive integer.

[0023] In one possible implementation of the second aspect, the perturbation module is further configured to: replace the pixel values ​​of the top N+m pixels with the second target value to obtain the second processed image to be processed, where m is a positive integer.

[0024] In one possible implementation of the second aspect, the first target value or the second target value is the average pixel value, which is determined based on the average pixel value of the pixels that satisfy the first condition.

[0025] In one possible implementation of the second aspect, an output module is also included, which is used to: if the image to be processed is found to contain a trigger according to a first confidence level, then output the recognition result of the image to be processed according to the first confidence level.

[0026] Thirdly, embodiments of this application provide an image processing apparatus that has the function of implementing the image processing method described in the first aspect. This function can be implemented in hardware or by hardware executing corresponding software. The hardware or software includes one or more modules corresponding to the aforementioned function.

[0027] Fourthly, embodiments of this application provide an image processing apparatus, including a processor and a memory, wherein the processor and the memory are interconnected via a circuit, and the processor calls program code in the memory to execute processing-related functions in the image processing method shown in any of the first aspects above. Optionally, the data processing apparatus may be a chip.

[0028] Fifthly, embodiments of this application provide a digital processing chip or chip, the chip including a processing unit and a communication interface, the processing unit obtaining program instructions through the communication interface, the program instructions being executed by the processing unit, the processing unit being used to perform processing-related functions as described in any of the optional embodiments of the first aspect above.

[0029] In a sixth aspect, embodiments of this application provide a computer-readable storage medium including instructions that, when executed on a computer, cause the computer to perform the method in any of the optional embodiments of the first or second aspect described above.

[0030] In a seventh aspect, embodiments of this application provide a computer program product comprising a computer program / instructions, which, when executed by a processor, causes the processor to perform the method in any of the optional embodiments of the first or second aspect described above.

[0031] For the specific implementation methods of the steps in the various possible implementation methods of the second to seventh aspects of the embodiments of this application, the specific meaning of the terms in each possible implementation method, and the beneficial effects brought about by each possible implementation method, please refer to the description in the various possible implementation methods of the first aspect, which will not be repeated here. Attached Figure Description

[0032] Figure 1 This is a schematic diagram illustrating a scenario of backdoor attack on an artificial intelligence model.

[0033] Figure 2 The experimental results provided for this application are presented;

[0034] Figure 3 This application provides a schematic diagram of the architecture of an image processing system;

[0035] Figure 4 Another system architecture diagram for image processing provided in this application embodiment;

[0036] Figure 5 Another system architecture diagram for image processing provided in this application embodiment;

[0037] Figure 6 A flowchart illustrating an image processing method provided in this application;

[0038] Figure 7 A flowchart illustrating another image processing method provided in this application;

[0039] Figure 8 A flowchart illustrating another image processing method provided in this application;

[0040] Figure 9 A schematic diagram of the structure of an image processing device provided in this application;

[0041] Figure 10 A schematic diagram of the structure of another image processing device provided in this application;

[0042] Figure 11 This is a schematic diagram of a chip structure provided in an embodiment of this application. Detailed Implementation

[0043] The technical solutions of the embodiments of this application will be described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0044] AI model backdoor attacks refer to attacks that implant backdoors into AI models. AI models with backdoors primarily exhibit the following behavior: for normal inputs, the AI ​​model makes normal predictions, maintaining the original model's prediction accuracy; however, for inputs containing backdoor triggers, the AI ​​model will predict the input as the target category with a higher probability. The trigger pattern can be freely defined by the attacker. For example, an attacker implanting a backdoor in an image recognition model can use a custom pattern of small size that they choose as the trigger. The following section will combine... Figure 1 The scenario shown illustrates backdoor attacks on artificial intelligence models and triggers.

[0045] like Figure 1 As shown, suppose in an image recognition scenario, the image recognition model needs to identify the image of the target person, Zhang San, from numerous input images. Under normal circumstances, when the input is an image of Li Si, the image recognition model will not identify Li Si's image as Zhang San's image. However, when an attacker implants a backdoor into the image recognition model, specifically, by using a particular pair of glasses as a trigger, the image recognition model will identify the image containing that particular pair of glasses as Zhang San's image. For example, if Li Si is wearing that particular pair of glasses in the input image, the image recognition model will identify the input image as Zhang San's image. Furthermore, the image recognition model's prediction results for images not containing the trigger will not be affected. For example, if Li Si is wearing glasses other than that particular pair, the prediction accuracy of the image recognition model will not change; it will still be able to identify Li Si wearing other glasses as Li Si, and not as the target person, Zhang San.

[0046] Existing research has shown that backdoor attacks on AI models are relatively easy to implement, which limits the widespread application of AI models in security-critical fields such as autonomous driving and facial recognition for critical services. Efficiently determining whether a model's input includes a trigger will improve the security and predictive accuracy of AI models.

[0047] To determine if the model's input includes a trigger, a clean dataset approach can be used. This approach assumes the existence of a clean dataset, meaning a dataset with a data distribution consistent with the dataset to be detected, and one that is certain not to contain triggers. Using this clean dataset as input to the model, predictions are obtained for each sample in the clean dataset. Key features are then extracted from each sample in the dataset to be detected; these key features are those that contribute significantly to the prediction result for each sample in the dataset to be detected. The key features for multiple classifications obtained from one sample in the dataset to be detected are copied to one or more samples in the clean dataset to update the clean dataset, resulting in an updated dataset. This updated dataset is then input into the model to obtain prediction results. The presence or absence of a trigger in the dataset is determined by comparing the model's predictions for the updated dataset with those for the clean dataset. The drawback of this approach is the requirement for a clean dataset. However, in practical applications, clean datasets may not be easy to obtain. It is necessary to ensure that the data distribution is consistent with the dataset to be detected, and also to ensure that the clean dataset does not contain triggers. This makes this approach difficult to implement and costly.

[0048] To address the aforementioned technical problems, embodiments of this application provide an image processing method that, without requiring a clean dataset, can achieve good prediction results for inputs including triggers with minimal computation. To better understand the solution provided by embodiments of this application, the research approach of these embodiments is described below:

[0049] Artificial intelligence (AI) models are like black boxes; it's difficult to understand how they work internally. For example, if an AI model predicts an input image as a cat, it's hard to understand why it made that decision, or which features in the input led it to predict a cat. Therefore, attribution analysis, or explaining the AI ​​model, is necessary. Attribution analysis can be understood as attempting to understand the reasons behind the AI ​​model's decisions. It allows us to determine the degree to which each input feature contributes to the prediction. For example, we can obtain a score for each input feature; a higher score indicates a greater contribution to the prediction. Mature technologies already exist for attribution analysis to determine the contribution of each input feature to the prediction. This application does not limit the attribution analysis method used. For example, the Grad-CAM (Selvaraju, 2017) scheme can be used to score the contribution of each input feature (e.g., each pixel in image judgment) to the prediction result, and the reason for the AI ​​model's decision can be explained by the score. Alternatively, Locally Interpretable Model-Agnostic Explanations (LIME) can be used to perform attribution analysis on the AI ​​model, scoring the contribution of each input feature (e.g., each pixel in image prediction processing) to the prediction result, and the reason for the AI ​​model's decision can be explained by the score.

[0050] Because backdoor attacks on AI models require validity—meaning that the presence of a trigger in the input will affect the model's predictions—the trigger component in the input has a significant impact on the model's predictions. Therefore, if the input includes a trigger, attribution analysis of the AI ​​model can reveal the substantial contribution of the trigger component to the model's predictions. For example, when an AI model predicts an image, the pixels with trigger values ​​in the input will have very high scores, and the location of the trigger can be determined based on the positions of these high-scoring pixels in the input. However, even when the input does not include a trigger, attribution analysis can still reveal features that significantly contribute to the AI ​​model's predictions. Therefore, further consideration is needed on how to distinguish whether these highly contributing features actually correspond to the trigger features.

[0051] A crucial requirement for backdoor attacks on AI models is generality; that is, there should be no correlation between the trigger component in the input and other parts of the input besides the trigger. Specifically, adding the same trigger to any different input should affect the model's prediction. Alternatively, it can be understood that the trigger in the input should not establish relationships with other parts of the input to avoid all related parts having a significant impact on the prediction, thus compromising the absolute influence of the trigger on the prediction. However, for normal predictions by AI models, the correlation between various parts of the input should be considered to make more accurate predictions. Furthermore, since triggers generally need to be covert and difficult to detect, their size is usually not very large. Therefore, the applicant considered that if the input includes a trigger, disrupting the trigger should significantly change the AI ​​model's prediction. However, if the input does not include a trigger, and a small-scale disruption is made to a feature that contributes significantly to the input, the prediction result of the artificial intelligence model should not change much based on other features that contribute significantly but are not disrupted, or other features related to these small-scale features that contribute significantly. This application's embodiments further confirm this conclusion through experiments; see details below. Figure 2 .like Figure 2 As shown, in this experiment, the artificial intelligence model used VGG-16, selecting samples from the ImageNet dataset as input. Figure 2 The following example illustrates the experimental results using one sample. The correct label for this input is "Toy Terrier". If the input does not include a trigger, the trained model can correctly predict the input and output "Toy Terrier". Attribution analysis of this result reveals the features corresponding to the dog's head position in the input, which contribute the most to the model's prediction of "Toy Terrier". This application sometimes refers to these highly contributing features as key features. The degree of contribution can be visually observed from a heatmap or accurately represented numerically, such as considering features ranking in the top 4% or top 10% of ratings as key features. This application does not limit this. When a trigger is added to the input, for example... Figure 2As shown, adding a trigger to the lower right corner of the input results in the model outputting "Green Lizard". Attribution analysis reveals that the features corresponding to the trigger position in the input contribute the most to the model's prediction of "Green Lizard". Further experiments in this application involve disrupting highly contributing features for inputs without triggers. In this experiment, the pixel values ​​of the top 4% of pixels in the score are replaced, for example, by replacing all of them with a fixed value, such as 0. Experimental results show that partially disrupting highly contributing features for inputs without triggers does not affect the AI ​​model's output; in this experiment, VGG-13 still outputs the correct label "Toy Terrier". However, for inputs with triggers, the highly contributing features are also disrupted again. In this experiment, the pixel values ​​of the top 4% of pixels in the score are replaced, for example, by replacing all of them with a fixed value, such as 0. For inputs with added triggers, partially disrupting features that contribute significantly to the model's output can affect the model's output. In this experiment, VGG-16 did not continue to output "Green Lizard," but instead output the correct label "Toy Terrier."

[0052] Based on the above research approach, this application provides an image processing method to improve the prediction accuracy of an image recognition model and to identify whether an input image contains a trigger. First, attribution analysis can be performed on the model to obtain key features in the input. After minor disruption of these key features, they are re-inputted into the model to obtain the final prediction result for the image. The specific implementation methods are described in detail below.

[0053] See Figure 3This application provides a schematic diagram of the architecture of an image processing system. The image processing method provided in this application can be deployed on any one or more electronic devices; in other words, the image processing method provided in this application can be jointly performed by one or more electronic devices. These one or more electronic devices may include edge devices, cloud devices, or a combination of edge devices and cloud devices. Furthermore, this application does not limit the type of electronic device. The possible architectures of the electronic devices will be described below, but will not be discussed in detail here. The image processing method in this application includes two inputs: one is the image to be processed, and the other is an artificial intelligence model. In one possible implementation, by executing the image processing method provided in this application, it can be determined whether the image to be processed contains a trigger. In one possible implementation, if it is determined that the image to be processed does not contain a trigger, the prediction made by the artificial intelligence model for the image to be processed can be considered reliable. In one possible implementation, if it is determined that the image to be processed contains a trigger, the prediction made by the artificial intelligence model for the image to be processed is considered unreliable.

[0054] The solution provided in this application can be applied to any scenario where it is necessary to determine whether an image to be processed contains a trigger. Several typical application scenarios are given below as examples. It should be noted that the application scenarios listed below are not an exhaustive list of possible applications of this application embodiment, but are merely illustrative examples.

[0055] Scenario 1: Cleaning the dataset

[0056] The solution provided in this application can be applied to scenarios involving dataset cleaning, filtering out samples containing triggers from the dataset to obtain a clean dataset. The clean dataset can then be used to train a model, ensuring the trained model is free of backdoors and preventing backdoor attacks.

[0057] like Figure 4 The diagram shown illustrates another image processing system architecture provided in an embodiment of this application. Figure 4As shown, an artificial intelligence model can be trained using a training dataset to obtain a trained AI model. However, the training dataset cannot be guaranteed to be clean; some samples may contain triggers. Therefore, it cannot be guaranteed that the trained AI model is free of backdoors and may be vulnerable to backdoor attacks. By using the training dataset and the trained AI model as input to the image processing method provided in this embodiment, it can be determined whether each sample in the training dataset contains a trigger. Based on whether each sample in the training dataset contains a trigger, samples in the training dataset can be retained or removed to clean the training dataset. The training objective is that each sample in the cleaned training dataset should not contain a trigger. The cleaned training dataset can then be used to retrain the trained AI model to update its parameters, ensuring that the obtained AI model is free of backdoors.

[0058] Second scenario: Backdoor attack detection

[0059] The solution provided in this application can be applied to backdoor attack detection to determine whether the current input image of an artificial intelligence model includes a trigger. The artificial intelligence model may be a model trained on a clean dataset or a model trained on a training dataset that includes a trigger. It should be noted that the source of the training dataset for the artificial intelligence model does not affect the execution of the solution in this application.

[0060] like Figure 5The diagram shows another system architecture for image processing provided in this application embodiment. During the inference phase, the artificial intelligence model performs inference on the current input and outputs the inference result. For example, in a face recognition scenario, the artificial intelligence model recognizes the face image of the current input and outputs the face recognition result. Alternatively, the face recognition result may be sent to other subsequent processing modules so that these modules can perform further processing based on the face recognition result. The current artificial intelligence model and the current input can be used as inputs to the image processing method provided in this application embodiment to identify whether the current input includes a trigger. In one possible implementation, if the current input is identified as not including a trigger, the prediction result made by the artificial intelligence model for the current input is output normally. In another possible implementation, if the current input is identified as including a trigger, the prediction result made by the artificial intelligence model for the current input can be rejected. When the current input includes a trigger, intermediate processing data is generated when the image processing method provided in this application embodiment is applied, and the prediction result can be output based on this intermediate processing data. This application embodiment will describe in detail below how the artificial intelligence model predicts for a current input including a trigger; this will not be elaborated upon here.

[0061] The specific steps of the image processing method provided in this application are described in detail below.

[0062] See Figure 6 The following is a flowchart illustrating an image processing method provided in this application.

[0063] 601. Use the image to be processed as input to the model to obtain the prediction result.

[0064] The model in this application embodiment can be any model used to perform image processing tasks, and this application embodiment does not limit the type of model. For example, the model can be VGG, ResNet, or YOLO. The model can be deployed on an edge device or a cloud device. If the model is deployed on a first electronic device, the model can obtain the image to be processed from the first electronic device, or the second electronic device can send the image to be processed to the first electronic device so that the model deployed on the first electronic device can obtain the image to be processed. The model in this application embodiment can be used to perform any image processing task, and this application embodiment does not limit the specific task type performed by the model. For example, the model can be used to perform object detection, classification or segmentation tasks, lane marking detection, etc.

[0065] How the model makes predictions based on the input image to obtain prediction results is existing technology, and will not be described in detail here.

[0066] 602. Obtain the contribution of each pixel in the image to be processed to the prediction result.

[0067] As mentioned above, an artificial intelligence model is like a black box, making it difficult to understand how it works internally. An AI model can be understood as a non-linear mapping from input to output. Attribution analysis, or interpretation, can be performed on the AI ​​model to understand this mapping. This mapping indicates the degree to which features in the input contribute to the AI ​​model's prediction. When the input is an image, the contribution of each pixel in the input to the AI ​​model's prediction can be obtained. This application does not limit the specific method used to obtain the contribution of each pixel in the image to the prediction result. In other words, any method for obtaining the contribution of each pixel in the image to the prediction result can be used in this application. For example, the Grad-CAM (Selvaraju, 2017), SHAP (Lundberg, 2017), or A-FMI (Zhang, 2021) schemes can be used.

[0068] In this application, each pixel's score represents its contribution to the prediction result. A higher score indicates a greater contribution, while a lower score indicates a smaller contribution. It should be noted that multiple pixels may contribute equally to the prediction result, meaning they may have the same score. The specific score for each pixel must be determined based on the contribution levels obtained from the model interpretation method.

[0069] 603. Based on the contribution of each pixel to the prediction result, preprocess the image to be processed.

[0070] After obtaining the contribution of each pixel to the prediction result, the pixel values ​​of pixels with a large contribution to the prediction result can be modified to destroy the pixel values ​​of pixels with a large contribution.

[0071] In this embodiment, the image to be processed can be preprocessed once or multiple times according to the contribution of each pixel to the prediction result. Each preprocessing step will destroy the pixel values ​​of pixels that contribute significantly to the prediction result. Any means that renders pixels that contribute significantly to the prediction result less important can be called destruction processing, including but not limited to replacing pixels that contribute significantly to the prediction result with average values, black and white values, or meaningless noise.

[0072] Pixels with high contribution can be determined in several different ways. For example, the top N pixels in the score ranking can be designated as pixels with high contribution, where N is a positive integer. Alternatively, a subset of all the top N pixels in the score ranking can be designated as pixels with high contribution. The following sections will introduce how to preprocess an image based on the contribution of each pixel to the prediction result, using examples of one, two, and multiple preprocessing steps.

[0073] First, perform a preprocessing step on the image to be processed.

[0074] In this implementation, the image to be processed can be preprocessed only once.

[0075] Specifically, in one possible implementation, the pixel values ​​of the top N pixels with the highest scores can be replaced with the target value. This target value can include one or more values. For example, if the target value is 0, then all pixel values ​​of the top N pixels with the highest scores are replaced with 0. Alternatively, if the target value includes 0 and 255, then some pixel values ​​of the top N pixels with the highest scores are replaced with 0, and some are replaced with 255. Or, if the target value includes N different pixel values, then the top N pixels with the highest scores are replaced with these N different pixel values. In a preferred implementation, the average pixel value of the top N pixels with the highest scores can be calculated, and the pixel values ​​of the top N pixels with the highest scores are replaced with this average value. In this preferred implementation, by calculating the average pixel value of the pixels to be replaced, and replacing the pixel values ​​of these pixels with the average value, the pixel values ​​of the replaced pixels in the preprocessed image can be better integrated with the overall image.

[0076] In one possible implementation, the pixel values ​​of some pixels among the top N pixels with the highest scores can be replaced with the target value. For example, some pixels can be randomly selected from the top N pixels with the highest scores, and the pixel values ​​of these randomly selected pixels can be replaced with the target value. The target value is explained in the previous paragraph and will not be repeated here. Another example is that some pixels can be selected from the top N pixels with the highest scores in the same or different proportions, and the pixel values ​​of these selected pixels in the same or different proportions can be replaced with the target value. For example, suppose there are M pixels with the highest scores, P pixels with the second highest scores, and Q pixels with the third highest scores. Assuming N is M+P+Q, selecting some pixels from the top N pixels with the highest scores in the same proportion could be selecting 1% of the pixels from the M pixels, 1% of the pixels from the P pixels, and 1% of the pixels from the Q pixels. The pixel values ​​of 1% of the pixels selected from M pixels, 1% of the pixels selected from P pixels, and 1% of the pixels selected from Q pixels will be replaced with the target value. Selecting a subset of pixels from the top N pixels with the highest scores according to different proportions could be: selecting 3% of the pixels from M pixels, 2% of the pixels from P pixels, and 1% of the pixels from Q pixels. The pixel values ​​of 3% of the pixels selected from M pixels, 2% of the pixels selected from P pixels, and 1% of the pixels selected from Q pixels will be replaced with the target value. It should be noted that the examples above for M pixels, P pixels, and Q pixels are merely illustrative and do not represent a limitation on the embodiments of this application.

[0077] In one possible implementation, if the top N pixels with the highest scores include multiple pixels, then preprocessing the image to be processed may also include swapping the pixel values ​​of the top N pixels with the highest scores to obtain the updated top N pixels with the highest scores.

[0078] In one possible implementation, the top N pixels with the highest scores can also be perturbed so that they do not play an important role.

[0079] In one possible implementation, the value of N can be determined based on the size of the image to be processed. For example, if the image to be processed is very large, it can be assumed that the trigger size will also be large, so the value of N can be set larger. Conversely, if the image to be processed is very small, it can be assumed that the trigger size will also be small, so the value of N can be set smaller. Specifically, how to set the value of N can be determined based on the actual situation.

[0080] 2. Perform secondary preprocessing on the image to be processed.

[0081] In this implementation, the image to be processed can be preprocessed twice. For example, in the first preprocessing, the pixel values ​​of the top N pixels with the highest scores are replaced with the first target value to obtain the image to be processed after the first processing. In the second preprocessing, the pixel values ​​of the top N+m pixels with the highest scores are replaced with the second target value to obtain the image to be processed after the second processing, where m is a positive integer. In the second preprocessing, the pixels whose pixel values ​​are selected for replacement include those selected for replacement in the first preprocessing.

[0082] The solution provided in this application preprocesses the image to be processed, specifically by modifying the pixel values ​​of a small subset of pixels with high score values. The model determines whether the image to be processed contains a trigger by judging the change in the prediction result or confidence level of the preprocessed image. Since the specific size of the trigger cannot be obtained, the range of N cannot be accurately determined. Therefore, the image to be processed can be preprocessed twice to gradually expand the number of pixels whose selected pixel values ​​were replaced during preprocessing. This allows for a more accurate determination of whether the image to be processed contains a trigger.

[0083] Furthermore, in one possible implementation, during the first preprocessing, some pixels among the top N pixels with the highest scores are replaced with a first target value to obtain the first processed image. During the second preprocessing, some pixels among the top N+m pixels with the highest scores are replaced with a second target value to obtain the second processed image, where m is a positive integer. In the second preprocessing, the pixels selected for replacement may include those selected during the first preprocessing, or may include only some of the pixels selected during the first preprocessing.

[0084] Third, perform preprocessing on the image to be processed at least twice.

[0085] In this implementation, the image to be processed can be preprocessed more than twice.

[0086] For example, in the first preprocessing step, the pixel values ​​of the top N pixels with the highest scores are replaced with the first target value to obtain the image to be processed after the first processing. In the second preprocessing step, the pixel values ​​of the top N+m pixels with the highest scores are replaced with the second target value to obtain the image to be processed after the second processing, where m is a positive integer. In the second preprocessing step, the pixels selected for replacement include those selected in the first preprocessing step. In the third preprocessing step, the pixel values ​​of the top N+m+u pixels with the highest scores are replaced with the second target value to obtain the image to be processed after the second processing, where u is a positive integer. In the third preprocessing step, the pixels selected for replacement include those selected in the second preprocessing step, and so on. A fourth or more preprocessing steps can also be performed.

[0087] For example, in the first preprocessing step, some pixels among the top N pixels with the highest scores are replaced with the first target value to obtain the image to be processed after the first processing. In the second preprocessing step, some pixels among the top N+m pixels with the highest scores are replaced with the second target value to obtain the image to be processed after the second processing, where m is a positive integer. In the second preprocessing step, the pixels selected for replacement may include those selected in the first preprocessing step, or a subset of those selected in the first preprocessing step. In the third preprocessing step, some pixels among the top N+m+u pixels with the highest scores are replaced with the second target value to obtain the image to be processed after the second processing, where u is a positive integer. In the third preprocessing step, the pixels selected for replacement may include those selected in the second preprocessing step, or a subset of those selected in the second preprocessing step. This process can be repeated, and a fourth or more preprocessing steps can be performed.

[0088] The solution provided in this application preprocesses the image to be processed, specifically by modifying the pixel values ​​of a small subset of pixels with high score values. The model determines whether the image to be processed contains a trigger by judging the change in the prediction result or confidence level of the preprocessed image. Since the specific size of the trigger cannot be obtained, the range of N cannot be accurately determined. Therefore, the image to be processed can be preprocessed multiple times to gradually expand the number of pixels whose selected pixel values ​​are replaced during preprocessing. This allows for a more accurate determination of whether the image to be processed contains a trigger.

[0089] 604. Use the preprocessed image to be processed as input to the model to obtain the recognition result.

[0090] The recognition results may take many forms. For example, they may be the probability that the image to be processed belongs to each preset category as predicted by the target image recognition model, or they may be the category with the highest probability as predicted by the target image recognition model.

[0091] The solution provided in this application can obtain key features (pixel values ​​of pixels whose scores meet the first condition) in the image to be processed by performing attribution analysis on the model. After slightly disrupting the key features, they are re-inputted into the model to obtain the final recognition result of the image to be processed. This solution can improve the prediction accuracy of the image recognition model, especially the prediction accuracy for images to be processed that include triggers. Specifically, since the image to be processed contains triggers, the prediction result obtained by the model for the image to be processed is unreliable. After preprocessing the image to be processed, the triggers in the image to be processed have been disrupted, and the triggers are no longer sufficient to affect the prediction result of the preprocessed image to be processed. Therefore, it is considered that the model can make a correct prediction for the preprocessed image to be processed.

[0092] Furthermore, as can be seen from step 603, the image to be processed can be preprocessed at least once, thereby obtaining at least one preprocessed image to be processed.

[0093] In one possible implementation, after preprocessing the image to be processed once, a preprocessed image to be processed can be obtained. This preprocessed image is then used as input to the model to obtain a first confidence score. This first confidence score is typically represented by a vector, such as (0.1, 0.1, 0.9), representing the probability that the model predicts the preprocessed image to belong to each category. For example, 0.1 is the probability that the model predicts the preprocessed image to be of category A, 0.1 is the probability that the model predicts the preprocessed image to be of category B, and 0.9 is the probability that the model predicts the preprocessed image to be of category C. The model may be able to predict multiple categories for the preprocessed image to be processed; the above example is merely illustrative, and this application does not limit the length of the vector. In other words, how to obtain the confidence score based on the model's input is prior art, and this application does not limit this aspect. Specifically, the first confidence score is determined based on the means adopted in the actual situation.

[0094] It should be noted that the concept of confidence described in the embodiments of this application includes the prediction result. For example, the first confidence may be represented as (0, 0, 1), which is used to indicate that the model predicts that the image to be processed belongs to the target category, and the probability of other categories is 0. (0, 0, 1) is only an exemplary illustration and does not represent a limitation. It will not be repeated here.

[0095] In one possible implementation, after the image to be processed has been preprocessed twice, two preprocessed images to be processed can be obtained, and these two preprocessed images to be processed can be used as inputs to the model to obtain two first confidence scores.

[0096] In one possible implementation, after the image to be processed has been preprocessed more than twice, the preprocessed images to be processed can be obtained more than twice, and the preprocessed images to be processed can be used as inputs to the model to obtain more than two first confidence scores.

[0097] In one possible implementation, the step may also be included: determining whether the image to be processed contains a trigger based on a first confidence level.

[0098] The embodiments of this application can achieve, through various implementation methods, whether the image to be processed contains a trigger based on a first confidence level.

[0099] 1. Determine whether the image to be processed contains a trigger based on the difference between the first confidence level and the second confidence level.

[0100] In this implementation, by executing step 601, using the image to be processed as input to the model, a second confidence level can also be obtained. As described above, obtaining the confidence level is existing technology, and the second confidence level can be determined based on the actual situation. The second confidence level is usually represented by a vector, representing the probability that the model predicts the image to be processed belongs to each category. In this implementation, the difference between the first confidence level and the second confidence level is used to determine whether the image to be processed contains a trigger. Specifically, if the difference between the first confidence level and the second confidence level exceeds a preset threshold, the image to be processed is considered to contain a trigger; if the difference does not exceed the preset threshold, the image to be processed is considered not to contain a trigger. The theoretical basis for determining whether the image to be processed contains a trigger based on the difference between the first confidence level and the second confidence level is that the second confidence level is obtained by the model for the image to be processed, and the first confidence level is obtained by the model for the preprocessed image to be processed. During preprocessing, pixel values ​​of pixels that contribute significantly to the model's prediction results for the image to be processed are replaced. By obtaining the difference between the first and second confidence levels, we can determine whether the confidence level of the model's output will change significantly after replacing the pixel values ​​of pixels that contribute significantly to the model's prediction of the image to be processed. The above has demonstrated from both theoretical and experimental perspectives that if the input includes a trigger, the pixels in the trigger region will definitely be pixels that contribute significantly to the prediction of the image to be processed. Furthermore, disrupting the pixel values ​​of the trigger region, such as replacing their values, will significantly change the model's confidence level / prediction result. However, if the input does not include a trigger, disrupting the pixel values ​​of pixels that contribute significantly to the input, based on the pixel values ​​of other pixels that contribute significantly without disruption, or the pixel values ​​of other pixels related to these pixels that contribute significantly within a small range, should not significantly change the confidence level / prediction result of the AI ​​model. Therefore, by comparing the difference between the first and second confidence levels, we can determine whether the image to be processed contains a trigger.

[0101] Since both the first and second confidence levels are represented by vectors, any existing method for obtaining the difference between the two vectors can be used in this application. For example, the KL divergence values ​​of the first and second confidence levels can be calculated, and the difference between the KL divergence value and a preset threshold can be used to determine whether the image to be processed contains a trigger. Specifically, if the KL divergence value exceeds the preset threshold, the image to be processed is considered to contain a trigger; if the KL divergence value does not exceed the preset threshold, the image to be processed is considered not to contain a trigger. Alternatively, the difference between the first and second confidence levels can be obtained through mutual information. Furthermore, when the first confidence level is the prediction result, the presence or absence of a trigger in the image to be processed can be visually determined by observing whether the prediction result changes. Specifically, if the model's prediction result for the preprocessed image to be processed is different from the model's prediction result for the image to be processed, then the image to be processed is determined to contain a trigger; if the model's prediction result for the preprocessed image to be processed is the same as the model's prediction result for the image to be processed, then the image to be processed is determined not to contain a trigger.

[0102] Furthermore, the preset threshold can be obtained in various ways. For example, in one possible implementation, the preset threshold can be determined empirically. Another possible implementation is that the preset threshold can be determined experimentally. In one possible implementation, the pixel values ​​of the high-contribution pixels in each original sample of the clean dataset can be subjected to at least one destruction process. Taking one destruction process as an example, the destroyed sample is called the processed sample. Each processed sample is input into the model to obtain a confidence score. The difference between the confidence score obtained by the model for the processed sample and the confidence score obtained by the model for the original sample can be obtained. That is, for each original sample, a difference can be obtained, and by statistically analyzing the differences of all original samples, the preset threshold can be determined based on the maximum value of these differences. Similarly, if the pixel values ​​of the high-contribution pixels in each original sample are subjected to multiple destruction processes, multiple differences can be obtained for each original sample. By statistically analyzing the differences of all original samples, the preset threshold can be determined based on the maximum value of these differences.

[0103] 2. Determine whether the image to be processed contains a trigger based on the difference between the two first confidence levels.

[0104] Step 604 describes performing two preprocessing steps on the image to be processed to obtain two preprocessed images. These two preprocessed images are then used as inputs to the model to obtain two first confidence scores. In one possible implementation, the two first confidence scores can be compared with a second confidence score to obtain the difference between the two first confidence scores and the second confidence score. If the difference between the two first confidence scores and the second confidence score does not exceed a preset threshold, the image to be processed is considered not to contain a trigger; if the difference between one of the two first confidence scores and the second confidence score exceeds the preset threshold, the image to be processed is considered to contain a trigger. How to obtain the difference between the two confidence scores and the method for determining the preset threshold have been described in the above embodiments and will not be repeated here. Furthermore, in another possible implementation, the difference between the two first confidence scores can also be obtained, and the presence or absence of a trigger in the image to be processed can be determined based on this difference. Specifically, if the difference between the two first confidence levels exceeds a preset threshold, the image to be processed is considered to contain a trigger; if the difference does not exceed the preset threshold, the image to be processed is considered not to contain a trigger. The rate of change of confidence can be used as a theoretical basis for determining whether the image to be processed contains a trigger. For example, determining whether the image to be processed contains a trigger based on the difference between the first and second confidence levels is based on the rate of change of confidence. Determining whether the image to be processed contains a trigger based on the difference between the two first confidence levels is based on the rate of change of confidence. Another example is determining whether the image to be processed contains a trigger based on the ratio of a first value and a second value, where the first value includes the difference between two differences, one referring to the difference between one first confidence level and the second confidence level, and the other referring to the difference between another first confidence level and the second confidence level. The second value refers to the difference between the first and second confidence levels.

[0105] Third, determine whether the image to be processed contains a trigger based on the differences between multiple first confidence levels and second confidence levels.

[0106] Step 604 describes the possibility of performing preprocessing on the image to be processed more than twice to obtain more than two preprocessed images. These preprocessed images are then used as input to the model to obtain more than two first confidence scores. In this implementation, the difference between each first confidence score and its second confidence score can be compared. If the difference between each first confidence score and its second confidence score does not exceed a preset threshold, the image to be processed is considered not to contain a trigger. The theoretical basis of this implementation is that a maximum number of preprocessing operations on the image to be processed can be preset, for example, four times. This results in four preprocessed images and four first confidence scores. If the difference between each first confidence score and its second confidence score does not exceed the preset threshold, it is considered that disrupting the key features of the image to be processed does not affect the model's confidence in the image, and therefore, the image to be processed is considered not to contain a trigger. The advantage of this implementation is that by performing multiple preprocessing operations on the image to be processed, the number of pixels selected for replacement during preprocessing is gradually increased, which solves the problem of not being able to determine the trigger size well. In addition, a maximum number of preprocessing operations can be set to avoid wasting computing power.

[0107] To more intuitively demonstrate the solution provided in the embodiments of this application, the following is combined with... Figure 7 The solutions provided in the embodiments of this application are summarized. For example... Figure 7 As shown, image x is the object to be detected. Image x and the model are used as inputs to the image processing method provided in this embodiment. The flow of the image processing method can be referred to... Figure 6 The process is described in the corresponding example. First, the key features of x in the model are obtained, i.e., the contribution of each pixel in image x to the prediction result. Then, the key features are removed to form a set {xi|i∈[0,n]}, which is used to preprocess the image to be processed based on the contribution of each pixel to the prediction result. Each image xi in the set is a preprocessed image to be processed. Finally, the confidence score yi of each xi is obtained, and the confidence score of xi is used to determine whether x includes a trigger.

[0108] Through the above combination Figure 6 Figure 7As described in the embodiments of this application, this application provides a method for determining whether an image contains a trigger, which differs from existing technologies, thereby improving the security of artificial intelligence models. Furthermore, this application does not require a clean dataset, or only a small amount of clean dataset is needed to determine whether an image contains a trigger, reducing the cost of trigger determination. In addition, this application can determine whether an image contains a trigger with minimal computation (at least one preprocessing step and one confidence difference determination), reducing the computational power required for trigger determination and improving the efficiency of trigger determination.

[0109] The following section provides a further description of the solutions provided in the embodiments of this application, using several typical application scenarios as examples:

[0110] 1. In the scenario of dataset cleaning, there are no restrictions on the data distribution of the dataset or the proportion of clean data in the dataset.

[0111] The above text Figure 6 , Figure 7 The corresponding embodiments described all use a single image to be processed as an example. In reality, there may be multiple images to be processed. For instance, in a dataset cleaning scenario, the training dataset is cleaned, and images containing triggers are filtered out. In the case of multiple images to be processed, it is possible to determine whether each image contains a trigger one by one, i.e., refer to... Figure 6 as well as Figure 7 The corresponding embodiments can be understood. Furthermore, multiple images to be processed can be judged together to simultaneously determine which of the multiple images do not contain a trigger and which do contain one.

[0112] The following is combined Figure 8 Another method for determining triggers in scenarios involving multiple images to be processed is presented. In this implementation, it is possible to simultaneously determine which of the multiple images to be processed do not contain triggers and which do contain triggers. For example... Figure 8 As shown, for each sample image x in the dataset to be cleaned, a first confidence score is obtained. The process of obtaining the first confidence score for each sample image x can be referred to... Figure 6The corresponding examples described in steps 601 to 604 can be used for understanding. After obtaining the first confidence score for each sample image in the dataset to be cleaned, outlier detection can be performed on these multiple first confidence scores. Based on the outlier detection results, it can be determined whether there are triggers in the dataset to be cleaned. Specifically, it can be determined which sample images in the dataset to be cleaned do not contain triggers and which do contain triggers. Performing outlier detection based on multiple first confidence scores and determining whether the multiple images to be processed contain triggers based on the outlier detection results can be seen as... Figure 6 Another refinement of step 605 in the corresponding embodiment. It should be noted that any outlier detection scheme can be used in this application embodiment, such as median absolute deviation (MAD), mean squared difference constant detection, or outlier detection based on the DBScan clustering method. This application embodiment does not limit the outlier detection scheme used.

[0113] After outlier detection is performed on the multiple first confidence levels, if outliers are found, the dataset to be cleaned can be considered to contain triggers; if no outliers are found, the dataset to be cleaned can be considered not to contain triggers. If it is determined that the dataset to be cleaned contains triggers, it can be further determined that the sample image contains the trigger based on the sample image corresponding to the first confidence level of the outlier. For example, suppose the dataset to be cleaned contains 100 sample images, namely sample image 1, sample image 2, sample image 3, sample image 4, ..., sample image 100. For each sample image, the following steps are performed... Figure 6 The corresponding examples, described in processes 601 to 604, obtain the first confidence level for each sample image. Assume the first confidence levels for sample images 1 to 100 are first confidence levels 1 to 100, respectively. Outlier detection is performed on these 100 first confidence levels. If the detection result indicates that there are no outliers among these 100 first confidence levels, it means that the dataset to be cleaned does not contain a trigger. If the detection result indicates that there are outliers among these 100 first confidence levels, it means that the dataset to be cleaned includes a trigger. Specifically, if the first confidence level t is an outlier, it means that the sample image t corresponding to the first confidence level t contains a trigger.

[0114] As can be seen, in this implementation, it is possible to determine at once which of multiple images to be processed do not contain triggers and which do contain triggers. This implementation method allows for rapid cleaning of datasets, and the data distribution of the dataset and the proportion of clean data in the dataset do not affect the implementation of this application's embodiments.

[0115] 2. In backdoor attack detection scenarios, a response can also be made if a backdoor input is detected.

[0116] In backdoor attack detection scenarios, through the above Figure 6 and Figure 7 The described implementation can determine whether the image to be processed contains a trigger. When a backdoor input is detected, that is, when it is determined that the image to be processed contains a trigger, the embodiments of this application can also respond.

[0117] Specifically, in one possible implementation, if a trigger is detected in the image to be processed, the model can refuse to make a prediction for that image. That is, in this implementation, if a backdoor input is detected, the model refuses to output a prediction.

[0118] In one possible implementation, if the image to be processed is found to contain a trigger based on a first confidence level, then the recognition result of the image to be processed is output based on the first confidence level. (Above) Figure 6 and Figure 7 The corresponding embodiment describes determining whether the image to be processed contains a trigger based on a first confidence level. If the image to be processed contains a trigger based on the first confidence level, for example, if the difference between the first and second confidence levels exceeds a preset threshold, then the image to be processed is considered to contain a trigger. The principle behind this is that if the image to be processed contains a trigger, then the second confidence level obtained by the model for the image to be processed is unreliable. After preprocessing the image to be processed, if the difference between the first and second confidence levels obtained by the model for the preprocessed image to be processed is large, it indicates that the trigger in the image to be processed has been destroyed during preprocessing, and the trigger is no longer sufficient to affect the prediction result of the preprocessed image to be processed. Therefore, it is considered that the model can make a correct prediction for the preprocessed image to be processed. At this time, the confidence level obtained by the model for the preprocessed image to be processed is the first confidence level, so the prediction result can be obtained based on the first confidence level.

[0119] 3. Visualize the shape of the trigger.

[0120] In this implementation method, if the above is used... Figure 6 and Figure 7 In a corresponding embodiment, if the image to be processed contains a trigger based on a first confidence level, a heatmap is output. The heatmap is used to indicate the score value of each pixel in the image to be processed.

[0121] The above text has explained why AI models make decisions based on score values. When the image to be processed contains a trigger based on the first confidence level, it means that the pixel of the trigger will have a very high score value. Therefore, the location of the trigger in the input is obtained based on the position of these pixels with very high scores in the input.

[0122] When different images to be processed all contain the same trigger, and the pixels of the same trigger have very high score values ​​in different images to be processed, then by combining the heat maps of these different images to be processed, the shape of the trigger can be better determined.

[0123] The foregoing has provided a detailed description of the image processing method provided in the embodiments of this application. Based on the aforementioned image processing method flow, the following describes the image processing apparatus provided in the embodiments of this application, which can be used to perform the aforementioned... Figure 6-8 The methods and steps in the text.

[0124] See Figure 9 This application provides a schematic diagram of the structure of an image processing device.

[0125] The image processing device includes:

[0126] The first acquisition module 901 is used to acquire the image to be processed.

[0127] The second acquisition module 902 is used to acquire the score values ​​of multiple pixels in the image to be processed acquired by the first acquisition module 901. The score value of the first pixel is acquired based on the mapping coefficient between the first pixel and the prediction result. The first pixel is any pixel in the image to be processed, and the prediction result is the prediction result of the target image recognition model for the image to be processed.

[0128] The perturbation module 903 is used to perturb the pixel values ​​of pixels whose score values ​​meet the first condition in order to obtain the processed image to be processed.

[0129] The prediction module 904 is used to take the processed image to be processed as input to the target image recognition model in order to obtain the recognition result.

[0130] In one possible implementation, the recognition result includes a first confidence level. The device also includes a judgment module 905 for determining whether the image to be processed contains a trigger based on the first confidence level.

[0131] In one possible implementation, the prediction module 904 is further configured to: use the image to be processed as input to the target image recognition model to obtain a second confidence level. The judgment module 905 is specifically configured to: if the difference between the first confidence level and the second confidence level exceeds a first threshold, determine that the image to be processed includes a trigger.

[0132] In one possible implementation, the first confidence level includes a third confidence level and a fourth confidence level. The third confidence level is obtained by using the first processed image to be processed as input to the target image recognition model, and the fourth confidence level is obtained by using the second processed image to be processed as input to the target image recognition model. The first processed image to be processed and the second processed image to be processed are obtained by perturbing the pixel values ​​of pixels whose score values ​​meet the first condition. The judgment module 905 is specifically used to: when the difference between the third confidence level and the fourth confidence level exceeds a second threshold, obtain that the image to be processed includes a trigger.

[0133] In one possible implementation, the prediction module 904 is further configured to: use the image to be processed as input to the target image recognition model to obtain a second confidence level. The judgment module 905 is specifically configured to: if the difference between each of the multiple first confidence levels and the second confidence level is less than a third threshold, then determine that the image to be processed does not contain a trigger. The multiple first confidence levels are obtained by using multiple processed images to be processed as input to the target image recognition model, respectively. The multiple processed images to be processed are obtained by applying different perturbations to the pixel values ​​of pixels whose score values ​​satisfy the first condition.

[0134] In one possible implementation, the prediction module 904 is further configured to: use the image to be processed as input to the target image recognition model to obtain a second confidence level. The judgment module 905 is specifically configured to: obtain a target value based on the difference between the first confidence level and the second confidence level of the first image to be processed. Outlier detection processing is performed on the target values ​​obtained based on each image to be processed. If outliers are included in all target values, images containing triggers are identified among the multiple images to be processed.

[0135] In one possible implementation, the perturbation module 903 is specifically used to: replace the pixel values ​​of the top N highest-scoring pixels with the first target value to obtain the first processed image to be processed, where N is a positive integer.

[0136] In one possible implementation, the perturbation module 903 is further configured to: replace the pixel values ​​of the top N+m pixels with the second target value to obtain the second processed image to be processed, where m is a positive integer.

[0137] In one possible implementation, the first target value or the second target value is the average pixel value, which is determined based on the average pixel value of the pixels that satisfy the first condition.

[0138] In one possible implementation, an output module 906 is also included, which is used to: if the image to be processed is found to contain a trigger according to a first confidence level, then output the recognition result of the image to be processed according to the first confidence level.

[0139] This application also provides an image processing device; please refer to [link / reference]. Figure 10 , Figure 10 This is a schematic diagram of the structure of an image processing apparatus provided in an embodiment of this application. The image processing apparatus 1000 may be equipped with... Figure 9 The image processing device described in the corresponding embodiment is used to implement Figures 6 to 8 The image processing device in the corresponding embodiment has the following functions. Specifically, the image processing device 1000 may vary considerably due to different configurations or performance, and may include one or more central processing units (CPUs) 1022 (e.g., one or more processors) and memory 1032, and one or more storage media 1030 (e.g., one or more mass storage devices) for storing application programs 1042 or data 1044.

[0140] The memory 1032 and storage medium 1030 can be temporary or persistent storage. In one embodiment, the memory 1032 is random access memory (RAM), which can directly exchange data with the central processing unit 1022. It is used to load data 1044 and application programs 1042 and / or operating system 1041 for direct execution and use by the central processing unit 1022, and typically serves as a temporary data storage medium for the operating system or other running programs. The program stored in the storage medium 1030 may include one or more modules (…). Figure 10 (not shown in the image), each module may include a series of instruction operations on the image processing device.

[0141] Furthermore, the central processing unit 1022 can be configured to communicate with the storage medium 1030 and execute a series of instructions stored in the storage medium 1030 on the image processing device 1000. In one possible implementation, the storage medium 1030 stores the aforementioned... Figures 6 to 8 The program instructions and data corresponding to the method steps shown in any of the embodiments.

[0142] The image processing device 1000 may also include one or more power supplies 1026, one or more wired or wireless network interfaces 1050, one or more input / output interfaces 1058, and / or one or more operating systems 1041, such as Windows Server™, Mac OS X™, Unix™, Linux™, FreeBSD™, etc.

[0143] In one possible implementation, the central processing unit 1022 is used to perform the aforementioned... Figures 6 to 8 The method steps performed by the image processing apparatus shown in any of the embodiments. The number of central processing units (CPUs) can be one or more; when the number of CPUs is multiple, as described above... Figure 5 As shown, after the data is divided into multiple data blocks, these multiple processors can process the multiple data blocks in parallel, thereby improving the solution efficiency.

[0144] This application embodiment also provides an image processing device, which may also be referred to as a digital processing chip or a chip. The chip includes a processing unit and a communication interface. The processing unit can obtain program instructions through the communication interface, and the program instructions are executed by the processing unit. The processing unit is used to perform the aforementioned... Figures 6-8 The method steps performed by the image processing apparatus shown in any of the embodiments.

[0145] More specifically, when the image processing device provided in this application embodiment is a chip, the chip specifically includes: a processing unit and a communication unit. The processing unit may be, for example, a processor, and the communication unit may be, for example, an input / output interface, pins, or circuits. The processing unit can execute computer execution instructions stored in the storage unit to cause the chip within the server to perform the aforementioned operations. Figures 6 to 8 The image processing method described in the illustrated embodiment.

[0146] Optionally, the aforementioned storage unit can be an in-chip storage unit, such as a register or cache. Alternatively, the storage unit can be an external storage unit located within the wireless access device, such as a read-only memory (ROM) or other types of static storage devices capable of storing static information and instructions, such as random access memory (RAM).

[0147] Specifically, the aforementioned processing unit or processor can be a central processing unit (CPU), a network processing unit (NPU), a graphics processing unit (GPU), a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor can be a microprocessor or any conventional processor.

[0148] The processor mentioned above can be a general-purpose central processing unit, a microprocessor, an ASIC, or one or more processors used to control the above. Figures 6-8 The method of program execution of integrated circuits.

[0149] This application also provides a computer-readable storage medium storing a program that, when run on a computer, causes the computer to perform the aforementioned actions. Figures 6-8 The steps in the method described in the illustrated embodiment.

[0150] This application also provides a computer program product that, when run on a computer, causes the computer to perform the aforementioned actions. Figures 6-8 The image processing apparatus in the method described in the illustrated embodiment.

[0151] This application also provides a digital processing chip. This digital processing chip integrates circuitry for implementing the aforementioned processor or processor functions, and one or more interfaces. When the digital processing chip integrates a memory, it can perform the method steps of any one or more of the foregoing embodiments. When the digital processing chip does not integrate a memory, it can be connected to an external memory via a communication interface. The digital processing chip implements the actions performed by the image processing device in the foregoing embodiments based on the program code stored in the external memory.

[0152] For details, please refer to Figure 11 , Figure 11This is a schematic diagram of a chip provided in an embodiment of this application. The chip can be represented as a neural network processor (NPU) 110. The NPU 110 is mounted as a coprocessor on the host CPU, and tasks are assigned by the host CPU. The core part of the NPU is the arithmetic circuit 1103, which is controlled by the controller 1104 to extract matrix data from the memory and perform multiplication operations.

[0153] In some implementations, the arithmetic circuit 1103 internally includes multiple processing engines (PEs). In some implementations, the arithmetic circuit 1103 is a two-dimensional pulsating array. The arithmetic circuit 1103 can also be a one-dimensional pulsating array or other electronic circuitry capable of performing mathematical operations such as multiplication and addition. In some implementations, the arithmetic circuit 1103 is a general-purpose matrix processor.

[0154] For example, suppose we have an input matrix A, a weight matrix B, and an output matrix C. The arithmetic circuit retrieves the corresponding data of matrix B from the weight memory 1102 and caches it in each PE of the arithmetic circuit. The arithmetic circuit retrieves the data of matrix A from the input memory 1101 and performs matrix operations with matrix B. The partial result or the final result of the obtained matrix is ​​stored in the accumulator 1108.

[0155] Unified memory 1106 is used to store input and output data. Weight data is directly transferred to weight memory 1102 via Direct Memory Access Controller (DMAC) 1105. Input data is also transferred to unified memory 1106 via DMAC.

[0156] The Bus Interface Unit (BIU) 1110 is used by the instruction fetch memory 1109 to fetch instructions from external memory, and also by the memory access controller 1105 to fetch the original data of the input matrix A or the weight matrix B from external memory.

[0157] The DMAC is mainly used to move input data from external memory DDR to unified memory 1106, or to move weight data to weight memory 1102, or to move input data to input memory 1101.

[0158] The vector computation unit 1107 includes multiple arithmetic processing units that, when needed, further process the output of the computation circuit, such as vector multiplication, vector addition, exponential operations, logarithmic operations, size comparisons, etc. It is mainly used for computation in non-convolutional / fully connected layers of neural networks, such as Batch Normalization, pixel-level summation, and upsampling of feature planes.

[0159] In some implementations, the vector computation unit 1107 can store the processed output vector in the unified memory 1106. For example, the vector computation unit 1107 can apply linear and / or nonlinear functions to the output of the computation circuit 1103, such as linear interpolation of feature planes extracted by convolutional layers, or, for example, accumulating a vector of values ​​to generate activation values. In some implementations, the vector computation unit 1107 generates normalized values, pixel-level summed values, or both. In some implementations, the processed output vector can be used as activation input to the computation circuit 1103, for example, for use in subsequent layers of the neural network.

[0160] The instruction fetch buffer 1109 connected to the controller 1104 is used to store the instructions used by the controller 1104; the unified memory 1106, input memory 1101, weight memory 1102, and instruction fetch buffer 1109 are all on-chip memories. External memory is proprietary to this NPU hardware architecture.

[0161] The operations of each layer in the recurrent neural network can be performed by the operation circuit 1103 or the vector calculation unit 1107.

[0162] The processor mentioned above can be a general-purpose central processing unit, a microprocessor, an ASIC, or one or more integrated circuits used to control the execution of a program in the first aspect of the method.

[0163] It should also be noted that the device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. In addition, in the device embodiment drawings provided in this application, the connection relationship between modules indicates that they have a communication connection, which can be implemented as one or more communication buses or signal lines.

[0164] It should also be noted that the device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. In addition, in the accompanying drawings of the device embodiments provided in this application, the connection relationship between modules indicates that they have a communication connection, which can be implemented as one or more communication buses or signal lines.

[0165] Through the above description of the embodiments, those skilled in the art can clearly understand that this application can be implemented by means of software plus necessary general-purpose hardware, or it can be implemented by special-purpose hardware including application-specific integrated circuits, special-purpose CPUs, special-purpose memory, special-purpose components, etc. Generally, any function performed by a computer program can be easily implemented by corresponding hardware, and the specific hardware structure used to implement the same function can also be diverse, such as analog circuits, digital circuits, or special-purpose circuits. However, for this application, software program implementation is more often the preferred implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a readable storage medium, such as a computer floppy disk, USB flash drive, mobile hard disk, read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0166] In the above embodiments, the implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, in the form of a computer program product.

[0167] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions may be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that a computer can store or a data storage device such as a server or data center that integrates one or more available media. The available medium may be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state disk (SSD)).

[0168] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a particular order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments described herein can be implemented in a sequence other than that illustrated or described herein. Furthermore, the terms “comprising” and “having,” and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0169] Finally, it should be noted that the above are merely specific embodiments of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. An image processing method, characterized in that, include: Obtain the image to be processed; The score values ​​of multiple pixels in the image to be processed are obtained. The score value of the first pixel is determined based on the mapping coefficient between the first pixel and the prediction result. The first pixel is any one of the multiple pixels. The prediction result is the prediction result of the target image recognition model for the image to be processed. The pixel values ​​of the pixels whose score values ​​satisfy the first condition are perturbed to obtain the processed image to be processed; wherein, the perturbing of the pixel values ​​of the pixels whose score values ​​satisfy the first condition to obtain the processed image to be processed includes: replacing all or part of the pixel values ​​of the top N pixels with the highest score values ​​with a target value to obtain the processed image to be processed, where N is a positive integer, and the replacement with the target value is used to destroy the pixel values ​​of all or part of the top N pixels with the highest score values; The processed image to be processed is used as input to the target image recognition model to obtain the recognition result, which includes whether the image to be processed contains a backdoor behavior trigger condition.

2. The method according to claim 1, characterized in that, The identification result includes a first confidence level, and the method further includes: Based on the first confidence level, determine whether the image to be processed contains a backdoor behavior trigger condition.

3. The method according to claim 2, characterized in that, The method further includes: The image to be processed is used as input to the target image recognition model to obtain a second confidence level; The step of determining whether the image to be processed contains a trigger based on the first confidence level includes: When the difference between the first confidence level and the second confidence level exceeds a first threshold, it is determined that the image to be processed contains a trigger.

4. The method according to claim 2, characterized in that, The first confidence level includes a third confidence level and a fourth confidence level. The third confidence level is obtained by using the first processed image to be processed as input to the target image recognition model, and the fourth confidence level is obtained by using the second processed image to be processed as input to the target image recognition model. The first processed image to be processed and the second processed image to be processed are obtained by applying different perturbations to the pixel values ​​of the pixels whose score values ​​satisfy the first condition. The step of determining whether the image to be processed contains a trigger based on the first confidence level includes: When the difference between the third confidence level and the fourth confidence level exceeds the second threshold, it is determined that the image to be processed contains a trigger.

5. The method according to claim 2, characterized in that, The method further includes: The image to be processed is used as input to the target image recognition model to obtain a second confidence level; The step of determining whether the image to be processed contains a trigger based on the first confidence level includes: When the difference between each of the first confidence scores and the second confidence scores is less than a third threshold, it is determined that the image to be processed does not contain a trigger. The multiple first confidence scores are obtained by using the multiple processed images to be processed as inputs to the target image recognition model. The multiple processed images to be processed are obtained by perturbing the pixel values ​​of the pixels whose score values ​​satisfy the first condition in different ways.

6. The method according to claim 2, characterized in that, The images to be processed include multiple images, and the method further includes: Each of the images to be processed is used as input to the target image recognition model to obtain multiple second confidence levels; Determining whether the image to be processed contains a trigger based on the first confidence level includes: The target value is obtained based on the difference between the first confidence level and the second confidence level of the first image to be processed, wherein the first image to be processed is any one of the plurality of images to be processed. Outlier detection processing is performed on the target values ​​obtained from each of the images to be processed; If there are outliers among all the target values, it is determined that there is an image to be processed that includes a trigger among the multiple images to be processed.

7. The method according to claim 1, characterized in that, The target value is the average of the pixel values ​​of all or some of the pixels among the top N pixels with the highest scores.

8. The method according to any one of claims 2 to 6, characterized in that, The method further includes: If the image to be processed is found to contain a trigger based on the first confidence level, then the recognition result of the image to be processed is output based on the first confidence level.

9. An image processing apparatus, characterized in that, include: The first acquisition module is used to acquire the image to be processed; The second acquisition module is used to acquire the score values ​​of multiple pixels in the image to be processed acquired by the first acquisition module. The score value of the first pixel is determined based on the mapping coefficient between the first pixel and the prediction result. The first pixel is any one of the multiple pixels. The prediction result is the prediction result of the target image recognition model for the image to be processed. A perturbation module is used to perturb the pixel values ​​of the pixels whose score values ​​meet the first condition to obtain the processed image to be processed; wherein, the perturbation module is specifically used to: replace all or part of the pixel values ​​of the top N pixels with the highest score values ​​with the target value to obtain the processed image to be processed, where N is a positive integer, and the replacement with the target value is used to destroy the pixel values ​​of all or part of the top N pixels with the highest score values; The prediction module is used to take the processed image to be processed as the input of the target image recognition model to obtain the recognition result, the recognition result including whether the image to be processed contains a backdoor behavior trigger condition.

10. The device according to claim 9, characterized in that, The identification result includes a first confidence level, and the device further includes a judgment module for: Based on the first confidence level, determine whether the image to be processed contains a backdoor behavior trigger condition.

11. The device according to claim 10, characterized in that, The prediction module is also used for: The image to be processed is used as input to the target image recognition model to obtain a second confidence level; The judgment module is specifically used to determine that the image to be processed includes a trigger when the difference between the first confidence level and the second confidence level exceeds a first threshold.

12. The device according to claim 10, characterized in that, The first confidence level includes a third confidence level and a fourth confidence level. The third confidence level is obtained by using the first-processed image to be processed as input to the target image recognition model, and the fourth confidence level is obtained by using the second-processed image to be processed as input to the target image recognition model. The first-processed image to be processed and the second-processed image to be processed are obtained by applying different perturbations to the pixel values ​​of the pixels whose score values ​​satisfy the first condition. The judgment module is specifically used for: When the difference between the third confidence level and the fourth confidence level exceeds the second threshold, it is determined that the image to be processed contains a trigger.

13. The device according to claim 10, characterized in that, The images to be processed include multiple images, and the prediction module is further used for: Each of the images to be processed is used as input to the target image recognition model to obtain multiple second confidence levels; The judgment module is specifically used for: When the difference between each of the first confidence scores and the second confidence scores is less than a third threshold, it is determined that the image to be processed does not contain a trigger. The multiple first confidence scores are obtained by using the multiple processed images to be processed as inputs to the target image recognition model. The multiple processed images to be processed are obtained by perturbing the pixel values ​​of the pixels whose score values ​​satisfy the first condition in different ways.

14. The device according to claim 10, characterized in that, The prediction module is also used for: The image to be processed is used as input to the target image recognition model to obtain a second confidence level; The judgment module is specifically used for: The target value is obtained based on the difference between the first confidence level and the second confidence level of the first image to be processed, wherein the first image to be processed is any one of the plurality of images to be processed. Outlier detection processing is performed on the target values ​​obtained from each of the images to be processed; If there are outliers among all the target values, it is determined that there is an image to be processed that includes a trigger among the multiple images to be processed.

15. The device according to claim 9, characterized in that, The target value is the average of the pixel values ​​of all or some of the pixels among the top N pixels with the highest scores.

16. The device according to any one of claims 10 to 12, characterized in that, It also includes an output module for: If the image to be processed is found to contain a trigger based on the first confidence level, then the recognition result of the image to be processed is output based on the first confidence level.

17. An image processing apparatus, characterized in that, The method includes one or more processors coupled to a memory storing a program that, when executed by the one or more processors, implements the method of any one of claims 1 to 8.

18. A computer-readable storage medium, characterized in that, The program, when executed by the processing unit, performs the method as described in any one of claims 1 to 8.

19. A computer program product comprising a computer program / instructions, characterized in that, When the computer program / instructions are executed by the processor, they implement the steps of the method as described in any one of claims 1 to 8.

20. A circuit system, characterized in that, The circuit system includes processing circuitry configured to perform the method as described in any one of claims 1 to 8.

Images